Tarantool 2.10.0-beta2 | Tarantool
Check out the new release policy

Tarantool 2.10.0-beta2

Released on 2021-12-22.

2.10.0-beta2 is the beta version of the 2.10 release series.

This release introduces 64 new features and resolves 86 bugs since the 2.8 version. There can be bugs in less common areas. If you find any, feel free to report an issue on GitHub.

Notable changes are:

  • HTTP client now supports HTTP/2.
  • Preliminary support of new DATETIME type.
  • Improved type consistency in SQL.

Tarantool 2.x is backward compatible with Tarantool 1.10.x in the binary data layout, client-server protocol, and replication protocol.

Please upgrade using the box.schema.upgrade() procedure to unlock all the new features of the 2.x series.

Some changes below are labeled as [Breaking change]. It means that the old behavior was considered error-prone and therefore changed to protect users from unintended mistakes. However, there is a small probability that someone can rely on the old behavior, and this label is to bring attention to the things that have been changed.

  • Added msgpack.cfg.encode_error_as_ext configuration option to enable/disable encoding errors as MP_ERROR MsgPack extension. The option is enabled by default.
  • Removed box.session.setting.error_marshaling_enabled. Error marshaling is now enabled automatically if the connector supports it.
  • fiber.wakeup() in Lua and fiber_wakeup() in C became NOP on the currently running fiber.
  • The timeout() method of net.box connection was dropped.
  • The net.box console support was dropped.

  • The UUID field type is now part of the SCALAR field type (gh-6042).

  • The `UUID field type is now available in SQL. A new UUID can be generated using the new SQL built-in function UUID() (gh-5886).

  • [Breaking change] The timeout() method of the net.box connection, deprecated more than four years ago (in 1.7.4) has been dropped. It negatively affected the performance of hot net.box methods, such as call() and select() if they were called without a specified timeout (gh-6242).

  • Improved net.box performance by up to 70% by rewriting hot code paths in C (gh-6241).

  • Introduced compact tuples that allow saving 4 bytes per tuple in case of small userdata (gh-5385).

  • Now streams and interactive transactions over streams are implemented in iproto. Every stream is associated with its ID, which is unique within one connection. All requests with the same non-zero stream ID belong to the same stream. All requests in the stream are processed synchronously. The next request will not start executing until the previous one is completed. If a request’s stream ID is 0, it does not belong to any stream and is processed in the old way.

    In net.box, a stream is an object above the connection that has the same methods but allows executing requests sequentially. The ID is generated on the client side automatically. If a user writes his own connector and wants to use streams, they must transmit the stream_id over the iproto protocol.

    The primary purpose of streams is transactions via iproto. As each stream can start a transaction, several transactions can be multiplexed over one connection. There are multiple ways to begin, commit, and rollback a transaction. One can do that using the appropriate stream methods, call, eval, or execute with the SQL transaction syntax. Users can mix these methods. For example, one might start a transaction using stream:begin(), and commit it with stream:call('box.commit') or stream:execute('COMMIT'). If any request fails during the transaction, it will not affect the other requests in the transaction. If a disconnect occurs while there is an active transaction in the stream, that transaction will be rolled back if it hasn’t been committed before the connection failure.

  • Added the new memtx_allocator option to box.cfg{}. It allows selecting an appropriate allocator for memtx tuples if necessary. Possible values are system for the malloc allocator and small for the default small allocator.

    Implemented the system allocator based on malloc. The slab allocator, which is used for tuple allocation, has a particular disadvantage—it is prone to unresolvable fragmentation on specific workloads (size migration). In this case, the user should be able to choose another allocator. The system allocator is based on the malloc function but restricted by the same quota as the slab allocator. The system allocator does not alloc all the memory at the start. Instead, it allocates memory as needed, checking that the quota is not exceeded (gh-5419).

  • Added box.stat.net.thread() for reporting per thread net statistics (gh-6293).

  • Add the new metric STREAMS to box.stat.net, which contains statistics for iproto streams. STREAMS contains the same counters as the CONNECTIONS metric in box.stat.net: current, rps, and total (gh-6293).

  • Extended the network protocol (IPROTO) with a new request type (IPROTO_ID) that is supposed to be used for exchanging sets of supported features between server and client (gh-6253).

  • Added required_protocol_version and required_protocol_features to net.box connection options. The new options allow to specify the IPROTO protocol version and features that must be supported by the server for the connection to pass (gh-6253).

  • [Breaking change] Added msgpack.cfg.encode_error_as_ext configuration option to enable/disable encoding errors as MP_ERROR MsgPack extension. The option is enabled by default (gh-6433).

  • [Breaking change] Removed box.session.setting.error_marshaling_enabled. Error marshaling is now enabled automatically if the connector supports it (gh-6428).

  • Add new metrics REQUESTS_IN_PROGRESS and REQUESTS_IN_STREAM_QUEUE to box.stat.net, which contain detailed statistics for iproto requests. These metrics contain the same counters as other metrics in box.stat.net: current, rps, and total (gh-6293).

  • Implemented timeout for fiber:join in Lua (gh-6203).

  • Updated libev to version 4.33 (gh-4909).

  • Added the box.txn_id() function. It returns the ID of the current transaction if called within a transaction, nil otherwise.

  • Previously, if a yield occurred for a transaction that did not support it, Tarantool rolled back all its statements, but still processed its new statements (rolling them back with each yield). Also, the transaction was rolled back when a commit is attempted. Now Tarantool stopped processing any new statements right after the first yield if the transaction does not support it.

  • Implemented a timeout for transactions after which they are rolled back (gh-6177).

  • Implemented the new C API function box_txn_set_timeout to set timeout for transaction.

  • Implemented the new IPROTO_TIMEOUT 0x56 key. It is used to set a timeout for transactions over iproto streams. It is stored in the body of IPROTO_BEGIN request.

  • Introduced box.broadcast and box.watch functions to signal/watch user-defined state changes (gh-6257).

  • Added watchers support to the network protocol (gh-6257).

  • Added watchers support to the net.box connector (gh-6257).

  • Now error objects with the box.error.READONLY code have the additional fields explaining why the error has happened. Also, there is a new field box.info.ro_reason. It is nil on a writable instance, but reports a reason when the box.info.ro is true (gh-5568).

  • Implemented ability to open several listening sockets. In addition to ability to pass uri as a number or string, as previously, ability to pass uri as a table of numbers or strings has been added (gh-3554).

  • [Breaking change] net.box console support, marked deprecated in 1.10, has been dropped. Use require('console').connect() instead.

  • Added takes_raw_args Lua function option. It wraps the arguments in msgpack.object to skip the decoding (gh-3349).

  • Introduced the box.info.replication[n].downstream.lag field to monitor the state of replication. This represents the lag between the moment when the main node writes a certain transaction to its WAL and the moment it receives an ack for this transaction from the replica (gh-5447).
  • Introduced on_election triggers. The triggers can be registered via the box.ctl.on_election() interface and run asynchronously each time box.info.election changes (gh-5819).

  • Introduced support for LJ_DUALNUM mode in luajit-gdb.py (gh-6224).

  • Introduced preliminary support of GNU/Linux ARM64 and MacOS M1. In the scope of this activity, the following issues have been resolved:

    • Introduced support for the full 64-bit range of lightuserdata values (gh-2712).
    • Fixed memory remapping issue when the page left 47-bit segments (gh-2712).
    • Fixed M1 architecture detection (gh-6065).
    • Fixed variadic arguments handling in FFI on M1 (gh-6066).
    • Fixed table.move misbehaviour in case of table reallocation (gh-6084).
    • Fixed Lua stack inconsistency when xpcall is called on ARM64 with the invalid second argument (gh-6093).
    • Fixed BC_USETS bytecode semantics for closed upvalues and gray strings.
    • Fixed side exit jump target patching with regard to the range values of a particular instruction (gh-6098).
    • Fixed the issue on ARM64 where the current Lua coroutine wasn’t restored on the exceptional path (gh-6189).
  • Now the memory profiler records allocations from traces, grouping them by the trace number (gh-5814). The memory profiler parser can display the new type of allocation sources in the following format:

    | TRACE [<trace-no>] <trace-addr> started at @<sym-chunk>:<sym-line>

  • Introduced the new method table.equals. It compares two tables by value with respect to the __eq metamethod.

  • Added a new built-in module—datetime.lua. It allows operating timestamps and intervals values (gh-5941).

  • Introduced new hash types—xxhash32 and xxhash64—in the digest module` (gh-2003).

  • Introduced fiber_object:info() to get info from a fiber. Works as require(fiber).info() but only for one fiber.
  • Introduced fiber_object:csw() to get csw from a fiber (gh-5799).
  • Changed fiber.info() to hide backtraces of idle fibers (gh-4235).
  • Improved fiber fiber.self(), fiber.id() and fiber.find() performance by 2-3 times.

  • The log module now supports symbolic representation of log levels. Now it is possible to specify levels the same way as in the box.cfg{} call (gh-5882).

    For example, instead of

    require('log').cfg{level = 6}

    it is possible to use

    require('log').cfg{level = 'verbose'}

  • Added the return_raw net.box option for returning msgpack.object instead of decoding the response (gh-4861).

  • Now is_multikey option can be passed to box.schema.func.create directly, without opts sub-table.

  • Descriptions of type mismatch error and inconsistent type error have become more informative (gh-6176).

  • Removed explicit cast from BOOLEAN to numeric types and vice versa (gh-4770).

    For example, CAST(FALSE AS INTEGER) was 0 in version 2.8. Now it causes an error.

  • Removed explicit cast from VARBINARY to numeric types and vice versa (gh-4772, gh-5852).

  • Fixed a bug where a string that is not NULL-terminated could not be cast to BOOLEAN, even if the conversion would be successful according to the rules.

  • Now a numeric value can be cast to another numeric type only if the cast is precise. In addition, a UUID value cannot be implicitly cast to STRING/VARBINARY. Also, a STRING/VARBINARY value cannot be implicitly cast to a UUID (gh-4470).

  • Now any number can be compared to any other number, and values of any scalar type can be compared to any other value of the same type. A value of a non-numeric scalar type cannot be compared with a value of any other scalar type (gh-4230).

  • Removed SQL built-in functions from the _func system space (gh-6106).

  • The function is now looked up first in SQL built-in functions and then in user-defined functions.

  • Fixed incorrect error message in case of misuse of the function setting the default value.

  • The TYPEOF() function with NULL as an argument now returns NULL (gh-5956).

  • Reworked the SCALAR and NUMBER types in SQL. Removed the implicit cast from SCALAR to any other scalar type. Also, removed the implicit cast from NUMBER values to any other numeric type. It means that arithmetic and bitwise operations and concatenation are no longer allowed for SCALAR and NUMBER values. In addition, any SCALAR value can now be compared with values of any other scalar type using the SCALAR rules (gh-6221).

  • The field type DECIMAL is now available in SQL. Added an implicit cast from INTEGER and DOUBLE to DECIMAL and vice versa. DECIMAL can participate in arithmetic operations and comparisons with other defined numeric types (gh-4415).

  • The argument types of SQL built-in functions are now checked in most cases during parsing. In addition, the number of arguments is now always checked during parsing (gh-6105).

  • Now DECIMAL values can be bound in SQL (gh-4717).

  • A value consisting of digits and a decimal point is now parsed as DECIMAL (gh-6456).

  • The field type ANY is now available in SQL (gh-3174).

  • Built-in SQL functions now work correctly with DECIMAL values (gh-6355).

  • A default type is now defined in case the argument type of a SQL built-in function cannot be determined during parsing (gh-4415).

  • The field type ARRAY is now available in SQL. The syntax has also been implemented to allow the creation of ARRAY values (gh-4762).

  • Previously, the csw (Context SWitch) of a new fiber could be greater than 0, now it is always 0 (gh-5799).

  • Set FORCE_CONFIG=false for luarocks config to allow loading project-side .rocks/config-5.1.lua.

  • Reduced snapshot verbosity (gh-6620).

  • Added bundling of libnghttp2 for bundled libcurl to support HTTP/2 for http client. The CMake version requirement is updated from 3.2 to 3.3.
  • Fedora-34 build is now supported (gh-6074).
  • Fedora 28 and 29 builds are no longer supported.
  • Stopped support of Ubuntu Trusty (14.04) (gh-6502).
  • Bumped debian package compatibility level to 10 (gh-5429).
  • Bumped minimal required debhelper to version 10 (except for Ubuntu Xenial).
  • Removed Windows binaries from debian source packages (gh-6390).
  • Bumped debian control Standards-Version to 4.5.1 (gh-6390).

  • [Breaking change] fiber.wakeup() in Lua and fiber_wakeup() in C became NOP on the currently running fiber. Previously they allowed ignoring the next yield or sleep, which resulted in unexpected erroneous wake-ups. Calling these functions right before fiber.create() in Lua or fiber_start() in C could lead to a crash (in debug build) or undefined behaviour (in release build) (gh-6043).

    There was a single use case for the previous behaviour: rescheduling in the same event loop iteration, which is not the same as fiber.sleep(0) in Lua and fiber_sleep(0) in C. It could be done in the following way:

    in C:


    and in Lua:


    To get the same effect in C, one can use fiber_reschedule(). In Lua, it is now impossible to reschedule the current fiber directly in the same event loop iteration. One can reschedule self through a second fiber, but it is strongly discouraged:

    -- do not use this code
    local self = fiber.self()
    fiber.new(function() self:wakeup() end)
  • Fixed memory leak on box.on_commit() and box.on_rollback() (gh-6025).

  • fiber_join() now checks if the argument is a joinable fiber. The absence of this check could lead to unpredictable results. Note that the change affects the C level only; in the Lua interface, fiber:join() protection is already enabled.

  • Now Tarantool yields when it scans .xlog files for the latest applied vclock and finds the right place to start recovering from. It means that the instance becomes responsive right after the box.cfg call even if an empty .xlog was not created on the previous exit.

    This fix also prevents the relay from timing out when a freshly subscribed replica needs rows from the end of a relatively long (hundreds of MBs) .xlog file (gh-5979).

  • The counter in N rows processed log messages no longer resets on each newly recovered xlog.

  • Fixed wrong type specification when printing fiber state change. It could lead to negative fiber IDs in the logs (gh-5846).

    For example,

    main/-244760339/cartridge.failover.task I> Instance state changed

    instead of proper

    main/4050206957/cartridge.failover.task I> Instance state changed
  • Fiber IDs are now switched switched to monotonically increasing unsigned 8-byte integers, so there is no ID wrapping anymore. It allows detecting fiber precedence by ID (gh-5846).

  • Fixed a crash in JSON update on tuple/space, where the update included two or more operations that accessed fields in reversed order and these fields didn’t exist. Example: box.tuple.new({1}):update({{'=', 4, 4}, {'=', 3, 3}}) (gh-6069).

  • Fixed invalid results of the json module’s encode function when it was used from the Lua garbage collector. For example, this could happen in functions used as ffi.gc() (gh-6050).

  • Added a check for user input of the number of iproto threads: value must be greater than zero and less than or equal to 1000 (gh-6005).

  • Fixed error related to the fact that if user changed listen address, all iproto threads closed same socket multiple times. Fixed error, related to the fact, that tarantool not deleting the unix socket path, when it’s finishing work.

  • Simultaneously updating a key in different transactions does not longer result in a MVCC crash (gh-6131).

  • Fix a bug when memtx MVCC crashed during reading uncommitted DDL (gh-5515).

  • Fixed a bug when memtx MVCC crashed if an index was created in the transaction (gh-6137).

  • Fixed a MVCC segmentation fault that arose when updating the entire space concurrently (gh-5892).

  • Fixed a bug with failed assertion after a stress update of the same key (gh-6193).

  • Fixed a crash where box.snapshot could be called during an incomplete transaction (gh-6229).

  • Fixed console client connection failure in case of request time out (gh-6249).

  • Added a missing broadcast to net.box.future:discard() so that now fibers waiting for a request result wake up when the request is discarded (gh-6250).

  • box.info.uuid, box.info.cluster.uuid, and tostring(decimal) with any decimal number in Lua could sometimes return garbage if there were __gc handlers in the user’s code (gh-6259).

  • Fixed an error message that appeared in a particular case during MVCC operation (gh-6247).

  • Fixed a repeatable read violation after delete (gh-6206).

  • Fixed a bug where the MVCC engine didn’t track the select{} hash (gh-6040).

  • Fixed a crash in MVCC after a drop of space with several indexes (gh-6274).

  • Fixed a bug when the GC could leave tuples in secondary indexes (gh-6234).

  • Disallowed yields after DDL operations in MVCC mode. It fixes a crash that took place when several transactions referred to system spaces (gh-5998).

  • Fixed a bug in MVCC connected that happened on rollback after a DDL operation (gh-5998).

  • Fixed a bug when rollback resulted in unserializable behaviour (gh-6325)

  • Previously, when a net.box connection was closed, all requests that had not been sent were discarded. This patch fixed this behavior: all requests queued for sending before the connection is closed are guaranteed to be sent (gh-6338).

  • Fixed a crash during replace of malformed tuple into _schema system space (gh-6332).

  • Fixed dropping incoming messages when connection is closed or SHUT_RDWR received and net_msg_max or readahead limit is reached (gh-6292).

  • Fixed memory leak in case of replace during background alter of primary index (gh-6290)

  • Fixed a bug when rollbacked changes appears in built-in-background index (gh-5958).

  • Fixed a crash while encoding an error object in the MsgPack format (gh-6431).

  • Fixed a bug when index was inconsistent after background build in case when the primary index is hash (gh-5977).

  • Now inserting a tuple with the wrong id field into the _priv space will return the correct error (gh-6295).

  • Fixed dirty read in MVCC after space alter (gh-6263, gh-6318).

  • Fixed crash in case a fiber changing box.cfg.listen is woken up (gh-6480).

  • Fixed box.cfg.listen not reverted to the old address in case the new one is invalid (gh-6092).

  • Fixed a crash caused by a race between box.session.push() and closing connection (gh-6520).

  • Fixed a bug because of which the garbage collector could remove an xlog file that is still in use (gh-6554).

  • Fixed a crash during granting privileges from guest (gh-5389).

  • Fixed an error in listening when user pass uri in numerical form after listening unix socket (gh-6535).

  • Now memtx raises an error if “clear” dictionary was passed to s:select() (gh-6167).

  • Fixed possible keys divergence during secondary index build, which might lead to missing tuples(:taranto ol-issue:6045).
  • Fixed the race between Vinyl garbage collection and compaction that resulted in broken vylog and recovery failure (gh-5436).
  • Immediate removal of compacted run files created after the last checkpoint optimization now works for replica’s initial JOIN stage (gh-6568).

  • Fixed the use after free in the relay thread when using elections (gh-6031).

  • Fixed a possible crash when a synchronous transaction was followed by an asynchronous transaction right when its confirmation was being written (gh-6057).

  • Fixed an error where a replica, while attempting to subscribe to a foreign cluster with a different replicaset UUID, didn’t notice it is impossible and instead became stuck in an infinite retry loop printing a TOO_EARLY_SUBSCRIBE error (gh-6094).

  • Fixed an error where a replica, while attempting to join a cluster with exclusively read-only replicas available, just booted its own replicaset, instead of failing or retrying. Now it fails with an error about the other nodes being read-only so they can’t register the new replica (gh-5613).

  • Fixed error reporting associated with transactions received from remote instances via replication. Any error raised while such a transaction was being applied was always reported as Failed to write to disk regardless of what really happened. Now the correct error is shown. For example, Out of memory, or Transaction has been aborted by conflict, and so on

  • Fixed replication occasionally stopping with ER_INVALID_MSGPACK when the replica is under high load (gh-4040).

  • Fixed a cluster sometimes being unable to bootstrap if it contains nodes with election_mode set to manual or voter (gh-6018).

  • Fixed a possible crash when box.ctl.promote() was called in a cluster with more than three instances. The crash happened in the debug build. In the release build, it could lead to undefined behaviour. It was likely to happen if a new node was added shortly before the promotion (gh-5430).

  • Fixed a rare error appearing when MVCC (box.cfg.memtx_use_mvcc_engine) was enabled and more than one replica joined to a cluster. The join could fail with the error "ER_TUPLE_FOUND: Duplicate key exists in unique index 'primary' in space '_cluster'". The same could happen at the bootstrap of a cluster having more than three nodes (gh-5601).

  • Fixed replica reconnecting to a living master on any box.cfg{replication=...} change. Such reconnects could result in replica failure to restore connection for replication_timeout seconds (gh-4669).

  • Fixed a rare crash with leader election enabled (any mode except off), which could happen if a leader resigned from its role while another node was writing something elections-related to WAL. The crash was in the debug build, and in the release build it would lead to undefined behaviour (gh-6129).
  • Fixed an error where a new replica in a Raft cluster tried to join from a follower instead of a leader and failed with the error ER_READONLY (gh-6127).

  • Fixed optimization for single-char strings in IR_BUFPUT assembly routine.
  • Fixed slots alignment in the lj-stack command output when LJ_GC64 is enabled (gh-5876).
  • Fixed dummy frame unwinding in the lj-stack command.
  • Fixed detection of inconsistent renames even in the presence of sunk values (gh-4252, gh-5049, gh-5118).
  • Fixed the VM register allocation order provided by LuaJIT frontend in case of BC_ISGE and BC_ISGT (gh-6227).
  • Fixed inconsistency while searching for an error function when unwinding a C protected frame to handle a runtime error (an error in __gc handler).

  • Fixed a bug when multibyte characters broke space:fselect() output.
  • When an error occurs during encoding call results, the auxiliary lightuserdata value is not removed from the main Lua coroutine stack. Before the fix, it led to undefined behaviour during the next usage of this Lua coroutine (gh-4617).
  • Fixed Lua C API misuse, when the error is raised during call results encoding in an unprotected coroutine and expected to be caught in a different, protected coroutine (gh-6248).
  • Fixed net.box error in case connections are frequently opened and closed (gh-6217).
  • Fixed incorrect handling of argument variable numbers in the box.func:call() (gh-6405).

  • Fixed a possible crash in case trigger removes itself. Fixed a possible crash in case someone destroys a trigger when it yields (gh-6266).

  • User-defined functions can now return a VARBINARY result to SQL (gh-6024).
  • Fixed assert when a DOUBLE value greater than -1.0 and less than 0.0 is cast to INTEGER and UNSIGNED (gh-6225).
  • Removed spontaneous conversion from INTEGER to DOUBLE in a field of NUMBER type(gh-5335).
  • All arithmetic operations can now accept numeric values only (gh-5756).
  • Now QUOTE() returns the argument if the argument is DOUBLE, which is the same behavior as with other numeric types. For types different from numeric, the function returns a STRING (gh-6239).
  • The TRIM() function now does not lose collation when executed with the keywords BOTH, LEADING, or TRAILING (gh-6299).
  • Getting unsupported msgpack extension in SQL now throws the correct error (gh-6375).
  • Now, when copying an empty string, an error will not be set unnecessarily (gh-6157, gh-6399).
  • Fixed wrong comparison between DECIMAL and large DOUBLE values (gh-6376).
  • Fixed truncation of DECIMAL during implicit cast to INTEGER in LIMIT and OFFSET.
  • Fixed truncation of DECIMAL during implicit cast to INTEGER when value is used in an index.
  • Fixed assert on cast of DECIMAL value that greater than -1.0 and less than 0.0 to INTEGER (gh-6485).
  • The HEX() SQL built-in function no longer throws an assert when its argument consists of zero-bytes (gh-6113).

  • Fixed log.cfg getting updated on box.cfg error (gh-6086).
  • Fixed error message on attempt to insert into a tuple which size equals to box.schema.FIELD_MAX (gh-6198).

  • Fixed MVCC interaction with ephemeral spaces: TX manager now ignores them (gh-6095).
  • Fixed loss of tuple after a conflict exception (gh-6132).
  • Fixed a segfault during update/delete of the same tuple (gh-6021).

  • Bumped debian packages tarantool-common dependency to use luarocks 3 (gh-5429).
  • Fixes an error when it was possible to have new Tarantool package (version >= 2.2.1) installed with pre-luarocks 3 tarantool-common package (version << 2.2.1), which caused rocks install to fail.
  • The Debian package does not depend on binutils anymore (gh-6699).
  • Fixed build errors with glibc-2.34 (gh-6686).