Tarantool 2.10.0-beta1
Released on 2021-08-20.
- Release: v. 2.10.0-beta1.
- Tag:
2.10.0-beta1-0-g7da4b14
.
2.10.0-beta1 is the beta version of the 2 release series. It introduces 28 new features and resolves 56 bugs since the 2.8 version. There can be bugs in less common areas. If you find any, feel free to report an issue on GitHub.
Notable changes are:
- Preliminary support of ARM64 CPU (GNU/Linux).
- Preliminary support of Apple M1 CPU (macOS).
- Consistent SQL type system.
- Lightning fast net.box module.
- Interactive transactions in binary protocol.
- Small tuples are now stored more efficiently.
Tarantool 2.x is backward compatible with Tarantool 1.10.x in the binary data layout, client-server protocol, and replication protocol.
Please
upgrade
using the box.schema.upgrade()
procedure to unlock all the new
features of the 2.x series.
Some changes are labeled as [Breaking change]. It means that the old behavior was considered error-prone and therefore changed to protect users from unintended mistakes. However, there is a small probability that someone can rely on the old behavior, and this label is to bring attention to the things that have been changed.
The
UUID
field type is now part of theSCALAR
field type (gh-6042).The
UUID
field type is now available in SQL. A new UUID can be generated using the new SQL built-in functionUUID()
(gh-5886).[Breaking change] The
timeout()
method of thenet.box
connection, marked deprecated more than four years ago (in1.7.4
), has been dropped. It negatively affected the performance of hotnet.box
methods, such ascall()
andselect()
if they were called without a specified timeout (gh-6242).Improved
net.box
performance by up to 70% by rewriting hot code paths in C (gh-6241).Introduced compact tuples that allow saving 4 bytes per tuple in case of small userdata (gh-5385).
Now streams and interactive transactions over streams are implemented in iproto. Every stream is associated with its ID, which is unique within one connection. All requests with the same non-zero stream ID belong to the same stream. All requests in the stream are processed synchronously. The next request will not start executing until the previous one is completed. If a request’s stream ID is
0
, it does not belong to any stream and is processed in the old way.In
net.box
, a stream is an object above the connection that has the same methods but allows executing requests sequentially. The ID is generated on the client side automatically. If a user writes his own connector and wants to use streams, they must transmit thestream_id
over the iproto protocol.The primary purpose of streams is transactions via iproto. As each stream can start a transaction, several transactions can be multiplexed over one connection. There are multiple ways to begin, commit, and rollback a transaction. One can do that using the appropriate stream methods,
call
,eval
, orexecute
with the SQL transaction syntax. Users can mix these methods. For example, one might start a transaction usingstream:begin()
, and commit it withstream:call('box.commit')
orstream:execute('COMMIT')
. If any request fails during the transaction, it will not affect the other requests in the transaction. If a disconnect occurs while there is an active transaction in the stream, that transaction will be rolled back if it hasn’t been committed before the connection failure.Added the new
memtx_allocator
option tobox.cfg{}
. It allows selecting an appropriate allocator for memtx tuples if necessary. The possible values aresystem
for the malloc allocator andsmall
for the default small allocator.Implemented the system allocator based on malloc. The slab allocator, which is used for tuple allocation, has a particular disadvantage—it is prone to unresolvable fragmentation on specific workloads (size migration). In this case, the user should be able to choose another allocator. The system allocator is based on the malloc function but restricted by the same quota as the slab allocator. The system allocator does not alloc all the memory at the start. Instead, it allocates memory as needed, checking that the quota is not exceeded (gh-5419).
- Introduced the
box.info.replication[n].downstream.lag
field to monitor the state of replication. This represents the lag between the moment when the main node writes a certain transaction to its WAL and the moment it receives an ack for this transaction from the replica (gh-5447). - Introduced
on_election
triggers. The triggers can be registered via thebox.ctl.on_election()
interface and run asynchronously each timebox.info.election
changes (gh-5819).
- Introduced support for
LJ_DUALNUM
mode inluajit-gdb.py
(gh-6224). - Introduced preliminary support of GNU/Linux ARM64 and MacOS M1.
In the scope of this activity, the following issues have been resolved:
- Introduced support for the full 64-bit range of lightuserdata values (gh-2712).
- Fixed memory remapping issue when the page left 47-bit segments (gh-2712).
- Fixed M1 architecture detection (gh-6065).
- Fixed variadic arguments handling in FFI on M1 (gh-6066).
- Fixed
table.move
misbehaviour in case of table reallocation (gh-6084). - Fixed Lua stack inconsistency when
xpcall
is called on ARM64 with the invalid second argument (gh-6093). - Fixed
BC_USETS
bytecode semantics for closed upvalues and gray strings. - Fixed side exit jump target patching with regard to the range values of a particular instruction (gh-6098).
- Fixed the issue on ARM64 where the current Lua coroutine wasn’t restored on the exception path (gh-6189).
- Introduced the new method
table.equals
. It compares two tables by value with respect to the__eq
metamethod.
- Introduced new hash types—
xxhash32
andxxhash64
—in the digest module. (gh-2003).
- Introduced
fiber_object:info()
to get info from a fiber. Works asrequire(fiber).info()
but only for one fiber. - Introduced
fiber_object:csw()
to getcsw
from a fiber. Also, nowcsw
(Context SWitch) of the new fiber is always equal to zero. Previously, it could be greater than zero (gh-5799). - Changed
fiber.info()
to hide backtraces of idle fibers (gh-4235).
The
log
module now supports symbolic representation of log levels. Now it is possible to specify levels the same way as in thebox.cfg{}
call (gh-5882).For example, instead of
require('log').cfg{level = 6}
it is possible to use
require('log').cfg{level = 'verbose'}
Descriptions of type mismatch error and inconsistent type error have become more informative (gh-6176).
Removed explicit cast from
BOOLEAN
to numeric types and vice versa (gh-4770).For example,
CAST(FALSE AS INTEGER)
was0
in version 2.8. Now it causes an error.Removed explicit cast from
VARBINARY
to numeric types and vice versa (gh-4772, gh-5852).Fixed a bug where a string that is not
NULL
-terminated could not be cast toBOOLEAN
, even if the conversion would be successful according to the rules.Now a numeric value can be cast to another numeric type only if the cast is precise. In addition, a
UUID
value cannot be implicitly cast toSTRING/VARBINARY
. Also, aSTRING/VARBINARY
value cannot be implicitly cast to aUUID
(gh-4470).Now any number can be compared to any other number, and values of any scalar type can be compared to any other value of the same type. A value of a non-numeric scalar type cannot be compared with a value of any other scalar type (gh-4230).
Removed SQL built-in functions from the
_func
system space (gh-6106).The function is now looked up first in SQL built-in functions and then in user-defined functions.
Fixed incorrect error message in case of misuse of the function setting the default value.
The
TYPEOF()
function withNULL
as an argument now returnsNULL
if the type cannot be determined from context. (gh-5956). Also,TYPEOF(-NaN)
will now returnDOUBLE
, andTYPEOF(map_column)
will now returnVARBINARY
instead ofmap
.For example,
SELECT TYPEOF(NULL)
wasBOOLEAN
in version 2.8. Now it isNULL
.Reworked the
SCALAR
andNUMBER
types in SQL. Removed the implicit cast fromSCALAR
to any other scalar type. Also, removed the implicit cast fromNUMBER
values to any other numeric type. It means that arithmetic and bitwise operations and concatenation are no longer allowed forSCALAR
andNUMBER
values. In addition, anySCALAR
value can now be compared with values of any other scalar type using theSCALAR
rules (gh-6221).The field type
DECIMAL
is now available in SQL. Added an implicit cast fromINTEGER
andDOUBLE
toDECIMAL
and vice versa.DECIMAL
can participate in arithmetic operations and comparisons with other defined numeric types (gh-4415).[Breaking change] The argument types of SQL built-in functions are now checked in most cases during parsing. In addition, the number of arguments is now always checked during parsing (gh-6105).
[Breaking change] for the
NUMBER
data type. Arithmetic (+ * - / % & | ~
) operations and bit-shift operations (>> <<
) are now illegal.For example,
SELECT number_column + 1
was legal in version 2.8. Now it causes an error.[Breaking change] for the
SCALAR
data type. Arithmetic (+ * - / % & | ~
) operations and bit-shift operations (>> <<
) are now illegal. Concatenation (||
) operations are now illegal. Values inSCALAR
columns now have data typeSCALAR
, not the value’s data type.For example,
TYPEOF(CAST(1 AS SCALAR))
wasINTEGER
in version 2.8. Now it isSCALAR
.[Breaking change]: Arithmetic operators must now have numeric operands. String operands are illegal.
For example,
SELECT 1 + '1'
was2
in version 2.8. Now it causes an error.[Breaking change] in operations on SCALAR columns. Since the type of a value no longer determines whether an operation is valid, comparisons and functions that require a specific type no longer work.
For example, table
T
has only one row with a scalar column containing'a'
.UPPER(scalar_column)
was'A'
in version 2.8. Now it causes an error.[Breaking change] for the
HEX()
function.STRING
arguments are no longer acceptable; onlyVARBINARY
arguments are allowed.For example,
HEX('a')
was'41'
in version 2.8. Now it causes an error.[Breaking change] for the
POSITION()
function.VARBINARY
arguments are no longer acceptable; onlySTRING
arguments are allowed.For example,
POSITION(X'41',X'41')
was1
in version 2.8. Now it causes an error.
- Previously csw (Context SWitch) of new fiber could be greater than 0, now it is always 0 (gh-5799).
- Set
FORCE_CONFIG=false
for luarocks config to allow loading project-side.rocks/config-5.1.lua
.
- Fedora 34 builds are now supported (gh-6074).
- Fedora 28 and 29 builds are no longer supported.
[Breaking change]
fiber.wakeup()
in Lua andfiber_wakeup()
in C became NOP on the currently running fiber. Previously they allowed ignoring the next yield or sleep, which resulted in unexpected erroneous wake-ups. Calling these functions right beforefiber.create()
in Lua orfiber_start()
in C could lead to a crash (in debug build) or undefined behaviour (in release build) (gh-6043).There was a single use case for the previous behaviour: rescheduling in the same event loop iteration, which is not the same as
fiber.sleep(0)
in Lua andfiber_sleep(0)
in C. It could be done in the following way:in C:
fiber_wakeup(fiber_self()); fiber_yield();
and in Lua:
fiber.self():wakeup() fiber.yield()
To get the same effect in C, one can now use
fiber_reschedule()
. In Lua, it is now impossible to reschedule the current fiber directly in the same event loop iteration. One can reschedule self through a second fiber, but it is strongly discouraged:-- do not use this code local self = fiber.self() fiber.new(function() self:wakeup() end) fiber.sleep(0)
Fixed memory leak on
box.on_commit()
andbox.on_rollback()
(gh-6025).fiber_join()
now checks if the argument is a joinable fiber. The absence of this check could lead to unpredictable results. Note that the change affects the C level only; in the Lua interface,fiber:join()
protection is already enabled.Now Tarantool yields when it scans
.xlog
files for the latest applied vclock and finds the right place to start recovering from. It means that the instance becomes responsive right after thebox.cfg
call even if an empty.xlog
was not created on the previous exit.This fix also prevents the relay from timing out when a freshly subscribed replica needs rows from the end of a relatively long (hundreds of MBs)
.xlog
file (gh-5979).The counter in
N rows processed
log messages no longer resets on each newly recoveredxlog
.Fixed wrong type specification when printing fiber state change. It could lead to negative fiber IDs in the logs (gh-5846).
For example,
main/-244760339/cartridge.failover.task I> Instance state changed
instead of proper
main/4050206957/cartridge.failover.task I> Instance state changed
Fiber IDs are now switched to monotonically increasing unsigned 8-byte integers, so there is no ID wrapping anymore. It allows detecting fiber precedence by ID (gh-5846).
Fixed a crash in JSON update on tuple/space, where the update included two or more operations that accessed fields in reversed order and these fields didn’t exist. Example:
box.tuple.new({1}):update({{'=', 4, 4}, {'=', 3, 3}})
(gh-6069).Fixed invalid results of the
json
module’sencode
function when it was used from the Lua garbage collector. For example, this could happen in functions used asffi.gc()
(gh-6050).Added a check for user input of the number of iproto threads: value must be greater than zero and less than or equal to 1000 (gh-6005).
Changing a listed address can no longer cause iproto threads to close the same socket several times.
Simultaneously updating a key in different transactions does not longer result in a MVCC crash (gh-6131).
Fixed a bug where memtx MVCC crashed during reading uncommitted DDL (gh-5515).
Fixed a bug where memtx MVCC crashed if an index was created in the transaction thread (gh-6137).
Fixed a MVCC segmentation fault that arose when updating the entire space concurrently (gh-5892).
Fixed a bug with failed assertion after a stress update of the same key (gh-6193).
Fixed a crash where
box.snapshot
could be called during an incomplete transaction (gh-6229).Fixed console client connection failure in case of request timeout (gh-6249).
Added a missing broadcast to
net.box.future:discard()
so that now fibers waiting for a request result wake up when the request is discarded (gh-6250).box.info.uuid
,box.info.cluster.uuid
, andtostring(decimal)
with any decimal number in Lua could sometimes return garbage if there were__gc
handlers in the user’s code (gh-6259).Fixed an error message that appeared in a particular case during MVCC operation (gh-6247).
Fixed a repeatable read violation after delete (gh-6206).
Fixed a bug where the MVCC engine didn’t track the
select{}
hash (gh-6040).Fixed a crash in MVCC after a drop of space with several indexes (gh-6274).
Fixed a bug where the GC could leave tuples in secondary indexes (gh-6234).
Disallow yields after DDL operations in MVCC mode. It fixes a crash that took place when several transactions referred to system spaces (gh-5998).
Fixed a bug in MVCC that happened on rollback after a DDL operation (gh-5998).
Fixed a bug where rollback resulted in unserializable behavior (gh-6325).
- Fixed the use after free in the relay thread when using elections (gh-6031).
- Fixed a possible crash when a synchronous transaction was followed by an asynchronous transaction right when its confirmation was being written (gh-6057).
- Fixed an error where a replica, while attempting to subscribe to a foreign
cluster with a different replicaset UUID, didn’t notice it is impossible
and instead became stuck in an infinite retry loop printing
a
TOO_EARLY_SUBSCRIBE
error (gh-6094). - Fixed an error where a replica, while attempting to join a cluster with exclusively read-only replicas available, just booted its own replicaset, instead of failing or retrying. Now it fails with an error about the other nodes being read-only so they can’t register the new replica (gh-5613).
- Fixed error reporting associated with transactions
received from remote instances via replication.
Any error raised while such a transaction was being applied was always reported as
Failed to write to disk
regardless of what really happened. Now the correct error is shown. For example,Out of memory
, orTransaction has been aborted by conflict
, and so on (gh-6027). - Fixed replication occasionally stopping with
ER_INVALID_MSGPACK
when the replica is under high load (gh-4040). - Fixed a cluster sometimes being unable to bootstrap if it contains
nodes with
election_mode
set tomanual
orvoter
(gh-6018). - Fixed a possible crash when
box.ctl.promote()
was called in a cluster with more than three instances. The crash happened in the debug build. In the release build, it could lead to undefined behaviour. It was likely to happen if a new node was added shortly before the promotion (gh-5430). - Fixed a rare error appearing when MVCC
(
box.cfg.memtx_use_mvcc_engine
) was enabled and more than one replica joined the cluster. The join could fail with the error"ER_TUPLE_FOUND: Duplicate key exists in unique index 'primary' in space '_cluster'"
. The same could happen at the bootstrap of a cluster having more than three nodes (gh-5601).
- Fixed a rare crash with leader election enabled (any mode except
off
), which could happen if a leader resigned from its role while another node was writing something elections-related to WAL. The crash was in the debug build, and in the release build it would lead to undefined behaviour (gh-6129). - Fixed an error where a new replica in a Raft cluster tried to join
from a follower instead of a leader and failed with the error
ER_READONLY
(gh-6127).
- Fixed optimization for single-char strings in the
IR_BUFPUT
assembly routine. - Fixed slots alignment in the
lj-stack
command output whenLJ_GC64
is enabled (gh-5876). - Fixed dummy frame unwinding in the
lj-stack
command. - Fixed detection of inconsistent renames even in the presence of sunk values (gh-4252, gh-5049, gh-5118).
- Fixed the VM register allocation order provided by LuaJIT frontend in case
of
BC_ISGE
andBC_ISGT
(gh-6227).
- Fixed a bug where multibyte characters broke
space:fselect()
output. - When an error occurs during encoding call results, the auxiliary lightuserdata value is not removed from the main Lua coroutine stack. Before the fix, it led to undefined behaviour during the next usage of this Lua coroutine (gh-4617).
- Fixed a Lua C API misuse when the error is raised during call results encoding in an unprotected coroutine and expected to be caught in a different, protected coroutine (gh-6248).
- Fixed a possible crash in case trigger removes itself. Fixed a possible crash in case someone destroys a trigger when it yields (gh-6266).
- User-defined functions can now return a
VARBINARY
result to SQL (gh-6024). - Fixed assert when a
DOUBLE
value greater than -1.0 and less than 0.0 is cast toINTEGER
andUNSIGNED
(gh-6225). - Removed spontaneous conversion from
INTEGER
toDOUBLE
in a field of theNUMBER
type (gh-5335). - All arithmetic operations can now accept numeric values only (gh-5756).
- Now
QUOTE()
returns the argument if the argument isDOUBLE
, which is the same behavior as with other numeric types. For types different from numeric, the function returns aSTRING
(gh-6239). - The
TRIM()
function now does not lose collation when executed with the keywordsBOTH
,LEADING
, orTRAILING
(gh-6299).