Tarantool 2.4.1

Core

• Possibility to create errors of a custom user type (gh-4398). Read more:
  • Error responses
  • MessagePack extensions (ERROR type)
  • box.error.new().
• Transparent marshalling error through net.box (gh-4398). Read more:
  • box.error.new()
  • MessagePack extensions (ERROR type).
• box_session_push(): a new public C API function. It takes const char * MessagePack and returns it to the client out of order, just like the Lua box.session.push() does (gh-4734). Read more: box_session_push().
• Introduce a new field type and a new index type: UUID (gh-4268, gh-2916). Read more:
  • UUID field type
  • mapping UUID field to index types
  • uuid.new()
• Add ability to encode/decode UUIDs to MsgPack (gh-4268, gh-2916). Read more: MessagePack extensions (UUID type).
• Introduce stacked diagnostic area: now each Lua table representing an error object features the .prev member and the :set_prev() method so that errors can be organized into lists. IProto protocol is extended with new command keys to support this feature as well (gh-1148). For details, refer to error_object.

Lua

• Added popen built-in module (gh-4031).

  The module provides popen implementation that is integrated with tarantool’s event loop (like built-in fio and socket modules).

  It support bidirectional communication with a process: the module can feed input to a process and capture its output. This way it allows to run streaming programs (like grep) and even work interactively with outside REPL (say, python -i).

  A key feature of the implementation is that it uses vfork() under hood and so does not copy virtual memory tables. Copying of them may be quite time consuming: os.execute() takes ~2.5 seconds when 80 GiB is allocated for memtx. Moreover, when memory overcommit is disabled (which is default) it would be not possible to fork a process when more then half of available physical memory is mapped to tarantool’s process.

  The API should be considered as beta: it is quite basic and will be extended with convenience features. On the other hand, it may be changed in a backward-incompatible manner in the future releases if it will be valuable enough.

  For more details, refer to the popen module documentation.

• fio descriptors are closed on garbage collection (gh-4727). Read more in description of fio.open().

• fio.tempdir() uses the $TMPDIR environment variable as a path indication to create temporary directories (gh-4794).

• Expose lua_checktuple function (gh-2553).

• fiber.name maximal length is extended to 255 (gh-4394).

• Add tarantoolctl rocks commands: build, config, download, init, lint, new_version, purge, which, write_rockspec (gh-4629). Read more in Команды для управления модулями Tarantool.

Разное

• box.info.listen: a new variable in the box.info. Shows the real port when bound to the port 0. For example, if the listen parameter of box.cfg is set to 127.0.0.1:0, the box.info.listen shows 127.0.0.1:<real_port> (gh-4620). Read more: box.info.listen.
• sequence:current(): a new function to get the current sequence value without changing it (gh-4752). Read more: sequence_object:current().

Core

• fiber.storage is cleaned between requests, and can be used as a request-local storage. Previously fiber.storage could contain some old values in the beginning of an iproto request execution, and it needed to be nullified manually. Now the cleanup is unneeded (gh-4662).
• tuple/space/index:update()/upsert() were fixed not to turn a value into an infinity when a float value was added to or subtracted from a float column and exceeded the float value range (gh-4701).
• Fix potential execution abort when operating the system runs under heavy memory load (gh-4722).
• Make RTREE indexes handle the out of memory error: before this fix, OOM during the recovery of an RTREE index resulted in segmentation fault (gh-4619).
• Fix the error message returned on using an already dropped sequence (gh-4753).
• Add cancellation guard to avoid WAL thread stuck (gh-4127).
• Fix execution abort when memtx_memory and vinyl_memory are set to more than 4398046510080 bytes. Now an error message is returned (gh-4705).
• box.error.new() does not add a created error to the Tarantool’s diagnostic area anymore (gh-4778). Read more:
  • box.error.new()
  • box.error.set().
• Add Lua output format support for box.session.push() (gh-4686).

Репликация

• Fix rebootstrap procedure not working in case replica itself is listed in box.cfg.replication (gh-4759).
• Fix possible user password leaking via replication logs (gh-4493).
• Refactor vclock map to be exactly 4 bytes in size to fit all 32 replicas regardless of the compiler used (see in this commit).
• Fix crash when the replication applier rollbacks a transaction (gh-4730, gh-4776).
• Fix segmentation fault on master side when one of the replicas transitions from anonymous to normal (gh-4731).
• Local space operations are now counted in 0th vclock component. Every instance may have its own 0-th vclock component not matching others’. Local space operations are not replicated at all, even as NOPs (gh-4114).
• Gc consumers are now ordered by their vclocks and not by vclock signatures. Only the WALS that contain no entries needed by any of the consumers are deleted (gh-4114).

Lua

• json: :decode() does not spoil instance’s options with per-call ones (when it is called with the second argument) (gh-4761).
• Handle empty input for uri.format() properly (gh-4779).
• os.environ() is now changed when os.setenv() is called (gh-4733).
• netbox.self:call/eval() now returns the same types as netbox_connection:call/eval. Previously it could return a tuple or box.error cdata (gh-4513).
• box.tuple.* namespace is cleaned up from private functions. box.tuple.is() description is added to documentation (gh-4684).
• tarantoolctl rocks search: fix the --all flag (gh-4529).
• tarantoolctl rocks remove: fix the --force flag (gh-3632).
• libev: backport fix for listening for more then 1024 file descriptors on Mac OS (gh-3867).

SQL

• Fix box.stat() behavior: now it collects statistics on the PREPARE and EXECUTE methods as expected (gh-4756).
• Add ability to drop any table constraint using the following statement: ALTER TABLE  <table_name> DROP CONSTRAINT <constraint_name>. Previously, it was possible to drop only foreign key constraints with such a statement (gh-4120). Read more in Alter Table.
• “No such constraint” error now contains the name of the table this constraint belongs to.
• Add an empty body to the UNPREPARE IProto response (gh-4769).
• Reset all the placeholders’ bound values after execution of a prepared statement (gh-4825).
• The inserted values are inserted in the order in which they are given in case of INSERT into space with autoincrement (gh-4256).

Types related changes

• Rework the NUMBER type in SQL: now it completely matches with NoSQL definition. Integers inserted into a field of the NUMBER type are no longer forced to floating point representation; CAST AS NUMBER operation applied to an integer value doesn’t change it; CAST AS NUMBER operation applied to a boolean value now converts it into 1, 0 or NULL depending on particular boolean value (gh-4233). Read more in Data Type Conversion.
• Fix the CAST AS NUMBER operation applied to blob values: floating point representation of the result is no longer forced (gh-4463).
• Fix integer overflow error during addition of the integer and floating point values stored in the SCALAR column (gh-4369).
• Explicit and implicit cast from string containing floating point value to integer or unsigned types are disallowed.
• Maximum length of a blob value that is allowed to be cast to integer or unsigned types are limited to 12287 bytes.
• Fix wrong result of CAST() operator from blob to integer type in case a blob value does not have terminating ‘\0’ (gh-4766).

LuaJIT

• __pairs/__ipairs metamethods handling is removed since we faced the issues with the backward compatibility between Lua 5.1 and Lua 5.2 within Tarantool modules as well as other third party code (gh-4770).
• Introduce luajit-gdb.py extension with commands for inspecting LuaJIT internals. The extension obliges one to provide gdbinfo for libluajit, otherwise loading fails. The extension provides the following commands:
  • lj-arch dumps values of LJ_64 and LJ_GC64 macro definitions
  • lj-tv dumps the type and GCobj info related to the given TValue
  • lj-str dumps the contents of the given GCstr
  • lj-tab dumps the contents of the given GCtab
  • lj-stack dumps Lua stack of the given lua_State
  • lj-state shows current VM, GC and JIT states
  • lj-gc shows current GC stats
• Fix string to number conversion: current implementation respects the buffer length (gh-4773).
• “FFI sandwich” (*) detection is introduced. If sandwich is detected while trace recording the recording is aborted. The sandwich detected while mcode execution leads to the platform panic.
• luaJIT_setmode call is prohibited while mcode execution and leads to the platform panic.

(*) The following stack mix is called FFI sandwich: Lua-FFI -> C routine -> Lua-C API -> Lua VM.

This sort of re-entrancy is explicitly not supported by LuaJIT compiler. For more info see gh-4427.

Vinyl

• Fix assertion fault due to triggered dump process during secondary index build (gh-4810).

Разное

• Fix crashes at attempts to use -e and -l command line options concatenated with their values, like this: -eprint(100) (gh-4775).
• Fix inability to upgrade from 2.1 if there was an automatically generated sequence (gh-4771).
• Prettify the error message for user.grant(): no extra ’ ’ for universal privileges (gh-714).
• Update libopenssl version to 1.1.1f since the previous one was EOLed (gh-4830).