Tarantool 2.11 (LTS) | Tarantool
Releases Tarantool 2.11 (LTS)

Tarantool 2.11 (LTS)

Release date: May 24, 2023

Releases on GitHub: v. 2.11.4, v. 2.11.3, v. 2.11.2, v. 2.11.1, v. 2.11.0

The 2.11 release of Tarantool includes many new features and fixes. This document provides an overview of the most important features for the Enterprise and Community editions.

2.11 is the long-term support (LTS) release with two years of maintenance. This means that you will receive all the necessary security fixes and bug fixes throughout this period, and be able to get technical support afterward. You can learn more about the Tarantool release policy from the corresponding document.

Tarantool provides the live upgrade mechanism that enables cluster upgrade without downtime. In case of upgrade issues, you can roll back to the original state without downtime as well.

To learn how to upgrade to Tarantool 2.11, see Upgrades.

Tarantool Enterprise Edition now supports encrypted SSL/TLS private key files protected with a password. Given that most certificate authorities generate encrypted keys, this feature simplifies the maintenance of Tarantool instances.

A password can be provided using either the new ssl_password URI parameter or in a text file specified using ssl_password_file, for example:

box.cfg{ listen = {
    uri = 'localhost:3301',
    params = {
        transport = 'ssl',
        ssl_key_file = '/path_to_key_file',
        ssl_cert_file = '/path_to_cert_file',
        ssl_ciphers = 'HIGH:!aNULL',
        ssl_password = 'topsecret'
    }
}}

To learn more, see Traffic encryption.

With 2.11, Tarantool Enterprise Edition includes new security enforcement options. These options enable you to enforce the use of strong passwords, set up a maximum password age, and so on. For example, the password_min_length configuration option specifies the minimum number of characters for a password:

box.cfg{ password_min_length = 10 }

To specify the maximum period of time (in days) a user can use the same password, you can use the password_lifetime_days option, which uses the system clock under the hood:

box.cfg{ password_lifetime_days = 365 }

Note that by default, new options are not specified. You can learn more about all the available options from the Authentication restrictions and Password policy sections.

By default, Tarantool uses the CHAP protocol to authenticate users and applies SHA-1 hashing to passwords. In this case, password hashes are stored in the _user space unsalted. If an attacker gains access to the database, they may crack a password, for example, using a rainbow table.

With the Enterprise Edition, you can enable PAP authentication with the SHA256 hashing algorithm. For PAP, a password is salted with a user-unique salt before saving it in the database.

Given that PAP transmits a password as plain text, Tarantool requires configuring SSL/TLS. Then, you need to specify the box.cfg.auth_type option as follows:

box.cfg{ auth_type = 'pap-sha256' }

Learn more from the Authentication protocol section.

Starting with 2.11, Tarantool Enterprise Edition provides the ability to create read views - in-memory snapshots of the entire database that aren’t affected by future data modifications. Read views can be used to make complex analytical queries. This reduces the load on the main database and improves RPS for a single Tarantool instance.

Working with read views consists of three main steps:

  1. To create a read view, call the box.read_view.open() function:

    tarantool> read_view1 = box.read_view.open({name = 'read_view1'})
    
  2. After creating a read view, you can access database spaces and their indexes and get data using the familiar select and pairs data-retrieval operations, for example:

    tarantool> read_view1.space.bands:select({}, {limit = 4})
    ---
    - - [1, 'Roxette', 1986]
      - [2, 'Scorpions', 1965]
      - [3, 'Ace of Base', 1987]
      - [4, 'The Beatles', 1960]
    
  3. When a read view is no longer needed, close it using the read_view_object:close() method:

    tarantool> read_view1:close()
    

To learn more, see the Read views topic.

Tarantool Enterprise Edition now includes the zlib algorithm for tuple compression. This algorithm shows good performance in data decompression, which reduces CPU usage if the volume of read operations significantly exceeds the volume of write operations.

To use the new algorithm, set the compression option to zlib when formatting a space:

box.space.my_space:format{
    {name = 'id', type = 'unsigned'},
    {name = 'data', type = 'string', compression = 'zlib'},
}

The new compress module provides an API for compressing and decompressing arbitrary data strings using the same algorithms available for tuple compression:

compressor = require('compress.zlib').new()

data = compressor:compress('Hello world!') -- returns a binary string
compressor:decompress(data) -- returns 'Hello world!'

Tarantool can use a write-ahead log not only to maintain data persistence and replication. Another use case is implementing a CDC (Change Data Capture) utility that transforms a data replication stream and provides the ability to replicate data from Tarantool to an external storage.

Write-ahead log extensions

With 2.11, Tarantool Enterprise Edition provides WAL extensions that add auxiliary information to each write-ahead log record. For example, you can enable storing old and new tuples for each write-ahead log record. This is especially useful for the update operation because a write-ahead log record contains only a key value.

See the WAL extensions topic to learn how to enable and configure WAL extensions.

The 2.11 version provides the ability to downgrade a database to the specified Tarantool version using the box.schema.downgrade() method. This might be useful in the case of a failed upgrade.

To prepare a database for using it on an older Tarantool instance, call box.schema.downgrade and pass the desired Tarantool version:

tarantool> box.schema.downgrade('2.8.4')

To see Tarantool versions available for downgrade, call box.schema.downgrade_versions(). The earliest release available for downgrade is 2.8.2.

In previous Tarantool versions, the replication_connect_quorum option was used to specify the number of running nodes to start a replica set. This option was designed to simplify a replica set bootstrap. But in fact, this behavior brought some issues during a cluster lifetime and maintenance operations, for example:

  • Users who didn’t change this option encountered problems with the partial cluster bootstrap.
  • Users who changed the option encountered problems during the instance restart.

With 2.11, replication_connect_quorum is deprecated in favor of bootstrap_strategy. This option works during a replica set bootstrap and implies sensible default values for other parameters based on the replica set configuration. Currently, bootstrap_strategy accepts two values:

  • auto: a node doesn’t boot if half or more of the other nodes in a replica set are not connected. For example, if the replication parameter contains 2 or 3 nodes, a node requires 2 connected instances. In the case of 4 or 5 nodes, at least 3 connected instances are required. Moreover, a bootstrap leader fails to boot unless every connected node has chosen it as a bootstrap leader.
  • legacy: a node requires the replication_connect_quorum number of other nodes to be connected. This option is added to keep the compatibility with the current versions of Cartridge and might be removed in the future.

Starting with 2.11, if a fiber works too long without yielding control, you can use a fiber slice to limit its execution time. The fiber_slice_default compat option controls the default value of the maximum fiber slice. In future versions, this option will be set to true by default.

There are two slice types - a warning and an error slice:

  • When a warning slice is over, a warning message is logged, for example:

    fiber has not yielded for more than 0.500 seconds
    
  • When an error slice is over, the fiber is cancelled and the FiberSliceIsExceeded error is thrown:

    FiberSliceIsExceeded: fiber slice is exceeded
    

Note that these messages can point at issues in the existing application code. These issues can cause potential problems in production.

The fiber slice is checked by all functions operating on spaces and indexes, such as index_object.select(), space_object.replace(), and so on. You can also use the fiber.check_slice() function in application code to check whether the slice for the current fiber is over.

The example below shows how to use fiber.set_max_slice() to limit the slice for all fibers. fiber.check_slice() is called inside a long-running operation to determine whether a slice for the current fiber is over.

-- app.lua --
fiber = require('fiber')
clock = require('clock')

fiber.set_max_slice({warn = 1.5, err = 3})
time = clock.monotonic()
function long_operation()
    while clock.monotonic() - time < 5 do
        fiber.check_slice()
        -- Long-running operation ⌛⌛⌛ --
    end
end
long_operation_fiber = fiber.create(long_operation)

The output should look as follows:

$ tarantool app.lua
fiber has not yielded for more than 1.500 seconds
FiberSliceIsExceeded: fiber slice is exceeded

To learn more about fiber slices, see the Limit execution time section.

Tarantool 2.11 adds support for modules in the logging subsystem. You can configure different log levels for each module using the box.cfg.log_modules configuration option. The example below shows how to set the info level for module1 and the error level for module2:

tarantool> box.cfg{log_level = 'warn', log_modules = {module1 = 'info', module2 = 'error'}}
---
...

To create a log module, call the require('log').new() function:

tarantool> module1_log = require('log').new('module1')
---
...
tarantool> module2_log = require('log').new('module2')
---
...

Given that module1_log has the info logging level, calling module1_log.info shows a message but module1_log.debug is swallowed:

tarantool> module1_log.info('Hello from module1!')
2023-05-12 15:10:13.691 [39202] main/103/interactive/module1 I> Hello from module1!
---
...
tarantool> module1_log.debug('Hello from module1!')
---
...

Similarly, module2_log swallows all events with severities below the error level:

tarantool> module2_log.info('Hello from module2!')
---
...

The HTTP client now automatically serializes the content in a specific format when sending a request based on the specified Content-Type header and supports all the Tarantool built-in types. By default, the client uses the application/json content type and sends data serialized as JSON:

local http_client = require('http.client').new()
local uuid = require('uuid')
local datetime = require('datetime')

response = http_client:post('https://httpbin.org/anything', {
    user_uuid = uuid.new(),
    user_name = "John Smith",
    created_at = datetime.now()
})

The body for the request above might look like this:

{
    "user_uuid": "70ebc08d-2a9a-4ea7-baac-e9967dd45ac7",
    "user_name": "John Smith",
    "created_at": "2023-05-15T11:18:46.160910+0300"
}

To send data in a YAML or MsgPack format, set the Content-Type header explicitly to application/yaml or application/msgpack, for example:

response = http_client:post('https://httpbin.org/anything', {
    user_uuid = uuid.new(),
    user_name = "John Smith",
    created_at = datetime.now()
}, {
    headers = {
        ['Content-Type'] = 'application/yaml',
    }
})

You can now encode query and form parameters using the new params request option. In the example below, the requested URL is https://httpbin.org/get?page=1.

local http_client = require('http.client').new()

response = http_client:get('https://httpbin.org/get', {
    params = { page = 1 },
})

Similarly, you can send form parameters using the application/x-www-form-urlencoded type as follows:

local http_client = require('http.client').new()

response = http_client:post('https://httpbin.org/anything', nil, {
    params = { user_id = 1, user_name = 'John Smith' },
})

The HTTP client now supports chunked writing and reading of request and response data, respectively. The example below shows how to get chunks of a JSON response sequentially instead of waiting for the entire response:

local http_client = require('http.client').new()
local json = require('json')

local timeout = 1
local io = http_client:get(url, nil, {chunked = true})
for i = 1, 3 do
     local data = io:read('\r\n', timeout)
     if len(data) == 0 then
         -- End of the response.
         break
     end
     local decoded = json.decode(data)
     -- <..process decoded data..>
end
io:finish(timeout)

Streaming can also be useful to upload a large file to a server or to subscribe to changes in etcd using the gRPC-JSON gateway. The example below demonstrates communication with the etcd stream interface. The request data is written line-by-line, and each line represents an etcd command.

local http_client = require('http.client').new()

local io = http_client:post('http://localhost:2379/v3/watch', nil, {chunked = true})
io:write('{"create_request":{"key":"Zm9v"}}')
local res = io:read('\n')
print(res)
-- <..you can feed more commands here..>
io:finish()

Linearizability of read operations implies that if a response for a write request arrived earlier than a read request was made, this read request should return the results of the write request. Tarantool 2.11 introduces the new linearizable isolation level for box.begin():

box.begin({txn_isolation = 'linearizable', timeout = 10})
box.space.my_space:select({1})
box.commit()

When called with linearizable, box.begin() yields until the instance receives enough data from remote peers to be sure that the transaction is linearizable.

There are several prerequisites for linearizable transactions:

  • Linearizable transactions may only perform requests to synchronous, local, or temporary memtx spaces.
  • Starting a linearizable transaction requires box.cfg.memtx_use_mvcc_engine to be set to true.
  • The node is the replication source for at least N - Q + 1 remote replicas. Here N is the count of registered nodes in the cluster and Q is replication_synchro_quorum. So, for example, you can’t perform a linearizable transaction on anonymous replicas.

Tarantool is primarily designed for OLTP workloads. This means that data reads are supposed to be relatively small. However, a suboptimal SQL query can cause a heavy load on a database.

The new sql_seq_scan session setting is added to explicitly cancel full table scanning. The request below should fail with the Scanning is not allowed for 'T' error:

SET SESSION "sql_seq_scan" = false;
SELECT a FROM t WHERE a + 1 > 10;

To enable table scanning explicitly, use the new SEQSCAN keyword:

SET SESSION "sql_seq_scan" = false;
SELECT a FROM SEQSCAN t WHERE a + 1 > 10;

In future versions, SEQSCAN will be required for scanning queries with the ability to disable the check using the sql_seq_scan session setting. The new behavior can be enabled using a corresponding compat option.

Leader election is implemented in Tarantool as a modification of the Raft algorithm. The 2.11 release adds the ability to specify the leader fencing mode that affects the leader election process.

Note

Currently, Cartridge does not support leader election using Raft.

You can control the fencing mode using the election_fencing_mode property, which accepts the following values:

  • In soft mode, a connection is considered dead if there are no responses for 4 * replication_timeout seconds both on the current leader and the followers.
  • In strict mode, a connection is considered dead if there are no responses for 2 * replication_timeout seconds on the current leader and 4 * replication_timeout seconds on the followers. This improves the chances that there is only one leader at any time.
Found what you were looking for?
Feedback