Rocks reference

Rocks reference

Rocks reference

This chapter covers open and closed source Lua modules for Tarantool Enterprise included in the distribution as an offline rocks repository.

Open source modules:

  • vshard is an automatic sharding system for Tarantool DBMS instances;
  • ldecnumber is the decimal arithmetic library;
  • http is an on-board HTTP-server;
  • stat is a selection of useful monitoring metrics;
  • checks is a type checker of functional arguments;
  • luarapidxml is a fast XML parser;
  • avro-schema is an assembly of Apache Avro schema tools.

Closed source modules:

  • cluster is a high-level cluster management interface that contains several modules:
    • rpc implements remote procedure calls between cluster instances and allows roles running on some instances to interact with other roles on other instances.
    • service-registry implements inter-role interaction and allows different roles to interact with each other in the scope of one instance.
    • confapplier implements cluster-wide configuration validation and application via a two-phase commit.
    • auth manages authentication.
    • pool reuses Tarantool’s net.box connections.
    • admin implements administration functions.
  • membership is a mesh of instances based on gossip protocol;
  • oracle is an Oracle connector for Lua applications though which they can access Oracle databases.

Installing and using modules

To use a module, install the following:

  1. All the necessary third-party software packages (if any). See the module’s prerequisites for the list.

  2. The module itself on every Tarantool instance:

    $ tarantoolctl rocks install <module_name> [<module_version>]
    
  3. Consult the module’s API documentation:

    $ tarantoolctl rocks doc <module-name>
    

    Or read the module’s reference to get a grasp on how to use it:

Module membership

This module is a membership library for Tarantool based on a gossip protocol.

This library builds a mesh from multiple Tarantool instances. The mesh monitors itself, helps members discover everyone else in the group and get notified about their status changes with low latency. It is built upon the ideas from Consul or, more precisely, the SWIM algorithm.

The membership module works over UDP protocol and can operate even before the box.cfg initialization.

Member data structure

A member is represented by the table with the following fields:

  • uri (string) is a Uniform Resource Identifier.

  • status (string) is a string that takes one of the values below.

    • alive: a member that replies to ping-messages is alive and well.

    • suspect: if any member in the group cannot get a reply from any other member, the first member asks three other alive members to send a ping-message to the member in question. If there is no response, the latter becomes a suspect.

    • dead: a suspect becomes dead after a timeout.

    • left: a member gets the left status after executing the leave() function.

      Note

      The gossip protocol guarantees that every member in the group becomes aware of any status change in two communication cycles.

  • incarnation (number) is a value incremented every time the instance is becomes a suspect, dead, or updates its payload.

  • payload (table) is auxiliary data that can be used by various modules.

  • timestamp (number) is a value of fiber.time64() which:

    • corresponds to the last update of status or incarnation;
    • is always local;
    • does not depend on other members’ clock setting.

Below is an example of the table:

tarantool> membership.myself()
---
uri: localhost:33001
status: alive
incarnation: 1
payload:
    uuid: 2d00c500-2570-4019-bfcc-ab25e5096b73
timestamp: 1522427330993752
...

API reference

Below is a list of membership’s common, encryption, subscription functions, and options.

Name Use
Common functions
init(advertise_host, port) Initialize the membership module.
myself() Get the member data structure of the current instance.
get_member(uri) Get the member data structure for a given URI.
members() Obtain a table with all members known to the current instance.
pairs() Shorthand for pairs(membership.members()).
add_member(uri) Add a member to the group.
probe_uri(uri) Check if the member is in the group.
broadcast() Discover members in LAN by sending a UDP broadcast message.
set_payload(key, value) Update myself().payload and disseminate it.
leave() Gracefully leave the group.
is_encrypted() Check if encryption is enabled.
Encryption functions
set_encryption_key(key) Set the key for low-level message encryption.
get_encryption_key() Retrieve the encryption key in use.
Subscription functions
subscribe() Subscribe for the members table updates.
unsubscribe() Remove the subscription.
Options
PROTOCOL_PERIOD_SECONDS Direct ping period.
ACK_TIMEOUT_SECONDS ACK message wait time.
ANTI_ENTROPY_PERIOD_SECONDS Anti-entropy synchronization period.
SUSPECT_TIMEOUT_SECONDS Timeout to mark a suspect dead.
NUM_FAILURE_DETECTION_SUBGROUPS Number of members to ping a suspect indirectly.

Common functions:

membership.init(advertise_host, port)

Initialize the membership module. This binds a UDP socket to 0.0.0.0:<port>, sets the advertise_uri parameter to <advertise_host>:<port>, and incarnation to 1.

The init() function can be called several times, the old socket will be closed and a new one opened.

If the advertise_uri changes during the next init(), the old URI is considered DEAD. In order to leave the group gracefully, use the leave() function.

Parameters:
  • advertise_host (string) – a hostname or IP address to advertise to other members
  • port (number) – a UDP port to bind
Returns:

true

Return type:

boolean

Raises:

socket bind error

membership.myself()
Returns:the member data structure of the current instance.
Return type:table
membership.get_member(uri)
Parameters:uri (string) – the given member’s advertise_uri
Returns:the member data structure of the instance with the given URI.
Return type:table
membership.members()

Obtain all members known to the current instance.

Editing this table has no effect.

Returns:a table with URIs as keys and corresponding member data structures as values.
Return type:table
membership.pairs()

A shorthand for pairs(membership.members()).

Returns:Lua iterator

It can be used in the following way:

for uri, member in memberhip.pairs()
  -- do something
end
membership.add_member(uri)

Add a member with the given URI to the group and propagate this event to other members. Adding a member to a single instance is enough as everybody else in the group will receive the update with time. It does not matter who adds whom.

Parameters:uri (string) – the advertise_uri of the member to add
Returns:true or nil in case of an error
Return type:boolean
Raises:parse error if the URI cannot be parsed
membership.probe_uri(uri)

Send a message to a member to make sure it is in the group. If the member is alive but not in the group, it is added. If it already is in the group, nothing happens.

Parameters:uri (string) – the advertise_uri of the member to ping
Returns:true if the member responds within 0.2 seconds, otherwise no response
Return type:boolean
Raises:ping was not sent if the hostname could not be resolved
membership.broadcast()

Discover members in local network by sending a UDP broadcast message to all networks discovered by a getifaddrs() C call.

Returns:true if broadcast was sent, false if getaddrinfo() fails.
Return type:boolean
membership.set_payload(key, value)

Update myself().payload and disseminate it along with the member status.

Increments incarnation.

Parameters:
  • key (string) – a key to set in payload table
  • value – auxiliary data
Returns:

true

Return type:

boolean

membership.leave()

Gracefully leave the membership group. The node will be marked with the left status and no other members will ever try to reconnect it.

Returns:true
Return type:boolean
membership.is_encrypted()
Returns:true if encryption is enabled, false otherwise.
Return type:boolean

Encryption functions:

membership.set_encryption_key(key)

Set the key used for low-level message encryption. The key is either trimmed or padded automatically to be exactly 32 bytes. If the key value is nil, the encryption is disabled.

The encryption is handled by the crypto.cipher.aes256.cbc Tarantool module.

For proper communication, all members must be configured to use the same encryption key. Otherwise, members report either dead or non-decryptable in their status.

Parameters:key (string) – encryption key
Returns:nil.
membership.get_encryption_key()

Retrieve the encryption key that is currently in use.

Returns:encryption key or nil if the encryption is disabled.
Return type:string

Subscription functions:

membership.subscribe()

Subscribe for updates in the members table.

Returns:a fiber.cond object broadcasted whenever the members table changes.
Return type:object
membership.unsubscribe(cond)

Remove subscription on cond obtained by the subscribe() function.

The cond’s validity is not checked.

Parameters:cond – the fiber.cond object obtained from subscribe()
Returns:nil.

Below is a list of membership options. They can be set as follows:

options = require('membership.options')
options.<option> = <value>
options.PROTOCOL_PERIOD_SECONDS

Period of sending direct pings. Denoted as T' in the SWIM protocol.

options.ACK_TIMEOUT_SECONDS

Time to wait for ACK message after a ping. If a member is late to reply, the indirect ping algorithm is invoked.

options.ANTI_ENTROPY_PERIOD_SECONDS

Period to perform the anti-entropy synchronization algorithm of the SWIM protocol.

options.SUSPECT_TIMEOUT_SECONDS

Timeout to mark suspect members as dead.

options.NUM_FAILURE_DETECTION_SUBGROUPS

Number of members to try pinging a suspect indirectly. Denoted as k in the SWIM protocol.

Module vshard

The vshard module introduces the sharding feature that enables horizontal scaling in Tarantool.

For more information, see the vshard module documentation.

Module http

In addition to the out-of-the-box HTTP client, this module includes an HTTP server but the latter has to be installed as described in the installation section.

For more information, see the HTTP README file in Tarantool’s GitHub repository.

Module checks

This is a library that declares a checks() function and checkers table that allow to check the parameters passed to a Lua function in a fast and unobtrusive way.

For more information, see the checks README file in Tarantool’s GitHub repository.

Module oracle

The oracle module is an Oracle connector for Tarantool. With this package, Lua applications can send and receive data to and from Oracle databases.

The advantage of the Tarantool-Oracle integration is that anyone can handle all the tasks with Oracle DBMSs (control, manipulation, storage, access) with the same high-level language (Lua) and with minimal delay.

Prerequisites

The oracle module requires the Instant Client Package (ICP) from Oracle. To install it via CLI, say:

curl -O https://raw.githubusercontent.com/bumpx/oracle-instantclient/master/oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm
rpm -Uvh oracle-instantclient12.2-basic-12.2.0.1.0-1.x86_64.rpm

Or get the package at oracle.com and install it the same way.

Note

Please, note the version is 12.2.

Quick start

Start Tarantool in the interactive mode and do the following:

  1. Connect to an Oracle database:

    tarantool> ora = require('oracle')
    tarantool> c, errmsg = ora.connect({username='system', password='oracle', db='localhost:1511/myspace'})
    tarantool> if not c then error("Failed to connect: "..errmsg) end
    
  2. Create a table and put some data into it:

    tarantool> c:exec('CREATE TABLE test(i int, s varchar(20))')
    tarantool> c:exec('INSERT INTO test(i, s) VALUES(1, \'Hello!\')')
    
  3. Retrieve the result:

    tarantool> rc, result = c:exec('SELECT * FROM test')
    tarantool> result[1]['S']
    ---
    'Hello!'
    ...
    

API reference

Below is a list of oracle functions.

Name Use
:exec(‘sql’) Execute an SQL request or raise an error.
:version() Get version string.
:exec('sql')

Executes an SQL request or raises an error.

Parameters:sql – SQL statement with the appropriate syntax.
Returns:
  • 0 in case of success; additionally, data and its size in case it was a SELECT request.
  • 1 in case of an ORA-error and the corresponding error message.

Examples

tarantool> c:exec('CREATE TABLE test(i int, s varchar(20))')
tarantool> c:exec('INSERT INTO test(i, s) VALUES(1, \'Hello!\')')
tarantool> c:exec('SELECT * FROM test')
:version()
Returns:versions of both the connector and OCI.
Return type:string

Example

tarantool> c:version()
---
- 'version: 1.1.0, OCI version(builded): 12.2, OCI version(linked): ?'
...