Version:

Reference / Rocks reference / Module membership
Reference / Rocks reference / Module membership

Module membership

Module membership

This module is a membership library for Tarantool based on a gossip protocol.

This library builds a mesh from multiple Tarantool instances. The mesh monitors itself, helps members discover everyone else in the group and get notified about their status changes with low latency. It is built upon the ideas from Consul or, more precisely, the SWIM algorithm.

The membership module works over UDP protocol and can operate even before the box.cfg initialization.

Member data structure

A member is represented by the table with the following fields:

  • uri (string) is a Uniform Resource Identifier.

  • status (string) is a string that takes one of the values below.

    • alive: a member that replies to ping-messages is alive and well.

    • suspect: if any member in the group cannot get a reply from any other member, the first member asks three other alive members to send a ping-message to the member in question. If there is no response, the latter becomes a suspect.

    • dead: a suspect becomes dead after a timeout.

    • left: a member gets the left status after executing the leave() function.

      Note

      The gossip protocol guarantees that every member in the group becomes aware of any status change in two communication cycles.

  • incarnation (number) is a value incremented every time the instance is becomes a suspect, dead, or updates its payload.

  • payload (table) is auxiliary data that can be used by various modules.

  • timestamp (number) is a value of fiber.time64() which:

    • corresponds to the last update of status or incarnation;
    • is always local;
    • does not depend on other members’ clock setting.

Below is an example of the table:

tarantool> membership.myself()
---
uri: localhost:33001
status: alive
incarnation: 1
payload:
    uuid: 2d00c500-2570-4019-bfcc-ab25e5096b73
timestamp: 1522427330993752
...

API reference

Below is a list of membership’s common, encryption, subscription functions, and options.

Name Use
Common functions
init(advertise_host, port) Initialize the membership module.
myself() Get the member data structure of the current instance.
get_member(uri) Get the member data structure for a given URI.
members() Obtain a table with all members known to the current instance.
pairs() Shorthand for pairs(membership.members()).
add_member(uri) Add a member to the group.
probe_uri(uri) Check if the member is in the group.
broadcast() Discover members in LAN by sending a UDP broadcast message.
set_payload(key, value) Update myself().payload and disseminate it.
leave() Gracefully leave the group.
is_encrypted() Check if encryption is enabled.
Encryption functions
set_encryption_key(key) Set the key for low-level message encryption.
get_encryption_key() Retrieve the encryption key in use.
Subscription functions
subscribe() Subscribe for the members table updates.
unsubscribe() Remove the subscription.
Options
PROTOCOL_PERIOD_SECONDS Direct ping period.
ACK_TIMEOUT_SECONDS ACK message wait time.
ANTI_ENTROPY_PERIOD_SECONDS Anti-entropy synchronization period.
SUSPECT_TIMEOUT_SECONDS Timeout to mark a suspect dead.
NUM_FAILURE_DETECTION_SUBGROUPS Number of members to ping a suspect indirectly.

Common functions:

membership.init(advertise_host, port)

Initialize the membership module. This binds a UDP socket to 0.0.0.0:<port>, sets the advertise_uri parameter to <advertise_host>:<port>, and incarnation to 1.

The init() function can be called several times, the old socket will be closed and a new one opened.

If the advertise_uri changes during the next init(), the old URI is considered DEAD. In order to leave the group gracefully, use the leave() function.

Parameters:
  • advertise_host (string) – a hostname or IP address to advertise to other members
  • port (number) – a UDP port to bind
Return:

true

Rtype:

boolean

Raises:

socket bind error

membership.myself()
Return:the member data structure of the current instance.
Rtype:table
membership.get_member(uri)
Parameters:
  • uri (string) – the given member’s advertise_uri
Return:

the member data structure of the instance with the given URI.

Rtype:

table

membership.members()

Obtain all members known to the current instance.

Editing this table has no effect.

Return:a table with URIs as keys and corresponding member data structures as values.
Rtype:table
membership.pairs()

A shorthand for pairs(membership.members()).

Return:Lua iterator

It can be used in the following way:

for uri, member in memberhip.pairs()
  -- do something
end
membership.add_member(uri)

Add a member with the given URI to the group and propagate this event to other members. Adding a member to a single instance is enough as everybody else in the group will receive the update with time. It does not matter who adds whom.

Parameters:
  • uri (string) – the advertise_uri of the member to add
Return:

true or nil in case of an error

Rtype:

boolean

Raises:

parse error if the URI cannot be parsed

membership.probe_uri(uri)

Send a message to a member to make sure it is in the group. If the member is alive but not in the group, it is added. If it already is in the group, nothing happens.

Parameters:
  • uri (string) – the advertise_uri of the member to ping
Return:

true if the member responds within 0.2 seconds, otherwise no response

Rtype:

boolean

Raises:

ping was not sent if the hostname could not be resolved

membership.broadcast()

Discover members in local network by sending a UDP broadcast message to all networks discovered by a getifaddrs() C call.

Return:true if broadcast was sent, false if getaddrinfo() fails.
Rtype:boolean
membership.set_payload(key, value)

Update myself().payload and disseminate it along with the member status.

Increments incarnation.

Parameters:
  • key (string) – a key to set in payload table
  • value – auxiliary data
Return:

true

Rtype:

boolean

membership.leave()

Gracefully leave the membership group. The node will be marked with the left status and no other members will ever try to reconnect it.

Return:true
Rtype:boolean
membership.is_encrypted()
Return:true if encryption is enabled, false otherwise.
Rtype:boolean

Encryption functions:

membership.set_encryption_key(key)

Set the key used for low-level message encryption. The key is either trimmed or padded automatically to be exactly 32 bytes. If the key value is nil, the encryption is disabled.

The encryption is handled by the crypto.cipher.aes256.cbc Tarantool module.

For proper communication, all members must be configured to use the same encryption key. Otherwise, members report either dead or non-decryptable in their status.

Parameters:
  • key (string) – encryption key
Return:

nil.

membership.get_encryption_key()

Retrieve the encryption key that is currently in use.

Return:encryption key or nil if the encryption is disabled.
Rtype:string

Subscription functions:

membership.subscribe()

Subscribe for updates in the members table.

Return:a fiber.cond object broadcasted whenever the members table changes.
Rtype:object
membership.unsubscribe(cond)

Remove subscription on cond obtained by the subscribe() function.

The cond’s validity is not checked.

Parameters:
  • cond – the fiber.cond object obtained from subscribe()
Return:

nil.

Below is a list of membership options. They can be set as follows:

options = require('membership.options')
options.<option> = <value>
options.PROTOCOL_PERIOD_SECONDS

Period of sending direct pings. Denoted as T' in the SWIM protocol.

options.ACK_TIMEOUT_SECONDS

Time to wait for ACK message after a ping. If a member is late to reply, the indirect ping algorithm is invoked.

options.ANTI_ENTROPY_PERIOD_SECONDS

Period to perform the anti-entropy synchronization algorithm of the SWIM protocol.

options.SUSPECT_TIMEOUT_SECONDS

Timeout to mark suspect members as dead.

options.NUM_FAILURE_DETECTION_SUBGROUPS

Number of members to try pinging a suspect indirectly. Denoted as k in the SWIM protocol.