Data model

Lua versus MsgPack

Field Type Details

nil. In Lua a nil type has only one possible value, also called nil (which Tarantool displays as null when using the default YAML format). Nils may be compared to values of any types with == (is-equal) or ~= (is-not-equal), but other comparison operations will not work. Nils may not be used in Lua tables; the workaround is to use box.NULL because nil == box.NULL is true. Example: nil.

boolean. A boolean is either true or false. Example: true.

integer. The Tarantool integer type is for integers between -9223372036854775808 and 18446744073709551615, which is about 18 quintillion. This corresponds to number in Lua and to integer in MsgPack. Example: -2^63.

unsigned. The Tarantool unsigned type is for integers between 0 and 18446744073709551615,. So it is a subset of integer. Example: 123456.

double. The double field type exists mainly so that there will be an equivalent to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s interface to MsgPack) the storage type is MP_DOUBLE and the size of the encoded value is always 9 bytes. In Lua, ‘double’ fields can only contain non-integer numeric values and cdata values with double floating-point numbers. Examples: 1.234, -44, 1.447e+44.
To avoid using the wrong kind of values inadvertently, use ffi.cast() when searching or changing ‘double’ fields. For example, instead of space_object:insert{value} say ffi = require('ffi') ... space_object:insert({ffi.cast('double',value)}). Example:

s = box.schema.space.create('s', {format = {{'d', 'double'}}})
s:create_index('ii')
s:insert({1.1})
ffi = require('ffi')
s:insert({ffi.cast('double', 1)})
s:insert({ffi.cast('double', tonumber('123'))})
s:select(1.1)
s:select({ffi.cast('double', 1)})

Arithmetic with cdata ‘double’ will not work reliably, so for Lua it is better to use the ‘number’ type. This warning does not apply for Tarantool/SQL because Tarantool/SQL does implicit casting.

number. In Lua a number is double-precision floating-point, but a Tarantool ‘number’ field may have both integer and floating-point values. Tarantool will try to store a Lua number as floating-point if the value contains a decimal point or is very large (greater than 100 trillion = 1e14), otherwise Tarantool will store it as an integer. To ensure that even very large numbers are stored as integers, use the tonumber64 function, or the LL (Long Long) suffix, or the ULL (Unsigned Long Long) suffix. Here are examples of numbers using regular notation, exponential notation, the ULL suffix and the tonumber64 function: -55, -2.7e+20, 100000000000000ULL, tonumber64('18446744073709551615').

decimal. The Tarantool decimal type is stored as a MsgPack ext (Extension). Values with the decimal type are not floating-point values although they may contain decimal points. They are exact with up to 38 digits of precision. Example: a value returned by a function in the decimal module.

string. A string is a variable-length sequence of bytes, usually represented with alphanumeric characters inside single quotes. In both Lua and MsgPack, strings are treated as binary data, with no attempts to determine a string’s character set or to perform any string conversion – unless there is an optional collation. So, usually, string sorting and comparison are done byte-by-byte, without any special collation rules applied. (Example: numbers are ordered by their point on the number line, so 2345 is greater than 500; meanwhile, strings are ordered by the encoding of the first byte, then the encoding of the second byte, and so on, so '2345' is less than '500'.) Example: 'A, B, C'.

bin. A bin (binary) value is not directly supported by Lua but there is a Tarantool type varbinary which is encoded as MsgPack binary. For an (advanced) example showing how to insert varbinary into a database, see the Cookbook Recipe for ffi_varbinary_insert. Example: "\65 \66 \67".

uuid. The Tarantool uuid type is stored as a MsgPack ext (Extension). Values with the uuid type are Universally unique identifiers.
Example: 64d22e4d-ac92-4a23-899a-e5934af5479.

array. An array is represented in Lua with {...} (braces). Examples: as lists of numbers representing points in a geometric figure: {10, 11}, {3, 5, 9, 10}.

table. Lua tables with string keys are stored as MsgPack maps; Lua tables with integer keys starting with 1 are stored as MsgPack arrays. Nils may not be used in Lua tables; the workaround is to use box.NULL. Example: a box.space.tester:select() request will return a Lua table.

tuple. A tuple is a light reference to a MsgPack array stored in the database. It is a special type (cdata) to avoid conversion to a Lua table on retrieval. A few functions may return tables with multiple tuples. For tuple examples, see box.tuple.

scalar. Values in a scalar field can be boolean or integer or unsigned or double or number or decimal or string or varbinary – but not array or map or tuple. Examples: true, 1, 'xxx'.

any. Values in an any field can be boolean or integer or unsigned or double or number or decimal or string or varbinary – or array or map or tuple. Examples: true, 1, 'xxx', {box.NULL, 0}.

Examples of insert requests with different field types:

tarantool> box.space.K:insert{1,nil,true,'A B C',12345,1.2345}
---
- [1, null, true, 'A B C', 12345, 1.2345]
...
tarantool> box.space.K:insert{2,{['a']=5,['b']=6}}
---
- [2, {'a': 5, 'b': 6}]
...
tarantool> box.space.K:insert{3,{1,2,3,4,5}}
---
- [3, [1, 2, 3, 4, 5]]
...

Indexed field types

Indexes restrict values which Tarantool may store with MsgPack. This is why, for example, 'unsigned' and 'integer' are different field types although in MsgPack they are both stored as integer values – an 'unsigned' index contains only non-negative integer values while an ‘integer’ index contains any integer values.

Here again are the field types described in Field Type Details, and the index types they can fit in. The default field type is 'unsigned' and the default index type is TREE. Although 'nil' is not a legal indexed field type, indexes may contain nil as a non-default option. Full information is in section Details about index field types.

Options for box.schema.sequence.create()

Once a sequence exists, it can be altered, dropped, reset, forced to generate the next value, or associated with an index.

For an initial example, we generate a sequence named ‘S’.

tarantool> box.schema.sequence.create('S',{min=5, start=5})
---
- step: 1
  id: 5
  min: 5
  cache: 0
  uid: 1
  max: 9223372036854775807
  cycle: false
  name: S
  start: 5
...

The result shows that the new sequence has all default values, except for the two that were specified, min and start.

Then we get the next value, with the next() function.

tarantool> box.sequence.S:next()
---
- 5
...

The result is the same as the start value. If we called next() again, we would get 6 (because the previous value plus the step value is 6), and so on.

Then we create a new table, and say that its primary key may be generated from the sequence.

tarantool> s=box.schema.space.create('T')
---
...
tarantool> s:create_index('I',{sequence='S'})
---
- parts:
  - type: unsigned
    is_nullable: false
    fieldno: 1
  sequence_id: 1
  id: 0
  space_id: 520
  unique: true
  type: TREE
  sequence_fieldno: 1
  name: I
...
---
...

Then we insert a tuple, without specifying a value for the primary key.

tarantool> box.space.T:insert{nil,'other stuff'}
---
- [6, 'other stuff']
...

The result is a new tuple where the first field has a value of 6. This arrangement, where the system automatically generates the values for a primary key, is sometimes called “auto-incrementing” or “identity”.

For syntax and implementation details, see the reference for box.schema.sequence.

Data operations

The basic data operations supported in Tarantool are:

• five data-manipulation operations (INSERT, UPDATE, UPSERT, DELETE, REPLACE), and
• one data-retrieval operation (SELECT).

All of them are implemented as functions in box.space submodule.

Examples:

• INSERT: Add a new tuple to space ‘tester’.

  The first field, field[1], will be 999 (MsgPack type is integer).

  The second field, field[2], will be ‘Taranto’ (MsgPack type is string).

  tarantool> box.space.tester:insert{999, 'Taranto'}
  

• UPDATE: Update the tuple, changing field field[2].

  The clause “{999}”, which has the value to look up in the index of the tuple’s primary-key field, is mandatory, because update() requests must always have a clause that specifies a unique key, which in this case is field[1].

  The clause “{{‘=’, 2, ‘Tarantino’}}” specifies that assignment will happen to field[2] with the new value.

  tarantool> box.space.tester:update({999}, {{'=', 2, 'Tarantino'}})
  

• UPSERT: Upsert the tuple, changing field field[2] again.

  The syntax of upsert() is similar to the syntax of update(). However, the execution logic of these two requests is different. UPSERT is either UPDATE or INSERT, depending on the database’s state. Also, UPSERT execution is postponed until after transaction commit, so, unlike update(), upsert() doesn’t return data back.

  tarantool> box.space.tester:upsert({999, 'Taranted'}, {{'=', 2, 'Tarantism'}})
  

• REPLACE: Replace the tuple, adding a new field.

  This is also possible with the update() request, but the update() request is usually more complicated.

  tarantool> box.space.tester:replace{999, 'Tarantella', 'Tarantula'}
  

• SELECT: Retrieve the tuple.

  The clause “{999}” is still mandatory, although it does not have to mention the primary key.

  tarantool> box.space.tester:select{999}
  

• DELETE: Delete the tuple.

  In this example, we identify the primary-key field.

  tarantool> box.space.tester:delete{999}
  

Summarizing the examples:

• Functions insert and replace accept a tuple (where a primary key comes as part of the tuple).
• Function upsert accepts a tuple (where a primary key comes as part of the tuple), and also the update operations to execute.
• Function delete accepts a full key of any unique index (primary or secondary).
• Function update accepts a full key of any unique index (primary or secondary), and also the operations to execute.
• Function select accepts any key: primary/secondary, unique/non-unique, full/partial.

See reference on box.space for more details on using data operations.

Complexity factors

In reference for box.space and Submodule box.index submodules, there are notes about which complexity factors might affect the resource usage of each function.