space_object:create_index()

Details about index field types

Index field types differ depending on what values are allowed, and what index types are allowed.

Allowing null for an indexed key

box.space.tester:create_index('I', {unique = true, parts = {{field = 2, type = 'number', is_nullable = true}}})

Creating an index using field names instead of field numbers

create_index() can use field names and/or field types described by the optional space_object:format() clause.

In the following example, we show format() for a space that has two columns named ‘x’ and ‘y’, and then we show five variations of the parts={} clause of create_index(), first for the ‘x’ column, second for both the ‘x’ and ‘y’ columns. The variations include omitting the type, using numbers, and adding extra braces.

box.space.tester:format({{name = 'x', type = 'scalar'}, {name = 'y', type = 'integer'}})

box.space.tester:create_index('I2', {parts = {{'x', 'scalar'}}})
box.space.tester:create_index('I3', {parts = {{'x', 'scalar'}, {'y', 'integer'}}})

box.space.tester:create_index('I4', {parts = {{1, 'scalar'}}})
box.space.tester:create_index('I5', {parts = {{1, 'scalar'}, {2, 'integer'}}})

box.space.tester:create_index('I6', {parts = {1}})
box.space.tester:create_index('I7', {parts = {1, 2}})

box.space.tester:create_index('I8', {parts = {'x'}})
box.space.tester:create_index('I9', {parts = {'x', 'y'}})

box.space.tester:create_index('I10', {parts = {{'x'}}})
box.space.tester:create_index('I11', {parts = {{'x'}, {'y'}}})

Creating an index using the path option for map fields (JSON-path indexes)

To create an index for a field that is a map (a path string and a scalar value), specify the path string during index creation, like this:

parts = {field-number, 'data-type', path = 'path-name'}

The index type must be TREE or HASH and the contents of the field must always be maps with the same path.

Example 1 – The simplest use of path:

tarantool> box.schema.space.create('T')
tarantool> box.space.T:create_index('I',{parts = {{field = 1, type = 'scalar', path = 'age'}}})
tarantool> box.space.T:insert({{age = 44}})
tarantool> box.space.T:select(44)
---
- [{'age': 44}]

Example 2 – path plus format() plus JSON syntax to add clarity:

tarantool> my_space = box.schema.space.create('T')
tarantool> format = {{'id', 'unsigned'}, {'data', 'map'}}
tarantool> my_space:format(format)
tarantool> parts = {{'data.FIO["firstname"]', 'str'}, {'data.FIO["surname"]', 'str'}}
tarantool> my_index = my_space:create_index('info', {parts = parts})
tarantool> my_space:insert({1, {FIO = {firstname = 'Ahmed', surname = 'Xi'}}})
---
- [1, {'FIO': {'surname': 'Xi', 'firstname': 'Ahmed'}}]

Note regarding storage engine: vinyl supports only the TREE index type, and vinyl secondary indexes must be created before tuples are inserted.

Creating a multikey index using the path option with [*]

The string in a path option can contain [*] which is called an array index placeholder. Indexes defined with this are useful for JSON documents that all have the same structure.

For example, when creating an index on field#2 for a string document that will start with {'data': [{'name': '...'}, {'name': '...'}], the parts section in the create_index request could look like:

parts = {{field = 2, type = 'str', path = 'data[*].name'}}

Then tuples containing names can be retrieved quickly with index_object:select({key-value}).

In fact a single field can have multiple keys, as in this example which retrieves the same tuple twice because there are two keys ‘A’ and ‘B’ which both match the request:

my_space = box.schema.space.create('json_documents')
my_space:create_index('primary')
multikey_index = my_space:create_index('multikey', {parts = {{field = 2, type = 'str', path = 'data[*].name'}}})
my_space:insert({1,
         {data = {{name = 'A'},
                  {name = 'B'}},
          extra_field = 1}})
multikey_index:select({''}, {iterator = 'GE'})

The result of the select request looks like this:

tarantool> multikey_index:select({''},{iterator='GE'})
---
- - [1, {'data': [{'name': 'A'}, {'name': 'B'}], 'extra_field': 1}]
- [1, {'data': [{'name': 'A'}, {'name': 'B'}], 'extra_field': 1}]
...

Some restrictions exist:

• [*] must be alone or must be at the end of a name in the path.
• [*] must not appear twice in the path.
• If an index has a path with x[*], then no other index can have a path with x.component.
• [*] must not appear in the path of a primary key.
• If an index has unique=true and has a path with [*], then duplicate keys from different tuples are disallowed, but duplicate keys for the same tuple are allowed.
• The field’s value must have the same structure as in the path definition, or be nil (nil is not indexed).
• In a space with multikey indexes, any tuple cannot contain more than ~8,000 elements indexed that way.

Creating a functional index

Functional indexes are indexes that call a user-defined function for forming the index key, rather than depending entirely on the Tarantool default formation. Functional indexes are useful for condensing or truncating or reversing or any other way that users want to customize the index.

There are several recommendations on building functional indexes:

• The function definition must expect a tuple, which has the contents of fields at the time a data-change request happens, and must return a tuple, which has the contents that will actually be put in the index.
• The create_index definition must include specification of all key parts, and the custom function must return a table which has the same number of key parts with the same types.
• The space must have a memtx engine.
• The function must be persistent and deterministic (see Creating a function with body).
• The key parts must not depend on JSON paths.
• The function must access key-part values by index, not by field name.
• Functional indexes must not be primary-key indexes.
• Functional indexes cannot be altered and the function cannot be changed if it is used for an index, so the only way to change them is to drop the index and create it again.
• Only sandboxed functions are suitable for functional indexes.

Example:

A function could make a key using only the first letter of a string field.

1. Make a space. The space needs a primary-key field, which is not the field that we will use for the functional index:

   box.schema.space.create('tester', {engine = 'memtx'})
   box.space.tester:create_index('i',{parts={{field = 1, type = 'string'}}})
   

2. Make a function. The function expects a tuple. In this example it will work on tuple[2] because the key source is field number 2 in what we will insert. Use string.sub() from the string module to get the first character:

   lua_code = [[function(tuple) return {string.sub(tuple[2],1,1)} end]]
   

3. Make the function persistent using the box.schema.func.create function:

   box.schema.func.create('my_func',
       {body = lua_code, is_deterministic = true, is_sandboxed = true})
   

4. Make a functional index. Specify the fields whose values will be passed to the function. Specify the function:

   box.space.tester:create_index('func_idx',{parts={{field = 1, type = 'string'}},func = 'my_func'})
   

5. Test. Insert a few tuples. Select using only the first letter, it will work because that is the key. Or, select using the same function as was used for insertion:

   box.space.tester:insert({'a', 'wombat'})
   box.space.tester:insert({'b', 'rabbit'})
   box.space.tester.index.func_idx:select('w')
   box.space.tester.index.func_idx:select(box.func.my_func:call({{'tester', 'wombat'}}));
   

   The results of the two select requests will look like this:

   tarantool> box.space.tester.index.func_idx:select('w')
   ---
   - - ['a', 'wombat']
   ...
   tarantool> box.space.tester.index.func_idx:select(box.func.my_func:call({{'tester','wombat'}}));
   ---
   - - ['a', 'wombat']
   ...
   

Here is the full code of the example:

box.schema.space.create('tester', {engine = 'memtx'})
box.space.tester:create_index('i',{parts={{field = 1, type = 'string'}}})
lua_code = [[function(tuple) return {string.sub(tuple[2],1,1)} end]]
box.schema.func.create('my_func',
    {body = lua_code, is_deterministic = true, is_sandboxed = true})
box.space.tester:create_index('func_idx',{parts={{field = 1, type = 'string'}},func = 'my_func'})
box.space.tester:insert({'a', 'wombat'})
box.space.tester:insert({'b', 'rabbit'})
box.space.tester.index.func_idx:select('w')
box.space.tester.index.func_idx:select(box.func.my_func:call({{'tester', 'wombat'}}));

Functions for functional indexes can return multiple keys. Such functions are called “multikey” functions.

To create a multikey function, the options of box.schema.func.create() must include is_multikey = true. The return value must be a table of tuples. If a multikey function returns N tuples, then N keys will be added to the index.

Example:

s = box.schema.space.create('withdata')
s:format({{name = 'name', type = 'string'},
          {name = 'address', type = 'string'}})
pk = s:create_index('name', {parts = {{field = 1, type = 'string'}}})
lua_code = [[function(tuple)
               local address = string.split(tuple[2])
               local ret = {}
               for _, v in pairs(address) do
                 table.insert(ret, {utf8.upper(v)})
               end
               return ret
             end]]
box.schema.func.create('address',
                        {body = lua_code,
                         is_deterministic = true,
                         is_sandboxed = true,
                         is_multikey = true})
idx = s:create_index('addr', {unique = false,
                              func = 'address',
                              parts = {{field = 1, type = 'string',
                                      collation = 'unicode_ci'}}})
s:insert({"James", "SIS Building Lambeth London UK"})
s:insert({"Sherlock", "221B Baker St Marylebone London NW1 6XE UK"})
idx:select('Uk')
-- Both tuples will be returned.