Appendix A. Tarantool audit module | Enterprise
Appendixes Appendix A. Tarantool audit module

Appendix A. Tarantool audit module

This document provides an overview of the Tarantool audit module.

Overview

The Tarantool audit module writes messages that record events from the Tarantool DBMS in plain text, CSV or JSON format.

It provides you with a detailed report of all security-related activities and helps you find and fix breaches to protect your business. For example, you can see who updated user privilege and when:

{"time": "2022-04-07T13:39:36.046+0300", "remote": "", "session_type": "background", "module": "tarantool", "user": "admin", "type": "user_priv", "tag": "", "description": "Update user guest privileges for role super from none to execute"}

It is up to each company to decide exactly what activities to audit and what actions to take. System administrators, security engineers and others in charge in the company may want to audit different events for different reasons. Tarantool provides such an option for each of them.

Types of events you can monitor

Tarantool records various types of audit log events that you can monitor and decide whether you need to take actions:

  • Admin activity – events related to actions performed by the administrator. For example, such logs record the creation of a user.
  • Data access and modification – events related to authorization and authentication of users. For example, such logs record failed attempts to access secure data.
  • System events – events related to modification or configuration of resources. For example, such logs record the replacement of a space.

For more details about these audit log events, see the table below.

Event Type of event written to the audit log Example of an event display
Audit log enabled for events audit_enable  
User authorized successfully auth_ok {“name”: “user”}
User authorization failed auth_fail {“name”: “user”}
User logged out or quit the session disconnect  
Failed attempt to access secure data (personal records, details, geolocation, etc.) access_denied {“name”: “obj_name”, “obj_type”: “space”, “access_type”: “read”}
User created user_create {“name”: “user”}
User dropped user_drop {“name”: “user”}
User disabled user_disable {“name”: “user”}
User enabled user_enable {“name”: “user”}
User privileges (roles, profiles, etc.) granted or changed user_priv {“name”: “user”, “obj_name”: “obj_name”, “obj_type”: “space”, “old_priv”: “”, “new_priv”: “read,write”}
Password reset for a specific user password_change {“name”: “user”}
Role created role_create {“name”: “role”}
Role privileges granted or changed role_priv {“name”: “role”, “obj_name”: “obj_name”, “obj_type”: “space”, “old_priv”: “”, “new_priv”: “read,write”}
Space created space_create {“space”: “name”}
Space altered space_alter {“space”: “name”}
Space dropped space_drop {“space”: “name”}
Tuple inserted into space space_insert {“tuple”: “name”} {“space”: “name”}
Tuple replaced in space space_replace {“tuple”: “name”} {“space”: “name”}
Tuple deleted from space space_delete {“tuple”: “name”} {“space”: “name”}
Iterator key selected from space.index space_select {“iterator key”: “name”} {“space.index”: “space.index”}
Function called with arguments call {“function”: “name”} {“arguments”: “arguments”}
Expressions with arguments evaluated in a string eval {“expression”: “name”} {“arguments”: “arguments”}

Note

The eval event displays data from the console module and the eval function of the net.box module. For more on how they work, see Module console and Module net.box – eval. To separate the data, specify console or binary in the session field.

Structure of audit log events

Each audit log event contains several fields to make it easy to filter and aggregate the resulting logs. They are described in the following table.

Field Description Example of a log field display
time Time of the event 2022-04-07T13:20:05.327+0300
remote Remote host that triggered the event 100.96.163.226:48722
session_type Session type console
module Audit log module. Set to tarantool for system events; can be overwritten for user-defined events tarantool
user User who triggered the event admin
type Audit event type access_denied
tag A text field that can be overwritten by the user  
description Human-readable event description Authenticate user Alice

Warning

You can set all these parameters only once. Unlike many other parameters in box.cfg, they cannot be changed.

Enable the Tarantool audit log

By default, the audit_enable option is set to false. Set the audit_enable option to true to start working with the Tarantool audit module.

You can also set this option back to false to disable audit logging.

Choose where you want to write logs

By default, the audit_log option is set to nil and the Tarantool audit module sends audit logs to the default error stream (stderr). If you want to send the audit log to a file, to a pipe, or to the system logger, you need to specify this audit_log option.

Writing to a file

box.cfg{audit_log = 'audit_tarantool.log'}
-- or
box.cfg{audit_log = 'file:audit_tarantool.log'}

This opens the audit_tarantool.log file for output in the server’s default directory. If the audit_log string has no prefix or the prefix file:, the string is interpreted as a file path.

Sending to a pipe

box.cfg{audit_log = '| cronolog audit_tarantool.log'}
-- or
box.cfg{audit_log = 'pipe: cronolog audit_tarantool.log'}'

This starts the cronolog program when the server starts and sends all audit_log messages to cronolog’s standard input (stdin). If the audit_log string starts with ‘|’ or contains the prefix pipe:, the string is interpreted as a Unix pipeline.

Sending to syslog

Warning

Below is an example of writing audit logs to a directory shared with the system logs. Tarantool allows this option, but it is not recommended to do this to avoid difficulties when working with audit logs. System and audit logs should be written separately. To do this, create separate paths and specify them.

This example setting sends the audit log to syslog:

box.cfg{audit_log = 'syslog:identity=tarantool'}
-- or
box.cfg{audit_log = 'syslog:facility=user'}
-- or
box.cfg{audit_log = 'syslog:identity=tarantool,facility=user'}
-- or
box.cfg{audit_log = 'syslog:server=unix:/dev/log'}

If the audit_log string starts with “syslog:”, it is interpreted as a message for the syslogd program, which normally runs in the background of any Unix-like platform. The setting can be ‘syslog:’, ‘syslog:facility=…’, ‘syslog:identity=…’, ‘syslog:server=…’ or a combination.

The syslog:identity setting is an arbitrary string that is placed at the beginning of all messages. The default value is tarantool.

The syslog:facility setting is currently ignored, but will be used in the future. The value must be one of the syslog keywords that tell syslogd where to send the message. The possible values are auth, authpriv, cron, daemon, ftp, kern, lpr, mail, news, security, syslog, user, uucp, local0, local1, local2, local3, local4, local5, local6, local7. The default value is local7.

The syslog:server setting is the locator for the syslog server. It can be a Unix socket path starting with “unix:” or an ipv4 port number. The default socket value is /dev/log (on Linux) or /var/run/syslog (on Mac OS). The default port value is 514, which is the UDP port.

If you log to a file, Tarantool will reopen the audit log at SIGHUP. If log is a program, its pid is stored in the audit_log.logger_pid variable. You need to send it a signal to rotate logs.

Configure a blocking mode

By default, the audit_nonblock option is set to true and Tarantool will not block during logging if the system is not ready to write, dropping the message instead. Using this value may improve logging performance at the cost of losing some log messages. This option only has an effect if the output goes to syslog: or pipe:. Setting audit_nonblock to true is not allowed if the output is to a file. In this case, set audit_nonblock to false.

Configure the format of audit log events

You can choose the format of audit log events – plain text, CSV or JSON format.

Plain text is used by default. This human-readable format can be efficiently compressed. The JSON format is more convenient to receive log events, analyze them and integrate them with other systems if needed. Using the CSV format allows you to view audit log events in tabular form.

Use these commands to configure the format of audit log events in Tarantool.

Plain text

box.cfg{audit_log = 'audit.log', audit_format = 'plain'}

JSON format

box.cfg{audit_log = 'audit.log', audit_format = 'json'}

CSV format

box.cfg{audit_log = 'audit.log', audit_format = 'csv'}

Use filters

Tarantool’s extensive filtering options help you write only the events you need to the audit log.

To set filters, use the box.cfg.audit_filter option.

You have more than 10 values for this option, which you can find in the following table.

Value Description
custom User-defined event logged with audit.log() Lua-function auth_ok
auth_ok Authentication of username
auth_fail Authentication of username failed
disconnect Close connection
user_create Create username
user_drop Drop username
role_create Create role name
role_drop Drop role name
user_enable Enable username
user_disable Disable username
user_grant_rights Grant read, write rights for space my_space to user my_user
user_revoke_rights Revoke write rights for space my_space from user my_user
role_grant_rights Grant execute rights for function my_func to role my_role
role_revoke_rights Revoke write rights for space my_space from role my_role
password_change access_denied Change password for username access_type access to object_type object_name denied

Note

You cannot specify a filter twice or specify a filter that does not exist. In this case you will get a configuration error. The default value for the box.cfg.audit_filter option is compatibility, which enables logging of all events available before 2.10.0.

Customize your filters

You can customize the filters and use different combinations of filters for your purposes.

Filter based on a specific event

You can set only certain events that you need to record.

For example, you can select password_change to monitor the users who have changed their passwords.

Filter based on a specific group

You can set one of the groups of events that you need to record.

For example, you can select compatibility to monitor only events of user authorization, granted privileges, disconnection, user password change, and denied access.

Filter based on multiple groups

You can specify multiple groups depending on the purpose.

For example, you can select auth and priv to see only events related to authorization and granted privileges.

Filter based on a group and a specific event

You can specify a group and a certain event depending on the purpose.

For example, you can select priv and disconnect to see only events related to granted privileges and disconnect events.

Example

Run the command to filter:

local audit = require('audit')

box.cfg{audit_log = 'audit.log', audit_filter = 'custom,user_create', audit_format = 'csv'}
-- The Tarantool audit module writes the event because a filter is set for it
box.schema.user.create('alice')
-- The Tarantool audit module will not write the event because no filter is set for it
box.schema.user.drop('alice')

Use event groups

You can simplify working with audit log events by using built-in groups in Tarantool. For example, you can set to record only events related to the enabling of the audit log, or only events related to a space.

Select one or more available groups to record the events you need:

Warning

Be careful when selecting all and data operations. The more events you record, the slower the requests will be processed over time. It is recommended that you select only those groups whose events your company really needs to monitor and analyze.

  • all – records absolutely all events.
  • audit (audit_enable) – records only events of audit enabling.
  • ddl (space_create, space_alter, space_drop) – records events for creating, altering and dropping a space.
  • data operations (space_select, space_insert, space_replace, space_delete) – records events for selecting, inserting, replacing, or deleting a space.
  • compatibility (auth, priv, disconnect, password_change , access_denied) – records only events of user authorization, granted privileges, disconnection, user password change, and denied access.
  • custom – records user-defined events.

Use API to create user-defined events

In addition, Tarantool provides you with an API that allows you to write user-defined audit log events.

For this, you can use the audit.log() function that takes one of the following values:

  • Message string. Printed to the audit log with type message. Example: audit.log('Hello, World!').
  • Format string and arguments. Passed to string format and then output to the audit log with type message. Example: audit.log('Hello, %s!', 'World').
  • Table with audit log field values. The table must contain at least one field – description. Example: audit.log({type = 'custom_hello', description = 'Hello, World!'}).

Using the field audit.new(), you can create a new log module that allows you to avoid passing all custom audit log fields each time audit.log() is called. It takes a table of audit log field values (same as audit.log()).

Example

local my_audit = audit.new({type = 'custom_hello', module = 'my_module'})

my_audit:log('Hello, Alice!')

my_audit:log({tag = 'admin', description = 'Hello, Bob!'})

-- is equivalent to

audit.log({type = 'custom_hello', module = 'my_module',

description = 'Hello, Alice!'})

audit.log({type = 'custom_hello', module = 'my_module',

tag = 'admin', description = 'Hello, Bob!'})

Some user-defined audit log fields (time, remote, session_type) are set in the same way as for a system event. If a field is not overwritten, it is set to the same value as for a system event.

Some audit log fields you can overwrite with audit.new() and audit.log():

  • type

  • user

  • module

  • tag

  • description

    Note

    To avoid confusion with system events, the value of the type field must either be message (default) or begin with custom_. Otherwise you will get the error message. User-defined events are filtered out by default. To enable user-defined audit log events, you must add custom to box.cfg.audit_filter.

Example

local audit = require('audit')

box.cfg{audit_log = 'audit.log', audit_filter = 'custom', audit_format = 'csv'}

audit.log('Hello, Alice!')

audit.log('Hello, %s!', 'Bob')

audit.log({type = 'custom_hello', description = 'Hello, Eve!'})

audit.log({type = 'custom_farewell', user = 'eve', module = 'custom', description = 'Farewell, Eve!'})

local my_audit = audit.new({module = 'my_module', tag = 'default'})

my_audit:log({description = 'Message 1'})

my_audit:log({description = 'Message 2', tag = 'my_tag'})

my_audit:log({description = 'Message 3', module = 'other_module'})

Use read commands

To easily read the audit log events in the needed form, use the different commands:

  • cat – prints one or more files

  • grep – prints a specific text

  • head – prints the first N lines of the file

  • tail – prints the last N lines of the file

    Note

    These are the basic commands to help you read the logs. If necessary, you can use other commands.

Tips

How many events can be recorded?

If you write to a file, the size of the Tarantool audit module is limited by the disk space. If you write to a system logger, the size of the Tarantool audit module is limited by the system logger. If you write to a pipe, the size of the Tarantool audit module is limited by the system buffer if the audit_nonblock = false; if audit_nonblock = true, there is no limit. However, it is not recommended to use the entire memory, as this may cause performance degradation and even loss of some logs.

How often should audit logs be reviewed?

Consider setting up a schedule in your company. It is recommended to review audit logs at least every 3 months.

How long should audit logs be stored?

It is recommended to store audit logs for at least one year.

What is the best way to process audit logs?

It is recommended to use SIEM systems for this issue.

Found what you were looking for?
Feedback