Appendix A. Tarantool audit module¶
This document provides an overview of the Tarantool audit module.
Overview¶
The Tarantool audit module writes messages that record events from the Tarantool DBMS in plain text, CSV or JSON format.
It provides you with a detailed report of all security-related activities and helps you find and fix breaches to protect your business. For example, you can see who updated user privilege and when:
{"time": "2022-04-07T13:39:36.046+0300", "remote": "", "session_type": "background", "module": "tarantool", "user": "admin", "type": "user_priv", "tag": "", "description": "Update user guest privileges for role super from none to execute"}
It is up to each company to decide exactly what activities to audit and what actions to take. System administrators, security engineers and others in charge in the company may want to audit different events for different reasons. Tarantool provides such an option for each of them.
Types of events you can monitor¶
Tarantool records various types of audit log events that you can monitor and decide whether you need to take actions:
- Admin activity – events related to actions performed by the administrator. For example, such logs record the creation of a user.
- Data access and modification – events related to authorization and authentication of users. For example, such logs record failed attempts to access secure data.
- System events – events related to modification or configuration of resources. For example, such logs record the replacement of a space.
For more details about these audit log events, see the table below.
Event | Type of event written to the audit log | Example of an event display |
---|---|---|
Audit log enabled for events | audit_enable |
|
User authorized successfully | auth_ok |
{“name”: “user”} |
User authorization failed | auth_fail |
{“name”: “user”} |
User logged out or quit the session | disconnect |
|
Failed attempt to access secure data (personal records, details, geolocation, etc.) | access_denied |
{“name”: “obj_name”, “obj_type”: “space”, “access_type”: “read”} |
User created | user_create |
{“name”: “user”} |
User dropped | user_drop |
{“name”: “user”} |
User disabled | user_disable |
{“name”: “user”} |
User enabled | user_enable |
{“name”: “user”} |
User privileges (roles, profiles, etc.) granted or changed | user_priv |
{“name”: “user”, “obj_name”: “obj_name”, “obj_type”: “space”, “old_priv”: “”, “new_priv”: “read,write”} |
Password reset for a specific user | password_change |
{“name”: “user”} |
Role created | role_create |
{“name”: “role”} |
Role privileges granted or changed | role_priv |
{“name”: “role”, “obj_name”: “obj_name”, “obj_type”: “space”, “old_priv”: “”, “new_priv”: “read,write”} |
Space created | space_create |
{“space”: “name”} |
Space altered | space_alter |
{“space”: “name”} |
Space dropped | space_drop |
{“space”: “name”} |
Tuple inserted into space | space_insert |
{“tuple”: “name”} {“space”: “name”} |
Tuple replaced in space | space_replace |
{“tuple”: “name”} {“space”: “name”} |
Tuple deleted from space | space_delete |
{“tuple”: “name”} {“space”: “name”} |
Iterator key selected from space.index |
space_select |
{“iterator key”: “name”} {“space.index”: “space.index”} |
Function called with arguments | call |
{“function”: “name”} {“arguments”: “arguments”} |
Expressions with arguments evaluated in a string | eval |
{“expression”: “name”} {“arguments”: “arguments”} |
Note
The eval
event displays data from the console
module
and the eval
function of the net.box
module.
For more on how they work, see Module console
and Module net.box – eval.
To separate the data, specify console
or binary
in the session field.
Structure of audit log events¶
Each audit log event contains several fields to make it easy to filter and aggregate the resulting logs. They are described in the following table.
Field | Description | Example of a log field display |
---|---|---|
time |
Time of the event | 2022-04-07T13:20:05.327+0300 |
remote |
Remote host that triggered the event | 100.96.163.226:48722 |
session_type |
Session type | console |
module |
Audit log module. Set to tarantool for system events;
can be overwritten for user-defined events |
tarantool |
user |
User who triggered the event | admin |
type |
Audit event type | access_denied |
tag |
A text field that can be overwritten by the user | |
description |
Human-readable event description | Authenticate user Alice |
Warning
You can set all these parameters only once. Unlike many other parameters in box.cfg
,
they cannot be changed.
Enable the Tarantool audit log¶
By default, the audit_enable
option is set to false
.
Set the audit_enable
option to true
to start working with the Tarantool audit module.
You can also set this option back to false
to disable audit logging.
Choose where you want to write logs¶
By default, the audit_log
option is set to nil
and the Tarantool audit module sends audit logs to the default error stream (stderr
).
If you want to send the audit log to a file, to a pipe, or to the system logger,
you need to specify this audit_log
option.
Writing to a file¶
box.cfg{audit_log = 'audit_tarantool.log'}
-- or
box.cfg{audit_log = 'file:audit_tarantool.log'}
This opens the audit_tarantool.log
file for output in the server’s default directory.
If the audit_log
string has no prefix or the prefix file:
, the string is interpreted as a file path.
Sending to a pipe¶
box.cfg{audit_log = '| cronolog audit_tarantool.log'}
-- or
box.cfg{audit_log = 'pipe: cronolog audit_tarantool.log'}'
This starts the cronolog program when the server starts
and sends all audit_log
messages to cronolog’s standard input (stdin
).
If the audit_log
string starts with ‘|’ or contains the prefix pipe:
,
the string is interpreted as a Unix pipeline.
Sending to syslog¶
Warning
Below is an example of writing audit logs to a directory shared with the system logs. Tarantool allows this option, but it is not recommended to do this to avoid difficulties when working with audit logs. System and audit logs should be written separately. To do this, create separate paths and specify them.
This example setting sends the audit log to syslog:
box.cfg{audit_log = 'syslog:identity=tarantool'}
-- or
box.cfg{audit_log = 'syslog:facility=user'}
-- or
box.cfg{audit_log = 'syslog:identity=tarantool,facility=user'}
-- or
box.cfg{audit_log = 'syslog:server=unix:/dev/log'}
If the audit_log
string starts with “syslog:”,
it is interpreted as a message for the syslogd program,
which normally runs in the background of any Unix-like platform.
The setting can be ‘syslog:’, ‘syslog:facility=…’, ‘syslog:identity=…’, ‘syslog:server=…’ or a combination.
The syslog:identity
setting is an arbitrary string that is placed at the beginning of all messages.
The default value is tarantool
.
The syslog:facility
setting is currently ignored, but will be used in the future.
The value must be one of the syslog keywords
that tell syslogd
where to send the message.
The possible values are auth
, authpriv
, cron
, daemon
, ftp
,
kern
, lpr
, mail
, news
, security
, syslog
, user
, uucp
,
local0
, local1
, local2
, local3
, local4
, local5
, local6
, local7
.
The default value is local7
.
The syslog:server
setting is the locator for the syslog server.
It can be a Unix socket path starting with “unix:” or an ipv4 port number.
The default socket value is /dev/log
(on Linux) or /var/run/syslog
(on Mac OS).
The default port value is 514, which is the UDP port.
If you log to a file, Tarantool will reopen the audit log at SIGHUP.
If log is a program, its pid is stored in the audit_log.logger_pid
variable.
You need to send it a signal to rotate logs.
Configure a blocking mode¶
By default, the audit_nonblock
option is set to true
and Tarantool will not block during logging if the system is not ready to write, dropping the message instead.
Using this value may improve logging performance at the cost of losing some log messages.
This option only has an effect if the output goes to syslog:
or pipe:
.
Setting audit_nonblock
to true
is not allowed if the output is to a file.
In this case, set audit_nonblock
to false
.
Configure the format of audit log events¶
You can choose the format of audit log events – plain text, CSV or JSON format.
Plain text is used by default. This human-readable format can be efficiently compressed. The JSON format is more convenient to receive log events, analyze them and integrate them with other systems if needed. Using the CSV format allows you to view audit log events in tabular form.
Use these commands to configure the format of audit log events in Tarantool.
Plain text¶
box.cfg{audit_log = 'audit.log', audit_format = 'plain'}
JSON format¶
box.cfg{audit_log = 'audit.log', audit_format = 'json'}
CSV format¶
box.cfg{audit_log = 'audit.log', audit_format = 'csv'}
Use filters¶
Tarantool’s extensive filtering options help you write only the events you need to the audit log.
To set filters, use the box.cfg.audit_filter
option.
You have more than 10 values for this option, which you can find in the following table.
Value | Description |
---|---|
custom |
User-defined event logged with audit.log() Lua-function auth_ok |
auth_ok |
Authentication of username |
auth_fail |
Authentication of username failed |
disconnect |
Close connection |
user_create |
Create username |
user_drop |
Drop username |
role_create |
Create role name |
role_drop |
Drop role name |
user_enable |
Enable username |
user_disable |
Disable username |
user_grant_rights |
Grant read, write rights for space my_space to user my_user |
user_revoke_rights |
Revoke write rights for space my_space from user my_user |
role_grant_rights |
Grant execute rights for function my_func to role my_role |
role_revoke_rights |
Revoke write rights for space my_space from role my_role |
password_change access_denied |
Change password for username access_type access to object_type object_name denied |
Note
You cannot specify a filter twice or specify a filter that does not exist.
In this case you will get a configuration error.
The default value for the box.cfg.audit_filter
option is compatibility
,
which enables logging of all events available before 2.10.0.
Customize your filters¶
You can customize the filters and use different combinations of filters for your purposes.
Filter based on a specific event¶
You can set only certain events that you need to record.
For example, you can select password_change
to monitor the users who have changed their passwords.
Filter based on a specific group¶
You can set one of the groups of events that you need to record.
For example, you can select compatibility
to monitor only events of user authorization,
granted privileges, disconnection, user password change, and denied access.
Filter based on multiple groups¶
You can specify multiple groups depending on the purpose.
For example, you can select auth
and priv
to see only events related to authorization and granted privileges.
Filter based on a group and a specific event¶
You can specify a group and a certain event depending on the purpose.
For example, you can select priv
and disconnect
to see only events related to
granted privileges and disconnect events.
Example¶
Run the command to filter:
local audit = require('audit')
box.cfg{audit_log = 'audit.log', audit_filter = 'custom,user_create', audit_format = 'csv'}
-- The Tarantool audit module writes the event because a filter is set for it
box.schema.user.create('alice')
-- The Tarantool audit module will not write the event because no filter is set for it
box.schema.user.drop('alice')
Use event groups¶
You can simplify working with audit log events by using built-in groups in Tarantool. For example, you can set to record only events related to the enabling of the audit log, or only events related to a space.
Select one or more available groups to record the events you need:
Warning
Be careful when selecting
all
anddata operations
. The more events you record, the slower the requests will be processed over time. It is recommended that you select only those groups whose events your company really needs to monitor and analyze.
all
– records absolutely all events.audit
(audit_enable
) – records only events of audit enabling.ddl
(space_create
,space_alter
,space_drop
) – records events for creating, altering and dropping a space.data operations
(space_select
,space_insert
,space_replace
,space_delete
) – records events for selecting, inserting, replacing, or deleting a space.compatibility
(auth
,priv
,disconnect
,password_change
,access_denied
) – records only events of user authorization, granted privileges, disconnection, user password change, and denied access.custom
– records user-defined events.
Use API to create user-defined events¶
In addition, Tarantool provides you with an API that allows you to write user-defined audit log events.
For this, you can use the audit.log()
function that takes one of the following values:
- Message string. Printed to the audit log with type message. Example:
audit.log('Hello, World!')
. - Format string and arguments. Passed to string format and then output to the audit log with type message.
Example:
audit.log('Hello, %s!', 'World')
. - Table with audit log field values. The table must contain at least one field – description.
Example:
audit.log({type = 'custom_hello', description = 'Hello, World!'})
.
Using the field audit.new()
, you can create a new log module that allows you
to avoid passing all custom audit log fields each time audit.log()
is called.
It takes a table of audit log field values (same as audit.log()
).
Example¶
local my_audit = audit.new({type = 'custom_hello', module = 'my_module'})
my_audit:log('Hello, Alice!')
my_audit:log({tag = 'admin', description = 'Hello, Bob!'})
-- is equivalent to
audit.log({type = 'custom_hello', module = 'my_module',
description = 'Hello, Alice!'})
audit.log({type = 'custom_hello', module = 'my_module',
tag = 'admin', description = 'Hello, Bob!'})
Some user-defined audit log fields (time
, remote
, session_type
)
are set in the same way as for a system event.
If a field is not overwritten, it is set to the same value as for a system event.
Some audit log fields you can overwrite with audit.new()
and audit.log()
:
type
user
module
tag
description
Note
To avoid confusion with system events, the value of the type field must either be
message
(default) or begin withcustom_
. Otherwise you will get the error message. User-defined events are filtered out by default. To enable user-defined audit log events, you must addcustom
tobox.cfg.audit_filter
.
Example¶
local audit = require('audit')
box.cfg{audit_log = 'audit.log', audit_filter = 'custom', audit_format = 'csv'}
audit.log('Hello, Alice!')
audit.log('Hello, %s!', 'Bob')
audit.log({type = 'custom_hello', description = 'Hello, Eve!'})
audit.log({type = 'custom_farewell', user = 'eve', module = 'custom', description = 'Farewell, Eve!'})
local my_audit = audit.new({module = 'my_module', tag = 'default'})
my_audit:log({description = 'Message 1'})
my_audit:log({description = 'Message 2', tag = 'my_tag'})
my_audit:log({description = 'Message 3', module = 'other_module'})
Use read commands¶
To easily read the audit log events in the needed form, use the different commands:
cat
– prints one or more filesgrep
– prints a specific texthead
– prints the first N lines of the filetail
– prints the last N lines of the fileNote
These are the basic commands to help you read the logs. If necessary, you can use other commands.
Tips¶
How many events can be recorded?¶
If you write to a file, the size of the Tarantool audit module is limited by the disk space.
If you write to a system logger, the size of the Tarantool audit module is limited by the system logger.
If you write to a pipe, the size of the Tarantool audit module is limited by the system buffer
if the audit_nonblock
= false
; if audit_nonblock
= true
, there is no limit.
However, it is not recommended to use the entire memory, as this may cause performance degradation
and even loss of some logs.
How often should audit logs be reviewed?¶
Consider setting up a schedule in your company. It is recommended to review audit logs at least every 3 months.
How long should audit logs be stored?¶
It is recommended to store audit logs for at least one year.
What is the best way to process audit logs?¶
It is recommended to use SIEM systems for this issue.