Top.Mail.Ru
API reference | Tarantool
Tarantool
Узнайте содержание релиза 2.8

API reference

API reference

An application using the metrics module has 4 primitives (called «collectors») at its disposal:

A collector represents one or more observations that are changing over time.

metrics.counter(name[, help])

Registers a new counter.

Параметры:
  • name (string) – Collector name. Must be unique.
  • help (string) – Help description.
Return:

Counter object

Rtype:

counter_obj

object counter_obj
counter_obj:inc(num, label_pairs)

Increments an observation under label_pairs. If label_pairs didn’t exist before, this creates it.

Параметры:
  • num (number) – Increase value.
  • label_pairs (table) – Table containing label names as keys, label values as values. Note that both labels names and values in label_pairs are treated as strings.
counter_obj:collect()
Return:Array of observation objects for the given counter.
{
    label_pairs: table,          -- `label_pairs` key-value table
    timestamp: ctype<uint64_t>,  -- current system time (in microseconds)
    value: number,               -- current value
    metric_name: string,         -- collector
}
Rtype:table
counter_obj:remove(label_pairs)

Removes an observation with label_pairs.

counter_obj:reset(label_pairs)

Set an observation under label_pairs to 0.

Параметры:
  • label_pairs (table) – Table containing label names as keys, label values as values. Note that both labels names and values in label_pairs are treated as strings.

metrics.gauge(name[, help])

Registers a new gauge. Returns a Gauge object.

Параметры:
  • name (string) – Collector name. Must be unique.
  • help (string) – Help description.
Return:

Gauge object

Rtype:

gauge_obj

object gauge_obj
gauge_obj:inc(num, label_pairs)

Same as Counter inc().

gauge_obj:dec(num, label_pairs)

Same as inc(), but decreases the observation.

gauge_obj:set(num, label_pairs)

Same as inc(), but sets the observation.

gauge_obj:collect()

Returns an array of observation objects for the given gauge. For observation description, see counter_obj:collect().

gauge_obj:remove(label_pairs)

Same as Counter remove().

metrics.histogram(name[, help, buckets])

Registers a new histogram.

Параметры:
  • name (string) – Collector name. Must be unique.
  • help (string) – Help description.
  • buckets (table) – Histogram buckets (an array of sorted positive numbers). Infinity bucket (INF) is appended automatically. Default is {.005, .01, .025, .05, .075, .1, .25, .5, .75, 1.0, 2.5, 5.0, 7.5, 10.0, INF}.
Return:

Histogram object

Rtype:

histogram_obj

Примечание

The histogram is just a set of collectors:

  • name .. "_sum" - A counter holding the sum of added observations. Contains only an empty label set.
  • name .. "_count" - A counter holding the number of added observations. Contains only an empty label set.
  • name .. "_bucket" - A counter holding all bucket sizes under the label le (low or equal). So to access a specific bucket x (x is a number), you should specify the value x for the label le.
object histogram_obj
histogram_obj:observe(num, label_pairs)

Records a new value in a histogram. This increments all buckets sizes under labels le >= num and labels matching label_pairs.

Параметры:
  • num (number) – Value to put in the histogram.
  • label_pairs (table) – Table containing label names as keys, label values as values (table). A new value is observed by all internal counters with these labels specified. Note that both labels names and values in label_pairs are treated as strings.
histogram_obj:collect()

Returns a concatenation of counter_obj:collect() across all internal counters of histogram_obj. For observation description, see counter_obj:collect().

histogram_obj:remove(label_pairs)

Same as Counter remove().

metrics.summary(name[, help, objectives])

Registers a new summary. Quantile computation is based on the algorithm «Effective computation of biased quantiles over data streams»

Параметры:
  • name (string) – Collector name. Must be unique.
  • help (string) – Help description.
  • objectives (table) – A list of „targeted“ φ-quantiles in the form {quantile = error, ... }. For example: {[0.5]=0.01, [0.9]=0.01, [0.99]=0.01}. A targeted φ-quantile is specified in the form of a φ-quantile and tolerated error. For example a {[0.5] = 0.1} means that the median (= 50th percentile) should be returned with 10 percent error. Note that percentiles and quantiles are the same concept, except percentiles are expressed as percentages. The φ-quantile must be in the interval [0, 1]. Note that a lower tolerated error for a φ-quantile results in higher usage of resources (memory and CPU) to calculate the summary.
  • params (table) – Table of summary parameters, used for configuring sliding window of time. „Sliding window“ consists of several buckets to store observations. New observations are added to each bucket. After a time period, the „head“ bucket (bucket from which observations are collected) is reset and the next bucket becomes a new „head“. I.e. each bucket will store observations for max_age_time * age_buckets_count seconds before it will be reset. max_age_time sets the duration of each bucket lifetime, i.e., how long observations are kept before they are discarded, in seconds age_buckets_count sets the number of buckets of the time window. It determines the number of buckets used to exclude observations that are older than max_age_time from the Summary. The value is a trade-off between resources (memory and CPU for maintaining the bucket) and how smooth the time window is moved. Default value is {max_age_time = math.huge, age_buckets_count = 1}
Return:

Summary object

Rtype:

summary_obj

Примечание

The summary is just a set of collectors:

  • name .. "_sum" - A counter holding the sum of added observations.
  • name .. "_count" - A counter holding the number of added observations.
  • name - It’s holding all quantiles under observation under the label quantile (low or equal). So to access a specific quantile x (x is a number), you should specify the value x for the label quantile.
object summary_obj
summary_obj:observe(num, label_pairs)

Records a new value in a summary.

Параметры:
  • num (number) – Value to put in the data stream.
  • label_pairs (table) – A table containing label names as keys, label values as values (table). A new value is observed by all internal counters with these labels specified. Label "quantile" are not allowed in summary. It will be added automatically. If max_age_time and age_buckets_count are set, the observed value will be added to each bucket. Note that both labels names and values in label_pairs are treated as strings.
summary_obj:collect()

Returns a concatenation of counter_obj:collect() across all internal counters of summary_obj. For observation description, see counter_obj:collect(). If max_age_time and age_buckets_count are set, quantile observations will be collect only from the head bucket in sliding window and not from every bucket. If there was no observations returns NaN in values.

summary_obj:remove(label_pairs)

Same as Counter remove().

All collectors support providing label_pairs on data modification. Labels are basically a metainfo that you associate with a metric in the format of key-value pairs. See tags in Graphite and labels in Prometheus. Labels are used to differentiate the characteristics of a thing being measured. For example, in a metric associated with the total number of http requests, you can use methods and statuses label pairs:

http_requests_total_counter:inc(1, {method = 'POST', status = '200'})

You don’t have to predefine labels in advance.

Using labels on your metrics allows you to later derive new time series (visualize their graphs) by specifying conditions on label values. In the example above, we could derive these time series:

  1. The total number of requests over time with method = «POST» (and any status).
  2. The total number of requests over time with status = 500 (and any method).

You can also set global labels by calling metrics.set_global_labels({ label = value, ...}).

metrics.enable_default_metrics([include, exclude])

Enables Tarantool metrics collections.

Параметры:
  • include (table) – Table containing names of default metrics which you need to enable.
  • exclude (table) – Table containing names of default metrics which you need to exclude.

Default metrics names:

  • «network»
  • «operations»
  • «system»
  • «replicas»
  • «info»
  • «slab»
  • «runtime»
  • «memory»
  • «spaces»
  • «fibers»
  • «cpu»
  • «vinyl»
  • «luajit»
  • «cartridge_issues»
  • «clock»

See metrics reference for details.

metrics.set_global_labels(label_pairs)

Set global labels that will be added to every observation.

Параметры:
  • label_pairs (table) – Table containing label names as string keys, label values as values (table).

Global labels are applied only on metrics collection and have no effect on how observations are stored.

Global labels can be changed on the fly.

Observation label_pairs has priority over global labels: if you pass label_pairs to an observation method with the same key as some global label, the method argument value will be used.

Note that both labels names and values in label_pairs are treated as strings.

metrics.collect()

Collect observations from each collector.

metrics.register_callback(callback)

Registers a function callback which will be called right before metrics collection on plugin export.

Параметры:
  • callback (function) – Function which takes no parameters.

Most common usage is for gauge metrics updates.

metrics.unregister_callback(callback)

Unregisters a function callback which will be called right before metrics collection on plugin export.

Параметры:
  • callback (function) – Function which takes no parameters.

Most common usage is for unregister enabled callbacks.

Functions to call with metrics = require('cartridge.roles.metrics') in init.lua

metrics.set_export(export)
Параметры:
  • export (table) – Table containing path and format of exporting metrics.

Configure endpoints of metrics role:

local metrics = require('cartridge.roles.metrics')
metrics.set_export({
    {
        path = '/path_for_json_metrics',
        format = 'json'
    },
    {
        path = '/path_for_prometheus_metrics',
        format = 'prometheus'
    },
    {
        path = '/health',
        format = 'health'
    }
})

You can add several entry points of the same format by different paths, like this:

metrics.set_export({
    {
        path = '/path_for_json_metrics',
        format = 'json'
    },
    {
        path = '/another_path_for_json_metrics',
        format = 'json'
    },
})
metrics.set_default_labels(label_pairs)

Add default global labels. Note that both labels names and values in label_pairs are treated as strings.

Параметры:
  • label_pairs (table) – Table containing label names as string keys,

label values as values (table).

local metrics = require('cartridge.roles.metrics')
metrics.set_default_labels({ ['my-custom-label'] = 'label-value' })

metrics also provides a middleware for monitoring HTTP (set by the http module) latency statistics.

metrics.http_middleware.configure_default_collector(type_name, name, help)

Registers a collector for the middleware and sets it as default.

Параметры:
  • type_name (string) – Collector type: «histogram» or «summary». Default is «histogram».
  • name (string) – Collector name. Default is «http_server_request_latency».
  • help (string) – Help description. Default is «HTTP Server Request Latency».

If a collector with the same type and name already exists in the registry, throws an error.

metrics.http_middleware.build_default_collector(type_name, name[, help])

Registers a collector for the middleware and returns it.

Параметры:
  • type_name (string) – Collector type: «histogram» or «summary». Default is «histogram».
  • name (string) – Collector name. Default is «http_server_request_latency».
  • help (string) – Help description. Default is «HTTP Server Request Latency».

If a collector with the same type and name already exists in the registry, throws an error.

metrics.http_middleware.set_default_collector(collector)

Sets the default collector.

Параметры:
  • collector – Middleware collector object.
metrics.http_middleware.get_default_collector()

Returns the default collector. If the default collector hasn’t been set yet, registers it (with default http_middleware.build_default_collector(...) parameters) and sets it as default.

metrics.http_middleware.v1(handler, collector)

Latency measure wrap-up for HTTP ver. 1.x.x handler. Returns a wrapped handler.

Параметры:
  • handler (function) – Handler function.
  • collector – Middleware collector object. If not set, uses the default collector (like in http_middleware.get_default_collector()).

Usage: httpd:route(route, http_middleware.v1(request_handler, collector))

For a more detailed example, see https://github.com/tarantool/metrics/blob/master/example/HTTP/latency_v1.lua

metrics.http_middleware.v2(collector)

Returns the latency measure middleware for HTTP ver. 2.x.x.

Параметры:
  • collector – Middleware collector object. If not set, uses the default collector (like in http_middleware.get_default_collector()).

Usage:

router = require('http.router').new()
router:route(route, request_handler)
router:use(http_middleware.v2(collector), {name = 'http_instrumentation'}) -- the second argument is optional, see HTTP docs

For a more detailed example, see https://github.com/tarantool/metrics/blob/master/example/HTTP/latency_v2.lua

CPU metrics work only on Linux. See metrics reference for details. To enable it you should register callback:

local metrics = require('metrics')

metrics.register_callback(function()
    local cpu_metrics = require('metrics.psutils.cpu')
    cpu_metrics.update()
end)

Collected metrics example

# HELP tnt_cpu_total Host CPU time
# TYPE tnt_cpu_total gauge
tnt_cpu_total 15006759
# HELP tnt_cpu_thread Tarantool thread cpu time
# TYPE tnt_cpu_thread gauge
tnt_cpu_thread{thread_name="coio",file_name="init.lua",thread_pid="699",kind="system"} 160
tnt_cpu_thread{thread_name="tarantool",file_name="init.lua",thread_pid="1",kind="user"} 949
tnt_cpu_thread{thread_name="tarantool",file_name="init.lua",thread_pid="1",kind="system"} 920
tnt_cpu_thread{thread_name="coio",file_name="init.lua",thread_pid="11",kind="user"} 79
tnt_cpu_thread{thread_name="coio",file_name="init.lua",thread_pid="699",kind="user"} 44
tnt_cpu_thread{thread_name="coio",file_name="init.lua",thread_pid="11",kind="system"} 294

Prometheus query aggregated by thread name

sum by (thread_name) (idelta(tnt_cpu_thread[$__interval]))
  / scalar(idelta(tnt_cpu_total[$__interval]) / tnt_cpu_count)

Below are examples of using metrics primitives.

Notice that this usage is independent of export-plugins such as Prometheus / Graphite / etc. For documentation on plugins usage, see their the Metrics plugins section.

Using counters:

local metrics = require('metrics')

-- create a counter
local http_requests_total_counter = metrics.counter('http_requests_total')

-- somewhere in the HTTP requests middleware:
http_requests_total_counter:inc(1, {method = 'GET'})

Using gauges:

local metrics = require('metrics')

-- create a gauge
local cpu_usage_gauge = metrics.gauge('cpu_usage', 'CPU usage')

-- register a lazy gauge value update
-- this will be called whenever the export is invoked in any plugins
metrics.register_callback(function()
    local current_cpu_usage = math.random()
    cpu_usage_gauge:set(current_cpu_usage, {app = 'tarantool'})
end)

Using histograms:

local metrics = require('metrics')

-- create a histogram
local http_requests_latency_hist = metrics.histogram(
    'http_requests_latency', 'HTTP requests total', {2, 4, 6})

-- somewhere in the HTTP requests middleware:
local latency = math.random(1, 10)
http_requests_latency_hist:observe(latency)

Using summaries:

local metrics = require('metrics')

-- create a summary with a window of 5 age buckets and 60s bucket lifetime
local http_requests_latency = metrics.summary(
    'http_requests_latency', 'HTTP requests total',
    {[0.5]=0.01, [0.9]=0.01, [0.99]=0.01},
    {max_age_time = 60, age_buckets_count = 5}
)

-- somewhere in the HTTP requests middleware:
local latency = math.random(1, 10)
http_requests_latency:observe(latency)