Grafana dashboard

Tarantool Grafana dashboard is available as part of Grafana Official & community built dashboards. There’s a version for Prometheus data source and one for InfluxDB data source. There are also separate dashboards for TDG applications: for Prometheus data source and for InfluxDB data source. Tarantool Grafana dashboard is a ready for import template with basic memory, space operations, and HTTP load panels, based on default metrics package functionality.

Dashboard requires using metrics 0.15.0 or newer for complete experience; 'alias' global label must be set on each instance to properly display panels (e.g. provided with cartridge.roles.metrics role).

To support CRUD statistics, install CRUD 0.11.1 or newer. Call crud.cfg on router to enable CRUD statistics collect with latency quantiles.

crud.cfg{
    stats = true,
    stats_driver='metrics',
    stats_quantiles=true
}

To support expirationd statistics, install expirationd 1.2.0 or newer. Call expirationd.cfg on instance to enable statistics export.

expirationd.cfg{metrics = true}

../../../_images/Prometheus_dashboard_1.png

../../../_images/Prometheus_dashboard_2.png

../../../_images/Prometheus_dashboard_3.png

Prepare a monitoring stack

Since there are Prometheus and InfluxDB data source Grafana dashboards, you can use

Telegraf as a server agent for collecting metrics, InfluxDB as a time series database for storing metrics, and Grafana as a visualization platform; or
Prometheus as both a server agent for collecting metrics and a time series database for storing metrics, and Grafana as a visualization platform.

For issues concerning setting up Prometheus, Telegraf, InfluxDB, or Grafana instances please refer to the corresponding project’s documentation.

Collect metrics with server agents

To collect metrics for Prometheus, first set up metrics output with prometheus format. You can use cartridge.roles.metrics configuration or set up the Prometheus output plugin manually. To start collecting metrics, add a job to Prometheus configuration with each Tarantool instance URI as a target and metrics path as it was configured on Tarantool instances:

scrape_configs:
  - job_name: tarantool
    static_configs:
      - targets:
        - "example_project:8081"
        - "example_project:8082"
        - "example_project:8083"
    metrics_path: "/metrics/prometheus"

To collect metrics for InfluxDB, use the Telegraf agent. First off, configure Tarantool metrics output in json format with cartridge.roles.metrics configuration or corresponding JSON output plugin. To start collecting metrics, add http input to Telegraf configuration including each Tarantool instance metrics URL:

[[inputs.http]]
    urls = [
        "http://example_project:8081/metrics/json",
        "http://example_project:8082/metrics/json",
        "http://example_project:8083/metrics/json"
    ]
    timeout = "30s"
    tag_keys = [
        "metric_name",
        "label_pairs_alias",
        "label_pairs_quantile",
        "label_pairs_path",
        "label_pairs_method",
        "label_pairs_status",
        "label_pairs_operation",
        "label_pairs_level",
        "label_pairs_id",
        "label_pairs_engine",
        "label_pairs_name",
        "label_pairs_index_name",
        "label_pairs_delta",
        "label_pairs_stream",
        "label_pairs_thread",
        "label_pairs_kind"
    ]
    insecure_skip_verify = true
    interval = "10s"
    data_format = "json"
    name_prefix = "tarantool_"
    fieldpass = ["value"]

Be sure to include each label key as label_pairs_<key> so it will be extracted with plugin. For example, if you use { state = 'ready' } labels somewhere in metric collectors, add label_pairs_state tag key.

For TDG dashboard, please use

[[inputs.http]]
    urls = [
        "http://example_tdg_project:8081/metrics/json",
        "http://example_tdg_project:8082/metrics/json",
        "http://example_tdg_project:8083/metrics/json"
    ]
    timeout = "30s"
    tag_keys = [
        "metric_name",
        "label_pairs_alias",
        "label_pairs_quantile",
        "label_pairs_path",
        "label_pairs_method",
        "label_pairs_status",
        "label_pairs_operation",
        "label_pairs_level",
        "label_pairs_id",
        "label_pairs_engine",
        "label_pairs_name",
        "label_pairs_index_name",
        "label_pairs_delta",
        "label_pairs_stream",
        "label_pairs_thread",
        "label_pairs_type",
        "label_pairs_connector_name",
        "label_pairs_broker_name",
        "label_pairs_topic",
        "label_pairs_request",
        "label_pairs_kind",
        "label_pairs_thread_name",
        "label_pairs_type_name",
        "label_pairs_operation_name",
        "label_pairs_schema",
        "label_pairs_entity",
        "label_pairs_status_code"
    ]
    insecure_skip_verify = true
    interval = "10s"
    data_format = "json"
    name_prefix = "tarantool_"
    fieldpass = ["value"]

If you connect Telegraf instance to InfluxDB storage, metrics will be stored with "<name_prefix>http" measurement ("tarantool_http" in our example).

Import the dashboard

Open Grafana import menu.

To import a specific dashboard, choose one of the following options:

paste the dashboard id (12567 for InfluxDB dashboard, 13054 for Prometheus dashboard, 16405 for InfluxDB TDG dashboard, 16406 for Prometheus TDG dashboard), or
paste a link to the dashboard ( https://grafana.com/grafana/dashboards/12567 for InfluxDB dashboard, https://grafana.com/grafana/dashboards/13054 for Prometheus dashboard, https://grafana.com/grafana/dashboards/16405 for InfluxDB TDG dashboard, https://grafana.com/grafana/dashboards/16406 for Prometheus TDG dashboard), or
paste the dashboard JSON file contents, or
upload the dashboard JSON file.

Set dashboard name, folder and uid (if needed).

../../../_images/grafana_import_setup.png

You can choose datasource and datasource variables after import.

../../../_images/grafana_variables_setup.png

Troubleshooting

If there are no data on the graphs, make sure that you picked datasource and job/measurement correctly.

If there are no data on the graphs, make sure that you have info group of Tarantool metrics (in particular, tnt_info_uptime).

If some Prometheus graphs show no data because of parse error: missing unit character in duration, ensure that you use Grafana 7.2 or newer.

If some Prometheus graphs display parse error: bad duration syntax "1m0" or similar error, you need to update your Prometheus version. See grafana/grafana#44542 for more details.

Версия:

Grafana dashboard

Prepare a monitoring stack

Collect metrics with server agents

Import the dashboard

Troubleshooting