Skip to content

Add new metrics

In this section we describe the process of adding new metrics in KOBE.

Step 1. Produce a log message that contains the metric

To add a new metric you must first modify your component to produce it. KOBE collects metrics by parsing log messages so to add the metric just add another log message in the component for this metric.

Except from the metric, your log message should contain the following information:

  • Experiment name
  • Start time of the experiment
  • Query name
  • Run

If the log message is produced by an evaluator, these parameters are obtained from the KOBE operator (see here). If the log message is produced by a federator, these parameters are obtained by a SPARQL comment (see here).

Step 2. Configure Fluentd with a corresponding regex pattern

All metrics are captured by Fluentd, so to collect the new metric you must edit the operator/deploy/efk-config/fluentd-values.yaml file and add another Fluentd filter at containers.input.conf for the introduced log pattern of the metric.

For example if the log pattern is

^Experiment: (?<experiment>[^ ]+) - Date: (?<date>[^ ]+ [^ ]+) - Query: (?<query>[^ ]+) - Run: (?<run>[0-9]+) - MyMetric: (?<my_metric>[0-9]+)$

you must add filter

<filter kubernetes.**>
  @type parser
  key_name message
  reserve_time true
  reserve_data true
  #suppress_parse_error_log true
  <parse>
    @type regexp
    expression ^Experiment: (?<experiment>[^ ]+) - Date: (?<date>[^ ]+ [^ ]+) - Query: (?<query>[^ ]+) - Run: (?<run>[0-9]+) - MyMetric: (?<my_metric>[0-9]+)$
    types my_metric:integer
  </parse>
</filter>

After redeploying Fluentd, every time the metric log message is produced, it will be captured by Fluentd and stored in ElasticSearch.

Step 3. Configure Kibana to visualize your metric

Finally, you must add a new Kibana Dashboard to visualize the new metric, or add a Visualization on one of the existing Dashboards.

Example

As an example, check out the Fluentd configuration for the metrics we currently support.