How to create a multi time chart of key value property?

[mskala]

Hi,

I have the following events in Splunk

{
  "field1": "something",
  "execution_times": {
    "service1": 100,
    "service2": 400,
    (...)
    "service_N": 600,
  },
  "field2": "something"
}

How can I create a multiline chart that would show p90 + p99 of each "service" in JSON map "execution_times" based on the values [here 100, 400, (...) 600].

The query should produce a chart with N*2 (for p90 and p99) different time series (lines) based on all "services" that were inside events.

Each event can contain different "services" in its execution_times map.

Thanks

[bowesmana]

How many 'services' will you have, as a timechart gets messy and difficult to manage beyond 20 or so lines (10 services).

Assuming your JSON auto extracts, then just use this line

| timechart limit=<your_max_limit> span=<your_span> perc90(execution_times.*) as p90_* perc99(execution_times.*) as p99_*

[mskala]

Thank you, that almost worked.

I will be using a Trellis layout to show a chart for each "service"

How can I tune this query to see p90 and p99 in a single chart?

The chart for service_1 should display p90 and p99 lines only for service_1.

Currently, in trellis, I am getting N*2 charts and I'd like to see just N.

Probably I need to set some split by but don't know how.

[bowesmana · Accepted solution]

I think that as all the services are in a single event, you need to expand out the events, so you can get an aggregation by service, so if you do this

| rename execution_times.* as *
| fields - field*
| timechart span=1m fixedrange=f perc90(*) as *_p90 perc99(*) as *_p99
| foreach service*_p* [ eval services=mvappend("service_<<MATCHSEG1>>".":"."<<MATCHSEG2>>".":".'<<FIELD>>',services) ]
| fields _time services
| mvexpand services
| rex field=services "(?<service>[^:]*):(?<p>[^:]*):(?<v>.*)"
| eval p_{p}=v
| fields - p services
| stats values(p*) as p* by _time service

which is taking the timechart and then expanding all the services to their own events.

At least this works for trellis if you then select service as the trellis aggregation

[mskala]

Hi,

Previous example almost work but it took all fields from event to create charts (like field1 and field2 from example event)

But after tuning it up it now works correctly.

Many thanks for your help @bowesmana .

Case closed.

| timechart span=1m fixedrange=f perc90(execution_times.*) as *_p90 perc99(execution_times.*) as *_p99
| foreach *_p* [ eval services=mvappend("service_<<MATCHSEG1>>".":"."<<MATCHSEG2>>".":".'<<FIELD>>',services) ]
| fields _time services
| mvexpand services
| rex field=services "(?<service>[^:]*):(?<p>[^:]*):(?<v>.*)"
| eval p_{p}=v
| fields - p services
| stats values(p*) as p* by _time service