topic JSON element names contains dynamic part - how to create table in Getting Data In

JSON element names contains dynamic part - how to create table

pszpor — Fri, 08 Dec 2017 07:40:24 GMT

My JSON log file contains metrics - below message example. Json elements name and number are not fixed. As you can see element meters.bytesInPerSec.APPLICATION_NAME can be repeated for all applications dynamically.

How should I configure props.conf and how should I write query to get following table:

application, m5_rate
my-application-name-a, 0
my-application-name-b, 0
my-application-name-c, 18.081537604791322
my-application-name-d, 0

If such question was already answered please point me to proper doc/answer.

Eg message:
{ "time": 1512637302765, "meters": { "bytesInPerSec.my-application-name-a": { "count": 0, "m1_rate": 0, "m5_rate": 0, "m15_rate": 0, "mean_rate": 0, "units": "bytes/SECONDS" }, "bytesInPerSec.my-application-name-b": { "count": 0, "m1_rate": 0, "m5_rate": 0, "m15_rate": 0, "mean_rate": 0, "units": "bytes/SECONDS" }, "bytesInPerSec.my-application-name-c": { "count": 152503217, "m1_rate": 16.733471413145928, "m5_rate": 17.948078497437745, "m15_rate": 18.081537604791322, "mean_rate": 39.63002548338987, "units": "bytes/SECONDS" }, "bytesInPerSec.my-application-name-d": { "count": 0, "m1_rate": 0, "m5_rate": 0, "m15_rate": 0, "mean_rate": 0, "units": "bytes/SECONDS" } } }

Re: JSON element names contains dynamic part - how to create table

harsmarvania57 — Fri, 08 Dec 2017 13:43:20 GMT

Hi @pszpor,

You can achieve this via search query based on sample data you have provided please try below query (First 2 lines are used to generate dummay data)

| makeresults
| eval _raw="{ \"time\": 1512637302765, \"meters\": { \"bytesInPerSec.my-application-name-a\": { \"count\": 0, \"m1_rate\": 0, \"m5_rate\": 0, \"m15_rate\": 0, \"mean_rate\": 0, \"units\": \"bytes/SECONDS\" }, \"bytesInPerSec.my-application-name-b\": { \"count\": 0, \"m1_rate\": 0, \"m5_rate\": 0, \"m15_rate\": 0, \"mean_rate\": 0, \"units\": \"bytes/SECONDS\" }, \"bytesInPerSec.my-application-name-c\": { \"count\": 152503217, \"m1_rate\": 16.733471413145928, \"m5_rate\": 17.948078497437745, \"m15_rate\": 18.081537604791322, \"mean_rate\": 39.63002548338987, \"units\": \"bytes/SECONDS\" }, \"bytesInPerSec.my-application-name-d\": { \"count\": 0, \"m1_rate\": 0, \"m5_rate\": 0, \"m15_rate\": 0, \"mean_rate\": 0, \"units\": \"bytes/SECONDS\" } } }"
| spath
| rename meters.bytesInPerSec.my-application-name-*.m15_rate AS my-application-name-*
| table my-application-name*
| transpose column_name=application
| rename "row 1" AS m15_rate

I hope this helps.

Thanks,
Harshil