Splunk Search

Kafka regex: Why is the command not working in Splunk search?

pswalia06
Explorer
{"topic": "amx", "total_lag": 2670, "partitions": [{"lag": 117, "partition_number": 0}, {"lag": 122, "partition_number": 1}, {"lag": 130, "partition_number": 2}, {"lag": 130, "partition_number": 3}, {"lag": 148, "partition_number": 4}, {"lag": 144, "partition_number": 5}, {"lag": 158, "partition_number": 6}, {"lag": 130, "partition_number": 7}, {"lag": 123, "partition_number": 8}, {"lag": 145, "partition_number": 9}, {"lag": 130, "partition_number": 10}, {"lag": 127, "partition_number": 11}, {"lag": 123, "partition_number": 12}, {"lag": 121, "partition_number": 13}, {"lag": 118, "partition_number": 14}, {"lag": 125, "partition_number": 15}, {"lag": 133, "partition_number": 16}, {"lag": 161, "partition_number": 17}, {"lag": 134, "partition_number": 18}, {"lag": 151, "partition_number": 19}]}


index=orion-platform  source="/opt/bda/logs/kafkalag.log" |spath output=AA path=counterList{1} | rex field=AA "\"lag\":\s(?.\w+)\,\s\"partition_number\"\:\s(?\d+)\}" max_match=100 | table State1,partition_number

Above command not working in splunk search.

0 Karma
1 Solution

kamlesh_vaghela
SplunkTrust
SplunkTrust

Hi

Can you please try the following search? I haven't used any regular expression but it will give you all the data from JSON event.

YOUR_SEARCH |
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| eval temp = mvzip(lag,partition_number) | stats count by _time total_lag,topic,temp
| eval lag = mvindex(split(temp,","),0) ,partition_number=mvindex(split(temp,","),1) | table topic total_lag lag partition_number

My Sample Search:

| makeresults 
| eval _raw="{\"topic\": \"amx\", \"total_lag\": 2670, \"partitions\": [{\"lag\": 117, \"partition_number\": 0}, {\"lag\": 122, \"partition_number\": 1}, {\"lag\": 130, \"partition_number\": 2}, {\"lag\": 130, \"partition_number\": 3}, {\"lag\": 148, \"partition_number\": 4}, {\"lag\": 144, \"partition_number\": 5}, {\"lag\": 158, \"partition_number\": 6}, {\"lag\": 130, \"partition_number\": 7}, {\"lag\": 123, \"partition_number\": 8}, {\"lag\": 145, \"partition_number\": 9}, {\"lag\": 130, \"partition_number\": 10}, {\"lag\": 127, \"partition_number\": 11}, {\"lag\": 123, \"partition_number\": 12}, {\"lag\": 121, \"partition_number\": 13}, {\"lag\": 118, \"partition_number\": 14}, {\"lag\": 125, \"partition_number\": 15}, {\"lag\": 133, \"partition_number\": 16}, {\"lag\": 161, \"partition_number\": 17}, {\"lag\": 134, \"partition_number\": 18}, {\"lag\": 151, \"partition_number\": 19}]}" 
| kv
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| eval temp = mvzip(lag,partition_number) | stats count by _time total_lag,topic,temp
| eval lag = mvindex(split(temp,","),0) ,partition_number=mvindex(split(temp,","),1) | table topic total_lag lag partition_number

Please let me know if assistance required.

Thanks

View solution in original post

0 Karma

pswalia06
Explorer

alt text

Here i have one more problem. If you see the below table topic name it is amx and amx1 but when i do line charts instead of showing two lines one for amx and one for amx1 it is showing only one line. How can we separate them?

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

HI @pswalia06,

Can you please try the following search?

YOUR_SEARCH
|kv
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| timechart latest(total_lag) as total_lag by topic

0 Karma

ddrillic
Ultra Champion

Is there a way to convert this feed to a json format? It's pretty close....

pswalia06
Explorer

it is json format only

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

Hi

Can you please try the following search? I haven't used any regular expression but it will give you all the data from JSON event.

YOUR_SEARCH |
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| eval temp = mvzip(lag,partition_number) | stats count by _time total_lag,topic,temp
| eval lag = mvindex(split(temp,","),0) ,partition_number=mvindex(split(temp,","),1) | table topic total_lag lag partition_number

My Sample Search:

| makeresults 
| eval _raw="{\"topic\": \"amx\", \"total_lag\": 2670, \"partitions\": [{\"lag\": 117, \"partition_number\": 0}, {\"lag\": 122, \"partition_number\": 1}, {\"lag\": 130, \"partition_number\": 2}, {\"lag\": 130, \"partition_number\": 3}, {\"lag\": 148, \"partition_number\": 4}, {\"lag\": 144, \"partition_number\": 5}, {\"lag\": 158, \"partition_number\": 6}, {\"lag\": 130, \"partition_number\": 7}, {\"lag\": 123, \"partition_number\": 8}, {\"lag\": 145, \"partition_number\": 9}, {\"lag\": 130, \"partition_number\": 10}, {\"lag\": 127, \"partition_number\": 11}, {\"lag\": 123, \"partition_number\": 12}, {\"lag\": 121, \"partition_number\": 13}, {\"lag\": 118, \"partition_number\": 14}, {\"lag\": 125, \"partition_number\": 15}, {\"lag\": 133, \"partition_number\": 16}, {\"lag\": 161, \"partition_number\": 17}, {\"lag\": 134, \"partition_number\": 18}, {\"lag\": 151, \"partition_number\": 19}]}" 
| kv
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| eval temp = mvzip(lag,partition_number) | stats count by _time total_lag,topic,temp
| eval lag = mvindex(split(temp,","),0) ,partition_number=mvindex(split(temp,","),1) | table topic total_lag lag partition_number

Please let me know if assistance required.

Thanks

0 Karma

pswalia06
Explorer

The amx value is showing continuesly and the total_lag is showing the same repeated value for each lag and partition_name

I want this
Topic_name total_lag partition_number lag
amx 240. 0. 20
1. 30

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@pswalia06

Are you looking for this?

| makeresults 
| eval _raw="{\"topic\": \"amx\", \"total_lag\": 2670, \"partitions\": [{\"lag\": 117, \"partition_number\": 0}, {\"lag\": 122, \"partition_number\": 1}, {\"lag\": 130, \"partition_number\": 2}, {\"lag\": 130, \"partition_number\": 3}, {\"lag\": 148, \"partition_number\": 4}, {\"lag\": 144, \"partition_number\": 5}, {\"lag\": 158, \"partition_number\": 6}, {\"lag\": 130, \"partition_number\": 7}, {\"lag\": 123, \"partition_number\": 8}, {\"lag\": 145, \"partition_number\": 9}, {\"lag\": 130, \"partition_number\": 10}, {\"lag\": 127, \"partition_number\": 11}, {\"lag\": 123, \"partition_number\": 12}, {\"lag\": 121, \"partition_number\": 13}, {\"lag\": 118, \"partition_number\": 14}, {\"lag\": 125, \"partition_number\": 15}, {\"lag\": 133, \"partition_number\": 16}, {\"lag\": 161, \"partition_number\": 17}, {\"lag\": 134, \"partition_number\": 18}, {\"lag\": 151, \"partition_number\": 19}]}" 
| kv
| rename partitions{}.lag as lag, partitions{}.partition_number as partition_number
| table topic total_lag  partition_number lag
0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...