Splunk Search

Getting a timechart

smomin
New Member

Hello,

I have following query, from which I am able to produce a table

However, the above query doesn't get results by hostname.
index=prod host=hostname* source="/home/logs/log" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 | table _time,action,ms | rename ms as "duration[ms]" | sort -_time

Summary:
are DELETE, ADD, MODIFY,SEARCH

What we have is 3 host, hostname01, 02 and 03. I am looking to generate a 'timechart' where I would obtain a taken on a host and the time it took to complete.

Any suggestion?

one tested but not getting the result

index=prod host=hostname* source="/home/logs/log" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 action="REMOVE" | timechart avg(ms) avg(action) | rename ms as "duration[ms]" | sort -_time

Thank you in advance for assistance.

Tags (1)
0 Karma

cramasta
Builder

This will give you the max ms for each host/action/time window

index=prod host=hostname* source="/home/logs/log" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 action="REMOVE" | eval host_action=host."_".action| timechart max(ms) AS max_duration_ms by host_action limit=30

smomin
New Member

Awesome. This worked.

Questions, when we are doing max(ms) in combination with limit=30, shouldn't it return 30 entries with MAXIMUM MS time?

If above statement is correct, then I am not getting top 30 entries.

In the 'event' output I am seeing 425 events occur, but its showing 46 entries under statistics.

Is there configuration issue on splunk end?

Hope you to hear from you soon.

Thank you so much for all your time.

0 Karma

cramasta
Builder

By defualt timechart only creates 10 unique series of your split by clause (everything else gets grouped into OTHER category. In this case the split by clause would be the host_action field. That means if you have 5 hosts with 3 possible actions you would have 15 total series to chart. Setting limit=30 tells time chart to create UP TO 30 unique series if they exist. Its just a upper limit to adhear to.

The output means that 425 events were found but timechart processed them down into to 46 results (or rows). Each row is a is a unique time span

0 Karma

cramasta
Builder

If this helped you please mark this Answered. Thanks

0 Karma

somesoni2
Revered Legend

Try something like this
avg(ms) for all actions

index=prod host=hostname* source="/logs/log*" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>30 | timechart avg(ms) by host

OR avg(ms for a single action

index=prod host=hostname* source="/logs/log*" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>30 ACTION="ProvideActionNameHere" | timechart avg(ms) by host
0 Karma

smomin
New Member

Hello Somesoni2,

Per my understanding avg(ms) would give 'average ms (time). instead of 'avg', how can I get the chart to list ALL the 'actions' occurring in MS by host.

index=prod host=hostname* source="/logs/log*"| rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 action=REMOVE OR MODIFY OR DELETE OR ADD OR SEARCH |chart avg(ms) by action host

Regards,
Sayena

0 Karma

cramasta
Builder

I'm guessing the site is removing the field names from the Rex commands?

0 Karma

cramasta
Builder

dont really understand what you are asking for but how about this?
index=prod host=hostname* source="/home/logs/log" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 action="REMOVE" | timechart avg(ms) AS avg_duration_ms by host

0 Karma

smomin
New Member

Hello cramasta,

Thankyou for your response.

Since my query here I have developed this:

index=prod host=hostname* source="/logs/log*" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>30 action=REMOVE OR MODIFY OR DELETE OR ADD OR SEARCH | chart avg(ms) by action host limit=100

However, with above I am getting the list by host, but it is giving me AVERAGE as avg(ms). I want to be able to ALL the ms time for all actions.

Regards,
Sayena

0 Karma

cramasta
Builder

Still not very clear what you want the final output to be but see if this is any closer
index=prod host=hostname* source="/home/logs/log" | rex field=_raw "TIME\s:\s(?[^\s]*)\s(?\d+)\smsec" | search ms>0 action="REMOVE" | eval host_action=host."_".action| timechart avg(ms) AS avg_duration_ms by host_action

Please replaced the Rex command with your original as I think the site is removing the field name assignments.

0 Karma

smomin
New Member

Thank you. This is getting closer. However, timechart avg(ms) AS avg_duration_ms by host_action

Instead of getting avg(ms), how can I get it to return top highest ms in the time window. I tried top limit =10

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...