Splunk Search

I would like to take the hosts field from the below search and show the most recent event from each host

mwcentracomm
Explorer

I created an alert from the search below, and it emails a pdf - is there a way to add the most recent event from each of the hosts in this search and add it to the email?

 

metadata type=hosts | where recentTime < now() - 10800| eval lastSeen = strftime(recentTime, "%F %T") | fields + host lastSeen

Labels (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Not in this search.

Remember that at each pipe you get only the results from subsequent commands. So if the metadata command gives you just some, well, metadata, you're not having any events.

You'd need to either run a very wide search and do stats latest(_raw) by host, which is not a very good idea or either use this one as part of a subsearch to generate time limiting conditions or pass this through map command. Both solutions aren't very pretty.

 

mwcentracomm
Explorer

Would there be a way to to search for the first host not reporting and see the most recent event and create an alert on that, and do it for the second and the third and so forth?

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Yes. But.

1. As this is a metadata search for hosts only, you'd have to search across all your indexes - you have no indication of which index that event is in.

2. It'd be a very ugly and resource intensive search spawning possibly huge number of subsearches. It's a good way to overstress your Splunk environment. That's why the map command is considered a risky command and not enabled for ordinary users by default.

But in general I think you're trying to solve a completely different problem here. The issue of finding hosts which stopped reporting was tackled many times already and there are even specific apps for this - like TrackMe - https://splunkbase.splunk.com/app/4621

0 Karma

mwcentracomm
Explorer

Yes, that is what I am trying to do, and I obviously have the host, is there a way to extract this out of the event to add to the alert?

 

Jan 4 13:07:57 HOST 1 2024-01-04T13:07:57.085-05:00 somenumber-somename-ks-srx rpd 2188 JTASK_SIGNAL_INFO [junos@2636.1.1.1.2.133 message-name="INFO Signal Info: Signal Number = " signal-number="1" name=" Consumed Count = " data-1="3"]

Thanks for all the help

0 Karma

yuanliu
SplunkTrust
SplunkTrust

How many hosts are typically returned?  If there are not many, you can just use metadata to filter index search.  This would meet your original requirement.

 

[metadata type=hosts | where recentTime < now() - 10800| stats values(host) as host]
| dedup host

 

If there are too many, performance can be a concern.  (You can also add other filters in addition to | metadata.)

As @PickleRick noted, you probably don't want to send raw events, especially not a lot of them, in E-mail.  In theory, you SHOULD have this "somenumber-somename-ks-srx" extracted in a field it means something.  Haven't you?  Assuming the field name is somefield.

 

[metadata type=hosts | where recentTime < now() - 10800| stats values(host) as host]
| dedup host
| table host _time somefield

 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...