Splunk Search

set of last events for several tags

wsw70
Communicator

Hello,

I am trying to find a way to analyze the last occurrence of different events. The data I work with is structured as follows

 T1 S1 some event
 T1 S1 some different event
 T2 S2 another different event
 T3 S2 and so on

The events come from scans and I forcefully set a unique timestamp (Tn above) for each scan. Each scan is of type of scans (Sn above). In practical terms there is a given scan of a building (S3 for instance - which means "a scan of building 3") and they happen at various times over the month (thus several timestamps T5, T6, ...)

I need to find a way to work only on the last timestamp for each type of scan, all mixed up.
In other words I would like to extract from the example above

T1 S1 some event
T1 S1 some different event
T3 S2 and so on

(assuming that T3 > T2)

The idea would be to:

  • extract a list of unique scan types (S1, S2 in my case), called ScanTypes
  • and then do a last(ScanTypes) which would extract all the latest events for each of the scan types

The documentation for last mentions that it is used with a field name, not a table of field names (?) (assuming that the concept of tables can be used here)

An alternative could be a loop (in pseudo code)

foreach Scan in ScanTypes
  last(Scan)

but I do not know if this possible in splunk.

Any idea how to attack this would be really welcome.

Tags (2)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

Okay, given the clarification, the following will work, though there may be more efficient ways to do it depending on your specific data:

... | eventstats latest(_time) as ltime by ScanType | where _time=ltime

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Okay, given the clarification, the following will work, though there may be more efficient ways to do it depending on your specific data:

... | eventstats latest(_time) as ltime by ScanType | where _time=ltime

wsw70
Communicator

It works great - thank you!

0 Karma

wsw70
Communicator

A scan will generate many events (say, of type S1) for a one given timestamp (say T1) - I need to get them all for that timestamp (in my example these were the two T1 S1 matched). There are many scans for a given type (each of them having many events) and I need the last one (the one with the latest timestamp (closest to "now")). Finally, there are many types of scans (S1, S2, and so on) so - overall - at one given time I need all the events for the latest timestamps of all scan types (the list of which, as you suggested, could come from a dedup). Sorry if I was not clear.

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

But you want multiple items for S1, not just the last one?

0 Karma

wsw70
Communicator

What I meant is that I want all events with the last timestamp for all the scan types. I run several scans a month in several places (the Sn in my example). These scans generate, for a given timestamp, many events. I want the most recent ones. In my example T2 S2 was earlier than T3 S2, so only T2 S2 events are to be matched (+ the two T1 S1, as they are from another scan type).

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

I think that as long as the timestamps are picked up correctly by Splunk (and provided I understand what you're asking, which i am not sure I do, see my comment above), you can just do:

... | dedup ScanType

or more explictly:

... | dedup ScanType sortby -_time

snoobzilla
Builder

sort/dedup approach worked great for my use case, thanks!

0 Karma

wsw70
Communicator

that could be good - dedup would pick the last of unique scan types. But this will not catch the last events (many of them) for each of them, right? (see my clarification above)

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Your example confuses me. You say want only the last event for each type, but you say you want two events for ScanType S1 in your results. Did you make a mistake or am I misunderstanding?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...