Knowledge Management

is there a way to extract raw event data from a summary index?

sahanapranesh
New Member

Please, could you let me know if there is a way to extract the raw data of an event from the summary index report?
I have a summary index report like:

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) by field1 field2

On top of which we fire a search:

index=summary report = si_SummaryIndexReport | stats min(_time) as earliest max(_time) as latest by field1 field2 | where latest>earliest | eval duration=(latest-earliest)/86400 | table field1 field2 earliest latest duration

Basically I want to calculate the time between the first occurrance of Operation1 and the last occurrance of Operation3.
The behavior should be similar to transaction command where we give the "startsWith" and "endsWith" attributes.
Due to performance issues I cannot use the transaction command.

This search behaves in the following way
Scenario 1: Logs for Success case
15/08/2015: Operation1 for field1=112233 –min(_time) will show 15/08/2015
16/08/2015: Operation2 for field1=112233
20/08/2015: Operation1 for field1=112233
26/08/2015: Operation3 for field1=112233 –max(_time) will show 26/08/15

This is the expected behavior and the search works fine for scenario 1.

Scenario 2: Logs for Failure case
15/08/2015: Operation1 for field1=112233 –min(_time) will show 15/08/2015
16/08/2015: Operation2 for field1=112233
20/08/2015: Operation1 for field1=112233 –max(_time) will show 20/08/2015

In this case there have been 2 events for Operation1 alone, hence it should not even be considered for the final table.
So, please let me know if there’s a way of extracting event data or repopulating an already existing summary index with a newly extracted field, say status, where status could be “Operation1” “Operation2” or “Operation3”
I want min(_time) to ALWAYS be a “Operation1” event and a max(_time) to ALWAYS be a “Operation3” event.

Thanks in advance!
Sahana

0 Karma
1 Solution

somesoni2
Revered Legend

Try this for your summary index search (assuming you've a field called Operation already extracted)

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) values(Operation) as Operations by field1 field2 | where isnotnull(mvfilter(match(Operations,"Operation3"))) | fields - Operations

This way, if your grouping of field1 and field2 will be discarded if it doesn't include Operation3

View solution in original post

0 Karma

somesoni2
Revered Legend

Try this for your summary index search (assuming you've a field called Operation already extracted)

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) values(Operation) as Operations by field1 field2 | where isnotnull(mvfilter(match(Operations,"Operation3"))) | fields - Operations

This way, if your grouping of field1 and field2 will be discarded if it doesn't include Operation3

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...