Transaction to Streamstats

[tb5821]

I'm trying to move from using a transaction command to a streamstats - I get most of the way there but I can't figure out the 'reset' with streamstats to proper group starts and ends together. Here's my current query which isn't finding the right end time since my BY query will match multiple places and the latest(_time) will change to the wrong latest time but yet the actual latest time ...

index="logfiles" appType=reports* "Generating * Status Report" | rex \.(?<reportName>Generate\w*) 
| eval reportName=replace (reportName,"Generate","")
| eval reportName=replace (reportName,"Report","")
| streamstats earliest(_time) as stime by reportName appType
| join appType,reportName [search index="logfiles" appType=reports* "Report generated successfully" | rex \.(?<reportName>\w+)ReportGenerator
| streamstats reset_on_change=true reset_after="("searchmatch(\"Report generated successfully\")")" latest(_time) as etime BY reportName appType]
| eval diff=etime-stime
| eval hhmmss=tostring(diff, "duration")
| convert timeformat=" %a  %b  %d %I:%M:%S.%3N %p %Z" ctime(stime) as StartTime
| table StartTime appType reportName hhmmss
|rename hhmmss as RunDuration

[niketn]

@tb5821 for the community to assist you better please provide more details like some sample (cooked up event, should have masking and anonymization for any sensitive data). Also provide your current SPL with transaction command which is also doing the job (but I am guessing it is not performing).

[tb5821]

anyone?

[ITWhisperer]

@tb5821 Do you have some sample events you can share to show what you are dealing with?

[tb5821]

the events are long and span hours if not close to a day... but the beginning of the event starts with the generating line and ends with a completed log line. The problem is that if server 1 picks up job A and runs for 28hrs. Then server 2 will pick up job A 12 hours into server 1's run. This is fine b/c they are different servers. BUT now the issue comes in ~24hours after server1 starts is that its time for the next run, and server 1 again on a different thread picks up that run. So its the same server1 and same job A but its start and end should be different.

Yet whats happening is my streamstats for the first event computes the event end as the end for the last (latest) run.

[ITWhisperer]

Hi @tb5821 

By events I meant splunk event e.g. log entries. What is the data you are trying to process with splunk?