How to arrange events into groups and count number of events in each group?

[hasham19833]

I have log lines like these:

2023/06/09 13:19:31.245 : AUDIT- INFO: Adding profile with id 00001 to TPT
2023/06/09 13:19:32.245 : AUDIT- INFO: Adding profile with id 00002 to TPT
2023/06/09 13:19:33.326 : Will stop adding profiles from id 00003 as maximum size has been exceeded

2023/06/09 13:19:34.245 : AUDIT- INFO: Adding profile with id 00003 to TPT

2023/06/09 13:19:34.245 : AUDIT- INFO: Adding profile with id 00003 to TPT
2023/06/09 13:19:35.245 : AUDIT- INFO: Adding profile with id 00004 to TPT
2023/06/09 13:19:36.326 : Will stop adding profiles from id 00005 as maximum size has been exceeded

2023/06/09 13:19:37.240 : AUDIT- INFO: Adding profile with id 00005 to TPT
2023/06/09 13:19:37.245 : AUDIT- INFO: Adding profile with id 00006 to TPT
2023/06/09 13:19:38.245 : AUDIT- INFO: Adding profile with id 00007 to TPT
2023/06/09 13:19:39.245 : AUDIT- INFO: Adding profile with id 00008 to TPT
2023/06/09 13:19:40.245 : AUDIT- INFO: Adding profile with id 00009 to TPT
2023/06/09 13:19:41.245 : AUDIT- INFO: Adding profile with id 00010 to TPT
2023/06/09 13:19:42.326 : Will stop adding profiles from id 00011 as maximum size has been exceeded

2023/06/09 13:19:43.245 : AUDIT- INFO: Adding profile with id 00011 to TPT
2023/06/09 13:19:44.245 : AUDIT- INFO: Adding profile with id 00012 to TPT
2023/06/09 13:19:45.245 : AUDIT- INFO: Adding profile with id 00013 to TPT
2023/06/09 13:19:46.245 : AUDIT- INFO: Adding profile with id 00014 to TPT

I want to group the events starting from "Adding profile with ID" and completing the group with "will stop adding profiles", and all messages in one group should be visible...so that i have 3 groups in total, and then the last 4 messages should not be a part of any group ( as their group has not completed yet)

The results should look something like this:

Group 1: profiles total:2 completed
Group 2: profiles total:2 completed
Group 3: profiles total:6 completed
Group 4: profiles total:4 -

[hasham19833]

i tried transaction but this only includes the 2 imediate events...i want to include all the events between the 2 "will stop adding profiles" messages, not just the immediate before the end.

[gcusello]

Hi @hasham19833,

please try this:

<your_search>
| transaction endswith="will stop adding profiles"

Ciao.

Giuseppe

[gcusello]

Hi @hasham19833,

this is one of the few situations to use transaction command:

<your_search>
| transaction startswith="Adding profile with ID" endswith="will stop adding profiles"
| table _time eventcount

Ciao.

Giuseppe

[PickleRick]

But you should be able to do it without transaction assuming that events are in order (for transaction you need this assumption as well since you don't have any "tracking field").

Just use streamstats with proper stop criteria.