Reporting

Transactions With Variable start/end?

dmoulais
New Member

I have a log with many instances of the following ... these can happen in parallel. I was attempting to use a transaction to mine these via startswith="Received Query" and endswith="Completed Query".

<2018.10.29 10:02:37 639 -0400> Received Query on NE:38.120.48.29,{"targetClass":"nsd-service:/services/eline-sites/site","operator":"=","field":"classId","value":"nsd-service:/services/eline-sites/site"}, For BOTH Attributes
<2018.10.29 10:02:37 639 -0400> Received Query on NE:38.120.48.29,{"targetClass":"nsd-service:/services/eline-sites/site","operator":"=","field":"classId","value":"nsd-service:/services/eline-sites/site"}, For BOTH Attributes
<2018.10.29 10:02:37 639 -0400> Received Query on NE:38.120.48.29,{"targetClass":"nsd-service:/services/eline-sites/site","operator":"=","field":"classId","value":"nsd-service:/services/eline-sites/site"}, For BOTH Attributes
<2018.10.29 10:02:38 696 -0400> Completed Query: NE 38.120.48.29 Target Class nsd-service:/services/eline-sites/site
<2018.10.29 10:02:38 696 -0400> Completed Query: NE 38.120.48.29 Target Class nsd-service:/services/eline-sites/site
<2018.10.29 10:02:38 696 -0400> Completed Query: NE 38.120.48.29 Target Class nsd-service:/services/eline-sites/site

The problem is that since these can happen in parallel, I can mismatch start and end events. The way you know which ones match are based on the thread number ... 24 or 25 or 26 in this case. Is there a way to extract that dynamic number and use it to form the transaction ...

transaction startswith="Received Query" and endswith="Completed Query" ... and "contains" the dynamically generated number?

Tags (1)
0 Karma

kmaron
Motivator

is the thread number in your event? or can you pull it out with a regex? If you can you could do something like this.

| transaction ThreadNum  startswith="Received Query" endswith="CompletedQuery" maxevents=2

dmoulais
New Member

Yes this works thanks - I frequently forget you can use a field name as a transaction grouping 🙂

index=xxhost=vm4 source="/opt/nsp/mediation/log/xx.log" "xx-grpc-exec" AND ( "Received Query" OR "Completed Query" ) | rex "-exec[(?\d+)]" | transaction threadId startswith="Received Query" endswith="Completed Query" | chart values(duration) as "Query Time" by _time

0 Karma

kmaron
Motivator

perfect! I'll convert it to an answer so you can accept it.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...