Splunk Search

StartsWith breaks the transaction

gmasy
New Member

Hello everyone,

I am trying to extract some data from the logs.
I have created a little search that works well:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId

It links the request and the response from the server by the extracted field RequestId.
alt text

However, I am trying to filter only some parameter contained in the field Login:
alt text

The login has many possibilities of different values, but it starts with either:

  • "BBC-*"
  • "BBF-*"
  • "BFL-*"
  • "BCIMR-*"

The login is only in the request, and not in the response.
I am trying to filter by putting the following

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith="BCIMR-*"

However, when I add the StartsWith, everything's broken, as the events are not linked together anymore.
See following comment for example as I can't add more than 2 images per question

0 Karma
1 Solution

to4kawa
Ultra Champion
customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
| rex "(?<status>LEAN-(REQUEST|RESPONSE))"
| stats dc(status) as flag values(_raw) as _raw values(_time) as _time by RequestId
| where flag > 1

you don't need transaction

transaction command consumes much search time.
stats is almost useful and no problem.
because stats makes events to combine one event by same field value.
Isn't it the same as the transaction command?

my previous answer has a few typo(statsu). I fix it.

Is there the login field? please try again.

View solution in original post

0 Karma

gmasy
New Member

Thanks to the three of you for your quick input about my problem!
I'm happily surprised of the reactivity on this forum, as this was my very first post...

However, none of the solutions provided did work for me.
@PavelP, although your solution seemed great, I got the same result as attached in previous screenshot. The response and request are not grouped in the same event when adding the startswith ...

@to4kawa and @woodcock , thank you for your suggestions, could you please explain me why transaction is overkill for what I am trying to achieve?
It's the only solution I have found in the Splunk documentation to match two corresponding events in the same splunk result.
It enables me to see what is in each request and what is the associated response.
stats here is not the thing I am trying to achieve just yet.

Thank you

0 Karma

to4kawa
Ultra Champion

I see. my answer is updated. please check and try.

0 Karma

woodcock
Esteemed Legend

Ditch transaction; it is overkill and does not scale well; try this:

index="YouShouldAlwaysSepcifyIndex" AND sourcetype="AndSourcetypeToo" AND "customergetservice" AND host="MBKBKKSPHTRSP0*" AND source="/var/log/jbossas/standalone/server.log"
| rex "LEAN-(?<status>REQUEST|RESPONSE)"
| reverse
| streamstats count(eval(searchmatch("BCIMR-*"))) AS TransactionID BY RequestId
| stats range(_time) AS duration count AS eventcount values(status) AS status dc(status) AS statuscount min(_time) AS _time BY TransactionID RequestId
0 Karma

to4kawa
Ultra Champion
customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
| rex "(?<status>LEAN-(REQUEST|RESPONSE))"
| stats dc(status) as flag values(_raw) as _raw values(_time) as _time by RequestId
| where flag > 1

you don't need transaction

transaction command consumes much search time.
stats is almost useful and no problem.
because stats makes events to combine one event by same field value.
Isn't it the same as the transaction command?

my previous answer has a few typo(statsu). I fix it.

Is there the login field? please try again.

0 Karma

gmasy
New Member

This one actually worked like a charm and is 10 times faster than the transaction method!
Thank you so much

Is there a way to do a timechart of the duration between the request and its response with the solution you provided?

0 Karma

to4kawa
Ultra Champion

a timechart of the duration between the request and its response
What's x-axis and y-axis?

There is no keep-alive log. Your log has only REQUEST and RESPONSE.

Everybody wants to make time chart. but It's very hard.
Good luck.

0 Karma

gmasy
New Member

x-axis would be the time
y-axis would be te average response time

We like to monitor the response time over a period, to detect if a component is down.

Please see example below of what I could achieve with transaction. However this method is very long and usually times out if I check on a very long time span.
alt text

0 Karma

to4kawa
Ultra Champion
 customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
 | rex "(?<status>LEAN-(REQUEST|RESPONSE))"
 | stats dc(status) as flag min(_time) as _time range(_time) as duration by RequestId
 | where flag > 1
 | fields _time RequestId duration
 | timechart span=1h avg(duration)

If you want to calculate by each RequestId, use by clause.

0 Karma

PavelP
Motivator

try to add match/searchmatch to startswith:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith=eval(searchmatch("BCIMR-"))
0 Karma

gmasy
New Member

alt text

If someone has any input it'd be greatly appreciated.
Thank you

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...