Splunk Search

StartsWith breaks the transaction

gmasy
New Member

Hello everyone,

I am trying to extract some data from the logs.
I have created a little search that works well:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId

It links the request and the response from the server by the extracted field RequestId.
alt text

However, I am trying to filter only some parameter contained in the field Login:
alt text

The login has many possibilities of different values, but it starts with either:

  • "BBC-*"
  • "BBF-*"
  • "BFL-*"
  • "BCIMR-*"

The login is only in the request, and not in the response.
I am trying to filter by putting the following

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith="BCIMR-*"

However, when I add the StartsWith, everything's broken, as the events are not linked together anymore.
See following comment for example as I can't add more than 2 images per question

0 Karma
1 Solution

to4kawa
Ultra Champion
customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
| rex "(?<status>LEAN-(REQUEST|RESPONSE))"
| stats dc(status) as flag values(_raw) as _raw values(_time) as _time by RequestId
| where flag > 1

you don't need transaction

transaction command consumes much search time.
stats is almost useful and no problem.
because stats makes events to combine one event by same field value.
Isn't it the same as the transaction command?

my previous answer has a few typo(statsu). I fix it.

Is there the login field? please try again.

View solution in original post

0 Karma

gmasy
New Member

Thanks to the three of you for your quick input about my problem!
I'm happily surprised of the reactivity on this forum, as this was my very first post...

However, none of the solutions provided did work for me.
@PavelP, although your solution seemed great, I got the same result as attached in previous screenshot. The response and request are not grouped in the same event when adding the startswith ...

@to4kawa and @woodcock , thank you for your suggestions, could you please explain me why transaction is overkill for what I am trying to achieve?
It's the only solution I have found in the Splunk documentation to match two corresponding events in the same splunk result.
It enables me to see what is in each request and what is the associated response.
stats here is not the thing I am trying to achieve just yet.

Thank you

0 Karma

to4kawa
Ultra Champion

I see. my answer is updated. please check and try.

0 Karma

woodcock
Esteemed Legend

Ditch transaction; it is overkill and does not scale well; try this:

index="YouShouldAlwaysSepcifyIndex" AND sourcetype="AndSourcetypeToo" AND "customergetservice" AND host="MBKBKKSPHTRSP0*" AND source="/var/log/jbossas/standalone/server.log"
| rex "LEAN-(?<status>REQUEST|RESPONSE)"
| reverse
| streamstats count(eval(searchmatch("BCIMR-*"))) AS TransactionID BY RequestId
| stats range(_time) AS duration count AS eventcount values(status) AS status dc(status) AS statuscount min(_time) AS _time BY TransactionID RequestId
0 Karma

to4kawa
Ultra Champion
customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
| rex "(?<status>LEAN-(REQUEST|RESPONSE))"
| stats dc(status) as flag values(_raw) as _raw values(_time) as _time by RequestId
| where flag > 1

you don't need transaction

transaction command consumes much search time.
stats is almost useful and no problem.
because stats makes events to combine one event by same field value.
Isn't it the same as the transaction command?

my previous answer has a few typo(statsu). I fix it.

Is there the login field? please try again.

0 Karma

gmasy
New Member

This one actually worked like a charm and is 10 times faster than the transaction method!
Thank you so much

Is there a way to do a timechart of the duration between the request and its response with the solution you provided?

0 Karma

to4kawa
Ultra Champion

a timechart of the duration between the request and its response
What's x-axis and y-axis?

There is no keep-alive log. Your log has only REQUEST and RESPONSE.

Everybody wants to make time chart. but It's very hard.
Good luck.

0 Karma

gmasy
New Member

x-axis would be the time
y-axis would be te average response time

We like to monitor the response time over a period, to detect if a component is down.

Please see example below of what I could achieve with transaction. However this method is very long and usually times out if I check on a very long time span.
alt text

0 Karma

to4kawa
Ultra Champion
 customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
 | rex "(?<status>LEAN-(REQUEST|RESPONSE))"
 | stats dc(status) as flag min(_time) as _time range(_time) as duration by RequestId
 | where flag > 1
 | fields _time RequestId duration
 | timechart span=1h avg(duration)

If you want to calculate by each RequestId, use by clause.

0 Karma

PavelP
Motivator

try to add match/searchmatch to startswith:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith=eval(searchmatch("BCIMR-"))
0 Karma

gmasy
New Member

alt text

If someone has any input it'd be greatly appreciated.
Thank you

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...