StartsWith breaks the transaction

[gmasy]

Hello everyone,

I am trying to extract some data from the logs.
I have created a little search that works well:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId

It links the request and the response from the server by the extracted field RequestId.

However, I am trying to filter only some parameter contained in the field Login:

The login has many possibilities of different values, but it starts with either:

• "BBC-*"
• "BBF-*"
• "BFL-*"
• "BCIMR-*"

The login is only in the request, and not in the response.
I am trying to filter by putting the following

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith="BCIMR-*"

However, when I add the StartsWith, everything's broken, as the events are not linked together anymore.
See following comment for example as I can't add more than 2 images per question

[gmasy]

Thanks to the three of you for your quick input about my problem!
I'm happily surprised of the reactivity on this forum, as this was my very first post...

However, none of the solutions provided did work for me.
@PavelP, although your solution seemed great, I got the same result as attached in previous screenshot. The response and request are not grouped in the same event when adding the startswith ...

@to4kawa and @woodcock , thank you for your suggestions, could you please explain me why transaction is overkill for what I am trying to achieve?
It's the only solution I have found in the Splunk documentation to match two corresponding events in the same splunk result.
It enables me to see what is in each request and what is the associated response.
stats here is not the thing I am trying to achieve just yet.

Thank you

[to4kawa]

I see. my answer is updated. please check and try.

[woodcock]

Ditch transaction; it is overkill and does not scale well; try this:

index="YouShouldAlwaysSepcifyIndex" AND sourcetype="AndSourcetypeToo" AND "customergetservice" AND host="MBKBKKSPHTRSP0*" AND source="/var/log/jbossas/standalone/server.log"
| rex "LEAN-(?<status>REQUEST|RESPONSE)"
| reverse
| streamstats count(eval(searchmatch("BCIMR-*"))) AS TransactionID BY RequestId
| stats range(_time) AS duration count AS eventcount values(status) AS status dc(status) AS statuscount min(_time) AS _time BY TransactionID RequestId

[to4kawa · Accepted solution]

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
| rex "(?<status>LEAN-(REQUEST|RESPONSE))"
| stats dc(status) as flag values(_raw) as _raw values(_time) as _time by RequestId
| where flag > 1

you don't need transaction

transaction command consumes much search time.
stats is almost useful and no problem.
because stats makes events to combine one event by same field value.
Isn't it the same as the transaction command?

my previous answer has a few typo(statsu). I fix it.

Is there the login field? please try again.

[gmasy]

This one actually worked like a charm and is 10 times faster than the transaction method!
Thank you so much

Is there a way to do a timechart of the duration between the request and its response with the solution you provided?

[to4kawa]

a timechart of the duration between the request and its response
What's x-axis and y-axis?

There is no keep-alive log. Your log has only REQUEST and RESPONSE.

Everybody wants to make time chart. but It's very hard.
Good luck.

[gmasy]

x-axis would be the time
y-axis would be te average response time

We like to monitor the response time over a period, to detect if a component is down.

Please see example below of what I could achieve with transaction. However this method is very long and usually times out if I check on a very long time span.

[to4kawa]

 customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log"  (login="BBC-*" OR login="BBF-*" OR login="BFL-*" OR login="BCIMR-*") OR "LEAN-RESPONSE"
 | rex "(?<status>LEAN-(REQUEST|RESPONSE))"
 | stats dc(status) as flag min(_time) as _time range(_time) as duration by RequestId
 | where flag > 1
 | fields _time RequestId duration
 | timechart span=1h avg(duration)

If you want to calculate by each RequestId, use by clause.

[PavelP]

try to add match/searchmatch to startswith:

customergetservice host=MBKBKKSPHTRSP0* source="/var/log/jbossas/standalone/server.log" | transaction RequestId startswith=eval(searchmatch("BCIMR-"))

[gmasy]

If someone has any input it'd be greatly appreciated.
Thank you