I have a scenario where typical HTTP requests are logged in Splunk.
Every request has an unique identifier which is saved in a "request_id" field.
Between request and response the server generates a set of logs/events and each one has this "request_id" added.
So far so good. Now it is possible to find the appropriate server logs for a client HTTP issue via the "request_id".
Now I often also have the requirement to find all errors for a particular device.
In this scenario the device identifier is part of the first log/event of the request, but the following logs/events do not have this information any more.
So basically something like:
1. req.begin "Started... deviceId=12345", request_id="1"
2. .... "request_id="1" deviceId=???
3. .... "request_id="1" deviceId=???
4. .... "request_id="1" deviceId=???
5. req.end ... "request_id="1" deviceId=???
The search would look like:
- Search for all "request.begin" events with device identifier "xyz"
- Get all "request_id" of those events
- and finally get all events containing one of the above "request_id"
Not sure how to build the query for this. 🤔
Would be very grateful for some tips!
Best regards
Tore
I misunderstood your question. Try this:
sourcetype=requests
| eventstats values(device_id) as device_id by request_id
| search device_id="XYZ"
Side note: you should start every search with "index=ABC" for much more optimized searching.
Greetings @moguai ,
Take a look at this search. It should get you close to what you're after. Note that the eventstats and the stats will work by themselves without the other. I included both so you will have more options with manipulating your dataset.
| makeresults | eval _raw = "req.begin \"Started... deviceId=\"12345\", request_id=\"1\""
| append [ | makeresults | eval _raw = ".... \"request_id=\"1\" deviceId=???" ]
| append [ | makeresults | eval _raw = ".... \"request_id=\"1\" deviceId=???" ]
| append [ | makeresults | eval _raw = ".... \"request_id=\"1\" deviceId=???" ]
| append [ | makeresults | eval _raw = "req.end ... \"request_id=\"1\" deviceId=???" ]
| reverse
| rex "deviceId=\"?(?<deviceId>\d+)\"?"
| rex "request_id=\"?(?<request_id>\d+)\"?"
| eventstats values(deviceId) as deviceId by request_id
| stats list(_raw) as events by request_id
Hi @jacobpevans ,
thanks for the fast response! I think I am still a bit confused. 😅
Maybe a bit more practical:
sourcetype="requests" device_id="XYZ"
As result I get e.g. 10 events with 10 different "request_id" (separate field) from that XYZ device.
Than I need to query all events matching the 10 "request_id" I got from the 10 events, because the "device_id" is only available for the first log.
sourcetype="requests" request_id="1" OR request_id="2" OR request_id="3" ...
Would still the approach you mentioned the way to go?
BR,
Tore
I misunderstood your question. Try this:
sourcetype=requests
| eventstats values(device_id) as device_id by request_id
| search device_id="XYZ"
Side note: you should start every search with "index=ABC" for much more optimized searching.
Thanks a lot!