topic Extend search results data by correlation-id (and exclude on other messages) in Splunk Search

Extend search results data by correlation-id (and exclude on other messages)

sscholl — Thu, 25 Apr 2024 13:25:28 GMT

Hello, I have 500 HTTP messages in my access log. Also I have corresponding events from other log sources with the same correlation-id. Now I want to join the information to enhance the results.

Access Log Events:

2024-04-25T11:00:26+00:00 [info] type=access status=500 xCorrelationId=90e2a321-f522-466f-9ffa-72cbdaa1a576 .... 2024-04-25T10:15:25+00:00 [info] type=access status=500 xCorrelationId=9b1833f5-776b-44c3-92d7-d603abdfecf8 ...

Other Events:

2024-04-25T10:15:24+00:00 xCorrelationId=9b1833f5-776b-44c3-92d7-d603abdfecf8 NoHandlerFoundException: No endpoint GET

My actual intention is, to exclude the results from main search, if there is another event with the same correlation-id but containing specific exceptions like "NoHandlerFoundException". That means, i need a search per result from the main search.

Do you know a solution for this?

Thanks!

Re: Extend search results data by correlation-id (and exclude on other messages)

ITWhisperer — Thu, 25 Apr 2024 14:10:29 GMT

<main index> NOT [search <other source> NoHandlerFoundException | stats count by xCorrelationId | fields xCorrelationId | format]

However, depending on how may exceptions you have, you may run into limitations as the sub-search with the format command will essentially return a long string which might be too large to be parsed in the main search.

Another way to do it is to search both sources, and correlate by xCorrelationId and exclude those xCorrelationId's which have the exception, but this still means you are retrieving both full sets of events and correlating them before you can filter out any.

Re: Extend search results data by correlation-id (and exclude on other messages)

bowesmana — Fri, 26 Apr 2024 01:32:38 GMT

The number of values in the subsearch cannot be too large as it will perform really badly, but one slight change to @ITWhisperer subsearch is to do

<main index> NOT [ search <other source> NoHandlerFoundException | stats values(xCorrelationId) as search | format]

which will perform faster. It will change the outer search from

<main index> NOT ( ( ( xCorrelationId=A OR xCorrelationId=B OR... ) ) )

<main index> NOT ( ( ( A OR B OR C OR D ... ) ) )

where A, B etc are the values or xCorrelationId

The key point is having a field name 'search' in the output rather than xCorrelationId, which changes the effect of the format command.