Grouping event by correlation id with the closest time

[lucavi]

Hi All,

We have a number of micro services with correlation id flowing across the request and responses. What i'm trying to do is to create a flow of request and response for 1 correlation id.

Example log

correlation id	time	source	message
123	12:00:00	Service A	Enter service A
123	12:00:01	Service A	Calling Service B
123	12:00:02	Service B	Routing to Service C
123	12:00:03	Service C	Result Found. Response User 1
123	12:00:04	Service B	Using User 1 to find resource
123	12:00:05	Service B	Resource Found. Calling Service D
123	12:00:06	Service D	Sub-resource not found. Response: null
123	12:00:07	Service B	Return result. Response User1, resource1
123	12:00:08	Service A	Return User1, resource1

From the example log, i would like to be able to group

Service A (12:00:00 -12:00:01)

Service B (12:00:02)

Service C (12:00:03)

Service B (12:00:04 -12:00:05)

Service D (12:00:06)

Service B (12:00:07)

Service A (12:00:08)

What i'm trying to do right now is a simple event results first before going to any further fancy visualization.

I tried using Transaction but i can't separate the source when there's a different call in between.

Here's the query that i've tried

123 
| eval _time=strptime(timegenerated,"%Y-%m-%dT%H:%M:%SZ")
| sort - _time
| transaction source

Any help is greatly appreciated.

Thanks,
Allen

[ITWhisperer]

| streamstats window=1 current=f global=f values(source) as previous by correlation_id
| eval change=if(source==previous,0,1)
| streamstats sum(change) as group by correlation_id
| stats min(time) as start max(time) as end list(message) as messages by correlation_id group source