How to generate a output bases on a common value from two different events?

[xp001975]

we have two separate events which have a common field x-provider-api-correlation-id .

In 1st event it is coming as part of HTTP response header and in second api it is coming as part of Http Request Header.


My requirement is to extract  start time (_time-(time_to_serve_request/1000), endtime which is _time from these two separate events  based of x-provider-api-correlation-id which is having same value .

[xp001975]

deleted the message

[ITWhisperer · Accepted solution]

Fieldformat does not apply to the subsearch - try this way

index=apic* app_name=Inter
| eval Starttime=strftime(_time-(time_to_serve_request/1000), "%F %T.%Q")
| table Starttime _time response_http_headers{}.x-provider-api-correlation-id time_to_serve_request status_code
| rename _time as ESLToInterTime | fieldformat ESLToInterTime=strftime(ESLToInterTime, "%F %T.%Q")
| rename Starttime as InterToESLTime
| join X-Provider-API-Correlation-Id [ search index=apic* app_name=Ava
| eval Starttime=strftime(_time-(time_to_serve_request/1000), "%F %T.%Q")
| table Starttime _time request_http_headers{}.X-Provider-API-Correlation-Id time_to_serve_request status_code
| rename _time as ESLToAvaTime 
| rename Starttime as AvaTOESLTime ]
| fieldformat ESLToAvaTime=strftime(ESLToAvaTime, "%F %T.%Q")

[xp001975]

Thanks ITWhisperer ! That works .

Is there a way we can join two different indexes , earlier  case is same index . 

[ITWhisperer]

A join will join the current events in the pipeline (from the first search) with the events from the subsearch. These searches are independent of each other and can be resolved using different indexes.

Similarly, you can use stats to gather events which can from different indexes in the same initial search.

(index=a sourcetype=b) OR (index=x sourcetype=y)
| rename fieldA as commonfieldname
| rename fieldX as commonfieldname
| stats values(*) as * by commonfieldname

[xp001975]

eip-microservice-metric: {"path":"/request","method":"POST","status":"200 OK","consCorrId":null,"apiTransId":"139efc926411fa1536b2488f","appName":null,"start":"2023-03-15T13:02:13.596-0400","end":"2023-03-15T13:02:13.992-0400","tTime":396,"aTime":1,"fTime":395,"e":null,"addl":{"X-Correlation-Id":"d7-7c455c12ac15","X-Provider-Correlation-Id":"586752.393268","X-Provider-API-Correlation-Id":"9176536b2488f"},"pMetrics":[{"provider":"API","function":"invokeXXXX","time":395,"e":null}],}


How to extract start and end from this eip-microservice-metric ?

[ITWhisperer]

Rename the fields so that they have the same name, you can then do stats using the by clause to gather values from the two events together.

[xp001975]

Not clearly able to understand "Rename the fields so that they have the same name" in particular. Is my Splunk query looks correct like you  suggested in previous response. 

[ITWhisperer]

Note that field names are case-sensitive and it is best to avoid special characters in field names.

| rename response_http_headers{}.x-provider-api-correlation-id as x_provider_api_correlation_id
| rename request_http_headers{}.X-Provider-API-Correlation-Id as  x_provider_api_correlation_id
| stats values(*) as * by x_provider_api_correlation_id

[xp001975]

All data looks good  except 

rename _time as EToAvaTime | fieldformat EToAvaTime=strftime(EToAvaTime, "%F %T.%Q")
is not giving YYYY-MM-DD HH:MM:SS.MilliSec format .
coming as 1677704096.378