I'm attempting to compute the total number of API calls from our backend engine. Initially, I process API identification text logs as events in the engine's index, enabling me to filter respective re...
See more...
I'm attempting to compute the total number of API calls from our backend engine. Initially, I process API identification text logs as events in the engine's index, enabling me to filter respective request IDs. Simultaneously, I process the target_num count within the same index/source. By merging these two logs through a join operation, I filter out all relevant requests to compute the total API calls accurately, achieving the desired outcome.
Subsequently, I aim to enhance this by joining the filtered request IDs with another platform's index/source. Here, I intend to determine the success or failure status of each request at the platform level and then multiply it by the original value of target_num. However, upon combining these queries, I'm experiencing discrepancies in the execution results. I'm uncertain about the missing piece causing this issue. My Final Query : <x-request-id is an existing field on platform index and there is no rex I am using> ----------------------
index=default-va6* sourcetype="myengine-stage" "API call is True for MyEngine"
| rex field=_raw "request_id=(?<reqID>.+?) - "
| dedup reqID
| join reqID [
search index=default-va6* sourcetype="myengine-stage" "Target language count"
| rex field=_raw "request_id=(?<reqID>.+?) - "
| rex field=_raw "Target language count (?<num_target>\d+)"
| dedup reqID
| fields reqID, num_target ]
| fields reqID, num_target
| stats count("reqID") as total_calls by num_target
| eval total_api_calls = total_calls * num_target
| stats sum(total_api_calls) as Total_Requests_Received
| rename reqID AS "x-request-id"
| join "x-request-id" [
search index=platform-va6 sourcetype="platform-ue*" "Marked request as"
| eval num_succeed = if(like(message, "Marked request as succeed%"), 1, 0)
| eval num_failed = if(like(message, "Marked request as failed%"), 1, 0)
| fields num_succeed, num_failed ]
| fields num_succeed, num_failed
| stats sum(num_succeed) as num_succeed, sum(num_failed) as num_failed
| eval total_succeed_calls = num_succeed * num_target, total_failed_calls = num_failed * num_target