Splunk Search

Correlating fields from two separate message types

tslnmx
Explorer

There are two types of messages I'd like to correlate in my logs:

  • An occasional message reporting an integer for the load my server is under
  • Common messages saying how long it took to execute certain operations in my app

I need a scatter chart showing the server load on one axis, and operation time in seconds on the other, with the points in the chart showing the load vs execution time of the operations.

The problem I'm having is, the field that announces the server load is not included in the same messages that announce the elapsed time for that particular operation. So when I'm searching for operations, I need a way to assign the last server load value that was logged before each operation message, to each individual operation message.

Let me know if this doesn't make sense :). I've been having a rough time working it out.

Tags (1)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

the time is probably your best join.

example :
source=logA format : load=12 (happening every 5 min)
source=logB format : duration=19 (happening any time)

a basic search may be like :


source=logA OR source=logB load OR duration | timechart span=5min max(load) AS load avg(duration) AS duration | table load duration

View solution in original post

yannK
Splunk Employee
Splunk Employee

the time is probably your best join.

example :
source=logA format : load=12 (happening every 5 min)
source=logB format : duration=19 (happening any time)

a basic search may be like :


source=logA OR source=logB load OR duration | timechart span=5min max(load) AS load avg(duration) AS duration | table load duration

tslnmx
Explorer

Novel solution, thanks!

0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...