Splunk Search

Correlating fields from two separate message types

tslnmx
Explorer

There are two types of messages I'd like to correlate in my logs:

  • An occasional message reporting an integer for the load my server is under
  • Common messages saying how long it took to execute certain operations in my app

I need a scatter chart showing the server load on one axis, and operation time in seconds on the other, with the points in the chart showing the load vs execution time of the operations.

The problem I'm having is, the field that announces the server load is not included in the same messages that announce the elapsed time for that particular operation. So when I'm searching for operations, I need a way to assign the last server load value that was logged before each operation message, to each individual operation message.

Let me know if this doesn't make sense :). I've been having a rough time working it out.

Tags (1)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

the time is probably your best join.

example :
source=logA format : load=12 (happening every 5 min)
source=logB format : duration=19 (happening any time)

a basic search may be like :


source=logA OR source=logB load OR duration | timechart span=5min max(load) AS load avg(duration) AS duration | table load duration

View solution in original post

yannK
Splunk Employee
Splunk Employee

the time is probably your best join.

example :
source=logA format : load=12 (happening every 5 min)
source=logB format : duration=19 (happening any time)

a basic search may be like :


source=logA OR source=logB load OR duration | timechart span=5min max(load) AS load avg(duration) AS duration | table load duration

tslnmx
Explorer

Novel solution, thanks!

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Event Series: Telemetry Pipeline Management

Balancing Scale and Spend: Gaining Control Over High-Volume Metrics in Splunk Observability Cloud As ...

Kick the Tires Before You Commit: A Hands-On Tour of the Splunk Observability Cloud ...

Evaluating an enterprise observability platform usually goes like this: fill out a form, get a free trial with ...

Deep insights, no barriers: Splunk Observability Cloud Free Edition

As software delivery cycles continue to accelerate, observability shouldn’t be a luxury — it should be a ...