Splunk Search

Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100
Path Finder

Hi - I'm having trouble in combining 2 separate searches and displaying the results on a single visualization (timechart). search 1 searches for events in the access log and search 2 searches for events in another log file (process log). Both log files have a field called "responseTime". In the process log the responseTime field represents "Processing Time". I'm able to run the searches independently and plot the timechart but when I combine them using join I don't get back any results. If I use append I get back results only from search 2. I am looking for 2 lines on the same timechart - one that represents processing time (from process log) and the other would be response time (from access log).

search1: index=xxx source=/x/x/x/localhost_access_log* field=value1
search2: index=xxx source=/x/x/x/process_log field=value2

Using join I tried the following queries: (No results)

search1 | join [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)

(and)

search1 | timechart avg(responseTime) | join [search2 | timechart avg(responseTime)]

Using append: (got back results only from search2)

search1 | append [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)
Tags (1)
0 Karma
1 Solution

pdpsplunk100
Path Finder

This did the trick.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]

View solution in original post

0 Karma

pdpsplunk100
Path Finder

This did the trick.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]
0 Karma

renjith_nair
Legend

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )|timechart avg(responseTime) as responseTime, avg(processTime) as processTime
---
What goes around comes around. If it helps, hit it with Karma 🙂
0 Karma

pdpsplunk100
Path Finder

I get how you use the OR to search both sources. But the field name is "responseTime" in both the log files. I would need to rename "responseTime" in the process_log to "processTime" before the timechart command. Where/How do I place the rename in your suggested query because I need to tell Splunk that the rename applies only to the process_log source.

0 Karma

sundareshr
Legend

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )| timechart avg(eval(match(source, "access") as  responseTime, avg(eval(match(source, "process") as processTime
0 Karma

pdpsplunk100
Path Finder

Tried using match but that does not seem to work. I get eval expression is malformed and timechart syntax is incorrect.

I was able to get what I wanted using the query below (using join). I'm still looking for ways to optimize the query as join is expensive but this does the job for now.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]

Thanks for your guidance folks.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...