topic Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines in Splunk Search

Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100 — Thu, 08 Sep 2016 00:50:49 GMT

Hi - I'm having trouble in combining 2 separate searches and displaying the results on a single visualization (timechart). search 1 searches for events in the access log and search 2 searches for events in another log file (process log). Both log files have a field called "responseTime". In the process log the responseTime field represents "Processing Time". I'm able to run the searches independently and plot the timechart but when I combine them using join I don't get back any results. If I use append I get back results only from search 2. I am looking for 2 lines on the same timechart - one that represents processing time (from process log) and the other would be response time (from access log).

search1: index=xxx source=/x/x/x/localhost_access_log* field=value1
search2: index=xxx source=/x/x/x/process_log field=value2

Using join I tried the following queries: (No results)

search1 | join [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)

(and)

search1 | timechart avg(responseTime) | join [search2 | timechart avg(responseTime)]

Using append: (got back results only from search2)

search1 | append [search2 | rename responseTime as processTime] | timechart avg(responseTime), avg(processTime)

Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

renjith_nair — Thu, 08 Sep 2016 09:27:47 GMT

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )|timechart avg(responseTime) as responseTime, avg(processTime) as processTime

Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100 — Tue, 29 Sep 2020 10:54:47 GMT

I get how you use the OR to search both sources. But the field name is "responseTime" in both the log files. I would need to rename "responseTime" in the process_log to "processTime" before the timechart command. Where/How do I place the rename in your suggested query because I need to tell Splunk that the rename applies only to the process_log source.

Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

sundareshr — Fri, 09 Sep 2016 01:09:24 GMT

Try this

index=xxx (source=/x/x/x/localhost_access_log* OR source=/x/x/x/process_log )| timechart avg(eval(match(source, "access") as  responseTime, avg(eval(match(source, "process") as processTime

Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100 — Fri, 09 Sep 2016 22:38:28 GMT

Tried using match but that does not seem to work. I get eval expression is malformed and timechart syntax is incorrect.

I was able to get what I wanted using the query below (using join). I'm still looking for ways to optimize the query as join is expensive but this does the job for now.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]

Thanks for your guidance folks.

Re: Need help combining 2 separate searches on different log files and show the results on a single timechart as 2 separate lines

pdpsplunk100 — Fri, 09 Sep 2016 22:40:07 GMT

This did the trick.

index=xxx (source=/x/x/x/process_log AND request_param=xxx) | rename responseTime as processTime | timechart avg(processTime) as "Processing Time" | join _time [search index=xxx (source=/x/x/x/localhost_access_log* AND request_param=yyy) | timechart avg(responseTime) as "Response Time"]