I have three searches that I want to merge into one single table as search output. I will try to explain my case through three individual searches:
Search 1 (base search)
(index="website_monitoring" sourcetype="url_check") title="wiki-via-bfrm-lbs" `filter_service_period_hours` total_time > `response_time_threshold` | table _time total_time time_namelookup corr_id | sort -_time
Output fields:
- _time
- total_time
- time_namelookup
- corr_id
Example event:
2017-04-13T10:06:03+0000 title="wiki-via-bfrm-lbs" corr_id=1492077961074495474 response_code=200 expected_content=True total_time=2604.000 time_redirect=0 time_appconnect=0 time_connect=2515.000 time_namelookup=2513.000 time_pretransfer=2515.000 time_starttransfer=2573.000 request_time=2604.000 timed_out=False content_size=55914 url_effective=http://mydomain.com/wiki/pages/viewpage.action?pageId=65538&corr_id=1492077961074495474&script=wiki-via-bfrm-lbs url=http://mydomain.com/wiki/pages/viewpage.action?pageId=65538 content="Let's edit this page (step 3 of 9) - Demonstration Space "
Search 2 (additional fields based on base search)
index=confluence_prod sourcetype="confluence:app:access" corr_id=1492068301527265031 | table requesttime_in_ms
Correlation fields
- field corr_id is present in Search 1 and Search 2. Value 1492068301527265031 is ment to be retrieved from Search 1
Output fields
- requesttime_in_ms
Example events
[2017-04-13T10:06:03+0000] ip=172.0.0.1 user=TestUser http_method=GET url=“/wiki/pages/viewpage.action?pageId=65538&corr_id=1492077961074495474&script=wiki-via-bfrm-lbs" protocol_version=HTTP/1.1 http_status_code=200 responsesize_bytes=55977 requesttime_in_ms=2176 referer="-" user_agent="curl/7.29.0"
Search 3 (additional fields based on base search)
index="os" sourcetype="cpu" host="aca-db*" cpu="all" earliest="04/13/2017:10:06:00" latest="04/13/2017:10:06:59" | eval total_usage=(100-pctIdle) | stats avg(total_usage) as cpu_usage
Correlation fields
- fields earliest and latest in Search 2 shall be equal to earliest and latest in Search 1.
Output fields
- cpu_usage
Desired search output (merged)
_time (from Search 1)
total_time (from Search 1)
time_namelookup (from Search 1)
requesttime_in_ms (from Search 2)
cpu_usage (from Search 3)
What I have tried so far
I have been reading different answers and Splunk doc about append , join , multisearch . I wanted to give a try solution described in the answer: https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-join-append-or-use-of-subsearches.html, but so far I have not succeeded for all three searches.
Search (merging output from Search 1, Search 2 and Search 3)
(index="website_monitoring" sourcetype="url_check" title="wiki-via-bfrm-lbs" total_time > `response_time_threshold`)
OR (index="confluence_prod" sourcetype="confluence:app:access") `filter_service_period_hours`
OR (index="os" sourcetype="cpu" host="aca-db*" cpu="all" earliest="04/13/2017:12:05:05" latest="04/13/2017:12:07:05")
| eval corr_id-{index}=corr_id
| eval Time=strftime(_time, "%F %T")
| eval total_usage=(100-pctIdle)
| stats values(corr_id-*) AS * values(total_time) as "Response time Monitoring" values(requesttime_in_ms) as "Response time App" values(time_namelookup) as "DNS lookup" avg(total_usage) as cpu_usage by corr_id Time
| mvexpand website_monitoring
| mvexpand confluence_prod
| where website_monitoring=confluence_prod
| fields - website_monitoring,confluence_prod
Output
Output shows correct values for fields
- _time (from Search 1)
- total_time (from Search 1)
- time_namelookup (from Search 1)
- requesttime_in_ms (from Search 2)
but for cpu_usage (from Search 3) I dont get any value in output.
I would really appreciate tips/hints on how to fulfil desired search output.
... View more