Splunk Search

merge 2 logs into one with at most one value for each field

shayhibah
Path Finder

Hi,

I would like to combine 2 logs (or more) as the following:

log #1:
time=1563281015|name=sh_lab|a=1|b=2|c=3|

log#2:
time=1563281010|a=2|d=4|

Output should be:
time=1563281015|name=sh_lab|a=1|b=2|c=3|d=4|

(time is taken from the first log since it is the latest log,d is taken from the second log since it does not part of the latest log, all other fields are taken from the first log since it is the latest log).

Is it possible to do it in Splunk?

Tags (1)
0 Karma

chinmoya
Path Finder

@shayhibah :

Appendcols might give you what you desire. below is something i tried with your data set.

| makeresults | eval time=1563281015, name="sh_lab", a=1, b=2, c=3
| appendcols [makeresults | eval time=1563281010,a=2,d=4 | table d]

Kindly know that there is a high possibility you might not get correct results.

Because it will correlate the first event of log one with the 1st event of log two, and so on.

It will not care about the time and fields of the events or when events came.

0 Karma

shayhibah
Path Finder

any idea??

0 Karma

@shayhibah how have you indexed the two logs? Are they having same/different metadata like index, sourcetype, source? Best approach would be to write you search to pull all required data from index. Then you can apply correlation afterwards (as per your example does not seem like there is any correlation between the two logs).

Please provide more information for the community to assist you better. What is your current SPL? How are you correlating log 1 data with log 2? Will name field serve as the key to correlate and find unique events in each log? How many fields can these logs have?

Based on your explanation so far, you can try something like the following:

index=blah sourcetype IN ("sourcetypeForLog1","sourceTypeForLog2") source="path\log*"
| fields name a,b,c,d,
| stats min(_time) as _time last(*) as * by name
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...