merge 2 logs into one with at most one value for each field

[shayhibah]

Hi,

I would like to combine 2 logs (or more) as the following:

log #1:
time=1563281015|name=sh_lab|a=1|b=2|c=3|

log#2:
time=1563281010|a=2|d=4|

Output should be:
time=1563281015|name=sh_lab|a=1|b=2|c=3|d=4|

(time is taken from the first log since it is the latest log,d is taken from the second log since it does not part of the latest log, all other fields are taken from the first log since it is the latest log).

Is it possible to do it in Splunk?

[chinmoya]

@shayhibah :

Appendcols might give you what you desire. below is something i tried with your data set.

| makeresults | eval time=1563281015, name="sh_lab", a=1, b=2, c=3
| appendcols [makeresults | eval time=1563281010,a=2,d=4 | table d]

Kindly know that there is a high possibility you might not get correct results.

Because it will correlate the first event of log one with the 1st event of log two, and so on.

It will not care about the time and fields of the events or when events came.

[shayhibah]

any idea??

[niketn]

@shayhibah how have you indexed the two logs? Are they having same/different metadata like index, sourcetype, source? Best approach would be to write you search to pull all required data from index. Then you can apply correlation afterwards (as per your example does not seem like there is any correlation between the two logs).

Please provide more information for the community to assist you better. What is your current SPL? How are you correlating log 1 data with log 2? Will name field serve as the key to correlate and find unique events in each log? How many fields can these logs have?

Based on your explanation so far, you can try something like the following:

index=blah sourcetype IN ("sourcetypeForLog1","sourceTypeForLog2") source="path\log*"
| fields name a,b,c,d,
| stats min(_time) as _time last(*) as * by name