topic Re: Comparing data from two log files and displaying results which are different . in Splunk Search

Comparing data from two log files and displaying results which are different .

smolcj — Mon, 28 Sep 2020 13:41:30 GMT

Hi,
My need is to compare two log files of same pattern . sometimes the log files will be entirely different because they can be the files of two different instance or they can be from same instance at a different time , in that case other than few dynamic fields in the product all other fields will be same. i have been using a search for the result from a single file and using join command i tried to find the diff values for the search .
please help me to find an efficient query for this need.

index=main source=SUCCESS
| transaction startswith="Source Summary" endswith="Load Summary"
| table summ_name
| mvexpand summ_name
| join summ_name[
search index=main source=SUCCESS
| fields summ_name summ_instance sum_out sum_affected sum_applied sum_rejected ]
|table summ_name summ_instance sum_out sum_affected sum_applied sum_rejected
|rename summ_name as Source |rename summ_instance as File1
|join type=outer Source [search index=main source=FAIL
| transaction startswith="Source Summary" endswith="Load Summary"
| table summ_name
| mvexpand summ_name
| join summ_name [
search index=main source=FAIL
| fields summ_name summ_instance sum_out sum_affected sum_applied sum_rejected ]
|table summ_name summ_instance sum_out sum_affected sum_applied sum_rejected
|rename summ_name as Source summ_name as summ_name1 sum_out as sum_out 1 sum_affected as sum_affected 1 sum_applied as sum_applied1 sum_rejected as sum_rejected1
|rename summ_instance as File2 ]
|where 'File1' != 'File2' ``

I am not able to provide a full outer join and display all the values from both the files
if some fields are same and other fields are different then i want diplay them in same row
SUCCESS and FAIL are 2 different files

please help
Thank You

Re: Comparing data from two log files and displaying results which are different .

Ayn — Mon, 08 Apr 2013 08:11:48 GMT

Did you have a look at set diff?

Re: Comparing data from two log files and displaying results which are different .

smolcj — Mon, 08 Apr 2013 09:37:49 GMT

Ayn, i have tried set diff command, but i am not able to find something that can meet my second requirement 😞
i.e if source field is same but if other fields are different i am not able to display the values from the second file .. how can i do that ? currently i am using sideview value setter and html modules to group those values under file1 and file2 but then i am facing the issue of full outer join ..
please help ..

Re: Comparing data from two log files and displaying results which are different .

smolcj — Fri, 12 Apr 2013 04:36:46 GMT

Please help, badly in need of a solution

Re: Comparing data from two log files and displaying results which are different .

Ayn — Fri, 12 Apr 2013 06:29:25 GMT

If you're not getting help here (I can't offer any, sorry) and really need to solve this problem, consider having Splunk PS come help you.

Re: Comparing data from two log files and displaying results which are different .

smolcj — Fri, 12 Apr 2013 11:55:20 GMT

Thanks Ayn, but how can i seek help of a product specialist ?

Re: Comparing data from two log files and displaying results which are different .

Ayn — Fri, 12 Apr 2013 12:08:05 GMT

By PS I mean Professional Services - contact Splunk sales to discuss details.

Re: Comparing data from two log files and displaying results which are different .

vj8210 — Wed, 17 Apr 2013 09:26:57 GMT

Hi Can you please paste sample log entries for both files?