topic Re: compare two xml file with Splunk in Splunk Search

How to compare two XML file with Splunk?

indeed_2000 — Fri, 29 Jul 2022 05:54:43 GMT

I need to compare two xml file with Splunk to find changes, is it possible?

sample file

Thanks

Re: compare two xml file with Splunk

gcusello — Fri, 29 Jul 2022 05:56:59 GMT

Hi @indeed_2000,

yes, it's possible.

You have to ingest them, creating an input and indexing them as one or more events.

The you have to run a simple search like the following.

index=your_index source IN (source1,source2) | stats dc(_raw) AS raw_count values(_raw) AS _raw BY source | where raw_count=1 | table source _raw

Ciao.

Giuseppe

Re: How to compare two XML file with Splunk?

ITWhisperer — Fri, 29 Jul 2022 06:09:15 GMT

Why do you need to use Splunk to do something that it is not intended for and there are other much better tools for this particular job?

Re: compare two xml file with Splunk

indeed_2000 — Fri, 29 Jul 2022 06:46:51 GMT

@gcusello should i do something else before add xml files?

because i add "data input" that continiously index this path /data/xml into the index="xml"

but after copy xml file can't see them in splunk!

FYI: i set source type " automatic"

any idea?

Re: compare two xml file with Splunk

indeed_2000 — Fri, 29 Jul 2022 06:57:51 GMT

I edited inputs.conf and added crcSalt = SOURCE to the monitor. It helped.

Re: compare two xml file with Splunk

indeed_2000 — Fri, 29 Jul 2022 07:11:21 GMT

@gcusello now it show a table like this:

source _raw

/data/xml1 content

/data/xml2 content

i can't see anything that show what is the different between these files.

any idea?

Re: compare two xml file with Splunk

gcusello — Fri, 29 Jul 2022 07:21:39 GMT

Hi @indeed_2000,

using my method you can have the differen files, I cannot highligh the single word.

Differences must be manually find.

Ciao.

Giuseppe

Re: compare two xml file with Splunk

indeed_2000 — Fri, 29 Jul 2022 07:30:31 GMT

@gcusello so it just put two xml beside eachother? compare must be done manually?

Re: compare two xml file with Splunk

gcusello — Fri, 29 Jul 2022 08:28:32 GMT

Hi @indeed_2000,

the above search find if there are some differences betwenn files.

To find differences, you could divide each file in rows, using regexes, and compare them on by one, but it's a big job.

Ciao.

Giuseppe

Re: compare two xml file with Splunk

indeed_2000 — Fri, 29 Jul 2022 08:49:13 GMT

@gcusello As far as I can see actually it’s not detect or show “some differences between files.”

Any idea?

Re: compare two xml file with Splunk

yuanliu — Fri, 29 Jul 2022 17:02:27 GMT

i can't see anything that show what is the different between these files.

Is your end goal to show something like diff output? As ITWhisperer said, Splunk is not intended to solve that kind of problem.

To make an approximation, assuming that you have the contents of two files in two fields, xml1 and xml2. You can do something like

| foreach xml* [ eval <<FIELD>> = split(<<FIELD>>, " ") ] | eval lines = mvrange(0, if(mvcount(xml1) > mvcount(xml2), mvcount(xml1), mvcount(xml2))) | eval diff = mvmap(lines, mvappend(diff, if(mvindex(xml1, lines) != mvindex(xml2, lines), lines . ": <" . mvindex(xml1, lines) . " >" . mvindex(xml2, lines), null())

This can give you a very poor facsimile of diff

diff	xml1	xml2
1: <<somefield /> ><someother />	<xml> <somefield /> </xml>	<xml> <someother /> </xml>

Similarly, you can simulate side-by-side diff output, and so on, even unified diff. But any way you slice it, Splunk is not an ideal tool for this.