How to compare two XML file with Splunk?

[indeed_2000]

Hi 

I need to compare two xml file with Splunk to find changes, is it possible?

sample file

Thanks 

[ITWhisperer]

Why do you need to use Splunk to do something that it is not intended for and there are other much better tools for this particular job?

[gcusello]

Hi @indeed_2000,

yes, it's possible.

You have to ingest them, creating an input and indexing them as one or more events.

The you have to run a simple search like the following.

index=your_index source IN (source1,source2)
| stats dc(_raw) AS raw_count values(_raw) AS _raw BY source
| where raw_count=1
| table source _raw

Ciao.

Giuseppe

[indeed_2000]

@gcusello now it show a table like this:

source             _raw

/data/xml1      content

/data/xml2      content

i can't see anything that show what is the different between these files.

any idea?

[yuanliu]

---

i can't see anything that show what is the different between these files.

Is your end goal to show something like diff output? As ITWhisperer said, Splunk is not intended to solve that kind of problem.

To make an approximation, assuming that you have the contents of two files in two fields, xml1 and xml2.  You can do something like

| foreach xml*
    [ eval <<FIELD>> = split(<<FIELD>>, "
") ]
| eval lines = mvrange(0, if(mvcount(xml1) > mvcount(xml2), mvcount(xml1), mvcount(xml2)))
| eval diff = mvmap(lines, mvappend(diff, if(mvindex(xml1, lines) != mvindex(xml2, lines), lines . ":
<" . mvindex(xml1, lines) . "
>" . mvindex(xml2, lines), null())

This can give you a very poor facsimile of diff

diff	xml1	xml2
1: <<somefield /> ><someother />	<xml> <somefield /> </xml>	<xml> <someother /> </xml>

Similarly, you can simulate side-by-side diff output, and so on, even unified diff.  But any way you slice it, Splunk is not an ideal tool for this.

[gcusello]

Hi @indeed_2000,

using my method you can have the differen files, I cannot highligh the single word.

Differences must be manually find.

Ciao.

Giuseppe

[indeed_2000]

@gcusello so it just put two xml beside eachother? compare must be done manually? 

[gcusello]

Hi @indeed_2000,

the above search find if there are some differences betwenn files.

To find differences, you could divide each file in rows, using regexes, and compare them on by one, but it's a big job.

Ciao.

Giuseppe

[indeed_2000]

@gcusello As far as I can see actually it’s not detect or show “some differences between files.”

Any idea?

[indeed_2000]

@gcusello should i do something else before add xml files?

because i add "data input" that continiously index this path /data/xml into the index="xml"

but after copy xml file can't see them in splunk!

FYI: i set source type " automatic"

any idea?

[indeed_2000]

I edited inputs.conf and added crcSalt = SOURCE to the monitor. It helped.