Splunk Search

How to compare two XML file with Splunk?

indeed_2000
Motivator

Hi 

I need to compare two xml file with Splunk to find changes, is it possible?

sample file

Thanks 

Labels (5)
Tags (1)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Why do you need to use Splunk to do something that it is not intended for and there are other much better tools for this particular job?

gcusello
SplunkTrust
SplunkTrust

Hi @indeed_2000,

yes, it's possible.

You have to ingest them, creating an input and indexing them as one or more events.

The you have to run a simple search like the following.

index=your_index source IN (source1,source2)
| stats dc(_raw) AS raw_count values(_raw) AS _raw BY source
| where raw_count=1
| table source _raw

Ciao.

Giuseppe

 

0 Karma

indeed_2000
Motivator

@gcusello now it show a table like this:

source             _raw

/data/xml1      content

/data/xml2      content

i can't see anything that show what is the different between these files.

 

any idea?

 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

 


i can't see anything that show what is the different between these files.

Is your end goal to show something like diff output? As ITWhisperer said, Splunk is not intended to solve that kind of problem.

To make an approximation, assuming that you have the contents of two files in two fields, xml1 and xml2.  You can do something like

 

 

| foreach xml*
    [ eval <<FIELD>> = split(<<FIELD>>, "
") ]
| eval lines = mvrange(0, if(mvcount(xml1) > mvcount(xml2), mvcount(xml1), mvcount(xml2)))
| eval diff = mvmap(lines, mvappend(diff, if(mvindex(xml1, lines) != mvindex(xml2, lines), lines . ":
<" . mvindex(xml1, lines) . "
>" . mvindex(xml2, lines), null())

 

 

This can give you a very poor facsimile of diff

diff
xml1
xml2
1:
<<somefield />
><someother />
<xml>
<somefield />
</xml>
<xml>
<someother />
</xml>

Similarly, you can simulate side-by-side diff output, and so on, even unified diff.  But any way you slice it, Splunk is not an ideal tool for this.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @indeed_2000,

using my method you can have the differen files, I cannot highligh the single word.

Differences must be manually find.

Ciao.

Giuseppe

0 Karma

indeed_2000
Motivator

@gcusello so it just put two xml beside eachother? compare must be done manually? 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @indeed_2000,

the above search find if there are some differences betwenn files.

To find differences, you could divide each file in rows, using regexes, and compare them on by one, but it's a big job.

Ciao.

Giuseppe

0 Karma

indeed_2000
Motivator

@gcusello As far as I can see actually it’s not detect or show “some differences between files.”

Any idea?

 

0 Karma

indeed_2000
Motivator

@gcusello should i do something else before add xml files?

because i add "data input" that continiously index this path /data/xml into the index="xml"

but after copy xml file can't see them in splunk!

 

FYI: i set source type " automatic"

 

any idea?

0 Karma

indeed_2000
Motivator

I edited inputs.conf and added crcSalt = SOURCE to the monitor. It helped.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...