How to add and parse XML data in Splunk?

[prijila]

Hi,
I have an XML file as my source file. It has the following structure:

<photos>
     <photo id="123" title="Birthday" format="jpg">
          <owner id="1111">Jason</owner>
          <comments>
              <comment ownerid="112">Good pic!</comment>
              <comment ownerif="223">Happy birthday</comment>
          <comments>
      </photo>
</photos>

How do i parse this and load this data into splunk?

Thank you in advance.

[vasanthmss · Accepted solution]

Hi,

sample xml,

<photos>
<photo id="1231" title="Birthday" format="jpg">
<owner id="1112">Jason</owner>
<comments>
<comment ownerid="113">Good pic!</comment>
<comment ownerid="224">Happy birthday</comment>
<comments>
</photo>
</photos>

props.conf

[yoursourcetype]
DATETIME_CONFIG = CURRENT
KV_MODE = xml
LINE_BREAKER = (<photos>)
MUST_BREAK_AFTER = \</photos\>
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 0
pulldown_type = 1
FIELDALIAS-rootfields = photo.owner as Owner photo{@id} as PhotoID photo{@title} as PhotoTitle photo.owner{@id} as PhotoOwnerID photo{@format} as PhotoFormat photo.comments.comment{@ownerid} as CommentOwnerID photo.comments.comment as Comment

Search

index=<yourindex> sourcetype=<yoursourcetype> | table  PhotoID,PhotoTitle,PhotoFormat,CommentOwnerID,Comment,Owner

[donaldwayne1975]

Very helpful. working with an XML that the designer decided to embed more fields in the value between the tags like this:
$field1=stuff,$field2=morestuff,$field3=evenmorestuff
Have not found a way to get these $fields to extract successfully. Suggestions?

[akocak]

@donaldwayne1975
You can add EVAL into your props for this.

[ifremer2015]

Which props.con should we modify ? ../etc/system or ..etc/apps

Thank you !

[krishnarajb2304]

In-case if the sourcetype belongs to one application you can update this in application (/etc/apps//local/props.conf.

The same sourcetype can be re-used in more than one application you can add it in system props.conf
/etc/system/local/props.conf.

[aljohnson_splun]

This might be helpful:
http://answers.splunk.com/answers/683/xml-input-line-breaking-and-field-extraction-how.html

[Raghav2384]

xmlkv and path should help you. Please refer this answer http://answers.splunk.com/answers/54957/how-to-extract-data-from-xml.html
Hope this helps.
Thanks,
Raghav

[RMcCurdyDOTcom]

nmap XML to SPLUNK HEC !!! 

https://github.com/freeload101/SCRIPTS/blob/b3f83288a9f289d86f6cdd04898478d0427097ce/Bash/NMAP_FRUIT.sh#L80

[RMcCurdyDOTcom]

got nasty gram for posting links

search online for freeload101 github in scripts nmap_fruit.sh