Dashboards & Visualizations

Parse XML event at index time

cefalan
New Member

Hi,
i'm trying to configure the props.conf to automatically parse the following event, but without getting good results. Seems that the KV_MODE = xml dosen't work

<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:24:22.267742Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
</AuditRecord>
<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:25:23.137592Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
</AuditRecord>

Can you help me ?

Tags (2)
0 Karma

niketn
Legend

@cefalan, Sorry I think I misunderstood the question. If KV_MODE=xml is not working out of the box for your data, then you need to ensure that only XML data is present in the raw data. If your raw events contain XML and some additional content, you would

  1. either need to remove additional content from the raw events while indexing using props and transforms config files to ensure KV_MODE=xml works on raw data
  2. or else during search time using rex and followed by spath to get the required XML data at search time.

Could you post the result of the following search after macking/anonymizing any sensitive data? Do you see only XML data or XML with some additional content:

index=<yourIndexWithXMLData>
| head 1
| table _raw
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

codebuilder
Influencer

The information/advice posted above by @niketnilay is absolutely spot on.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

codebuilder
Influencer

In the case of our production servers, we have this very issue. The XML logs are "mostly" valid XML, with the exception of a single value preceding each row/line.

In this situation, I use a combination of props/transforms to move the "errant" value into proper XML format, and rewrite _raw before sending to the indexers.

If examples would help, let me know.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

codebuilder
Influencer

If you have properly formatted XML, the search you posted is unnecessary.
Set KV_MODE=xml in your props.conf, and cycle your search head(s). It will automatically parse your events, including multi-value fields.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

niketn
Legend

@cefalan give a try to the following answer where REGEX is used through props and transforms: https://answers.splunk.com/answers/133533/xml-extraction.html

Then add WRITE_META=true as per the doc below: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureindex-timefieldextraction#Add_a_re...

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

codebuilder
Influencer

KV_MODE = xml works wonderfully in my experience. We are using it in several production environments.

Can you please post your full props.conf and some screenshots of your search results showing why you think it isn't working?

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval raw="<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:24:22.267742Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
 </AuditRecord>
 <AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:25:23.137592Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
 </AuditRecord>"
| makemv delim="
" raw
| mvexpand raw
| rename raw as _raw
| xmlkv
  • The 'xml' and 'json' modes do not extract any fields when used on data that isn't of the correct format (JSON or XML).
  • Default: auto

How about KV_MODE=auto ?

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Get the T-shirt to Prove You Survived Splunk University Bootcamp

As if Splunk University, in Las Vegas, in-person, with three days of bootcamps and labs weren’t enough, now ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...