Parse XML event at index time

[cefalan]

Hi,
i'm trying to configure the props.conf to automatically parse the following event, but without getting good results. Seems that the KV_MODE = xml dosen't work

<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:24:22.267742Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
</AuditRecord>
<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:25:23.137592Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
</AuditRecord>

Can you help me ?

[niketn]

@cefalan, Sorry I think I misunderstood the question. If KV_MODE=xml is not working out of the box for your data, then you need to ensure that only XML data is present in the raw data. If your raw events contain XML and some additional content, you would

1. either need to remove additional content from the raw events while indexing using props and transforms config files to ensure KV_MODE=xml works on raw data
2. or else during search time using rex and followed by spath to get the required XML data at search time.

Could you post the result of the following search after macking/anonymizing any sensitive data? Do you see only XML data or XML with some additional content:

index=<yourIndexWithXMLData>
| head 1
| table _raw

[codebuilder]

The information/advice posted above by @niketnilay is absolutely spot on.

[codebuilder]

In the case of our production servers, we have this very issue. The XML logs are "mostly" valid XML, with the exception of a single value preceding each row/line.

In this situation, I use a combination of props/transforms to move the "errant" value into proper XML format, and rewrite _raw before sending to the indexers.

If examples would help, let me know.

[codebuilder]

If you have properly formatted XML, the search you posted is unnecessary.
Set KV_MODE=xml in your props.conf, and cycle your search head(s). It will automatically parse your events, including multi-value fields.

[niketn]

@cefalan give a try to the following answer where REGEX is used through props and transforms: https://answers.splunk.com/answers/133533/xml-extraction.html

Then add WRITE_META=true as per the doc below: https://docs.splunk.com/Documentation/Splunk/latest/Data/Configureindex-timefieldextraction#Add_a_re...

[codebuilder]

KV_MODE = xml works wonderfully in my experience. We are using it in several production environments.

Can you please post your full props.conf and some screenshots of your search results showing why you think it isn't working?

[to4kawa]

| makeresults 
| eval raw="<AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:24:22.267742Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
 </AuditRecord>
 <AuditRecord><Field1>1</Field1><Field2>22</Field2><Field3>333</Field3><Field4>444</Field4><Field5>2019-12-13T12:25:23.137592Z</Field5><Field6>6666</Field6><Field7>77777</Field7>
 </AuditRecord>"
| makemv delim="
" raw
| mvexpand raw
| rename raw as _raw
| xmlkv

• The 'xml' and 'json' modes do not extract any fields when used on data that isn't of the correct format (JSON or XML).
• Default: auto

How about KV_MODE=auto ?