topic Parse XML data into fields while indexing in Getting Data In

Parse XML data into fields while indexing

varshini_97 — Wed, 24 Jan 2024 15:19:45 GMT

Sample data:

<?xml version="1.0" encoding="UTF-8" ?>
<Results xmlns:xsi="http://www.w3.org">
<Result>
<Code>OK</Code>
<Details>LoadMessageOverviewData</Details>
<Text>Successful</Text>
</Result>
<Data>
<ColumnNames>
<Column>Sender Component</Column>
<Column>Receiver Component</Column>
<Column>Interface</Column>
<Column>System Error</Column>
<Column>Waiting</Column>
</ColumnNames>
<DataRows>
<Row>
<Entry>XYZ</Entry>
<Entry>ABC</Entry>
<Entry>Mobile</Entry>
<Entry>-</Entry>
<Entry>3</Entry>
</Row>
</DataRows>
</Data>
</Results>

Hello,

I need to extract fields from the above xml data.

I have tried the below props, but still the data is not extracting properly.

Props.conf

CHARSET=UTF-8

BREAK_ONLY_BEFORE = <\/Row>

MUST_BREAK_AFTER = <Row>

SHOULD_LINEMERGE = true

KV_MODE = xml

pulldown_type = true

DATETIME_CONFIG = CURRENT

NO_BINARY_CHECK=true

TRUNCATE=0

description=describing props config

disabled=false

How to parse the data.?

Thanks in advance

Re: Parse XML data into fields while indexing

isoutamo — Thu, 25 Jan 2024 10:25:53 GMT

Your example picks those <Entry> lines to one event and the rests are separate events. So basically this is working, but output is not something what you are expecting? How you would like to divide this to separate events? Can you also add some more events/rows or are events always like this?

r. Ismo

Re: Parse XML data into fields while indexing

PickleRick — Thu, 25 Jan 2024 12:47:55 GMT

1. Please post your data/code samples in a pre-formatted way (using either the "preformatted" style or the code sample control in the editor. It makes the sample easier to read.

2. It's not clear what you want to get from this data.

3. Unless you have a very good reason and a strong use case, you should not parse data _into fields_ while indexing (in other words - create indexed fields). Most parsing in Splunk is done in search time.

4. Unless you have a very very good reason (even better one than the one for the indexed fields) you should not use SHOULD_LINEMERGE=true. It gives you a huge performance hit.

Re: Parse XML data into fields while indexing

varshini_97 — Mon, 29 Jan 2024 13:45:39 GMT

<?xml version="1.0" encoding="UTF-8" ?> <Results xmlns:xsi="http://www.w3.org"> <Result> <Code>OK</Code> <Details>LoadMessageOverviewData</Details> <Text>Successfull</Text> </Result> <Data> <ColumnNames> <Column>Sender Component</Column> <Column>Receiver Component</Column> <Column>Interface</Column> <Column>System Error</Column> <Column>Waiting</Column> </ColumnNames> <DataRows> <Row> <Entry>XYZ</Entry> <Entry>ABC</Entry> <Entry>Mobile</Entry> <Entry>-</Entry> <Entry>3</Entry> </Row> </DataRows> </Data> </MessageStatisticsQueryResults>

Hi @PickleRick ,

Above is my xml data,

My expected parsed data are below

Field Name	Field Value
Sender Component	XYZ
Receiver Component	ABC
Interface	Mobile
System Error	-
Waiting	3

Re: Parse XML data into fields while indexing

varshini_97 — Mon, 29 Jan 2024 13:49:02 GMT

Hi @isoutamo ,

Yes, I can see the Entry data in a separate event, but there are no fields parsed through this.

I need fields and values to be parsed like the below table.

Field Name	Field Value
Sender Component	XYZ
Receiver Component	ABC
Interface	Mobile
System Error	-
Waiting	3

Re: Parse XML data into fields while indexing

PickleRick — Mon, 29 Jan 2024 15:32:45 GMT

Argh. This is a case of a very badly prepared data. And without external manipulation you won't get it into Splunk (or parse it after ingestion). The <Data> part of your data is so ugly organized - there is no explicit relationship between the <ColumnNames> and <DataRows>. Whoever thought about preparing data in this format did it very very wrong. Even if you were to manipulate it with a solution different than Splunk you'd have a hard time to compose those associations between column names and their values because of the need to externally keep the order of those values.