topic Re: How to configure Splunk to extract fields from XML data? in Splunk Search

How to configure Splunk to extract fields from XML data?

smudge797 — Tue, 26 Apr 2016 22:01:22 GMT

Splunk is not recognizing the fields. What is the recommended method to extract these fields, especially username which can be upper or lower case letters followed by 6 digits:

Re: How to configure Splunk to extract fields from XML data?

diogofgm — Tue, 26 Apr 2016 23:08:29 GMT

Can you post the sample again?

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Wed, 27 Apr 2016 06:43:05 GMT

updated. thanks

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Wed, 27 Apr 2016 06:46:49 GMT

The username field can contain sometimes the first character of the users ID is uppercase, and other times they are lower case. In other cases, an engineer logs in for remote assistance and two or three sets of IDs where the right most ID is the one we want.

Re: How to configure Splunk to extract fields from XML data?

jmallorquin — Wed, 27 Apr 2016 09:25:55 GMT

Hi,

You can use the rex command to try that the extraction is correct, and then add to props.conf

Use this command in search to create the field user

|rex "\<username\>(?<user>[^\<]+)"

Hope i help you

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Wed, 27 Apr 2016 09:29:07 GMT

Thanks. So in a distributed environment where would that sit? Forwarder IDX or SH?

Re: How to configure Splunk to extract fields from XML data?

jmallorquin — Wed, 27 Apr 2016 09:44:07 GMT

Hi,

Only in the SH

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Wed, 27 Apr 2016 09:52:41 GMT

Thanks but it does not look like a valid stanza for props.

Re: How to configure Splunk to extract fields from XML data?

jmallorquin — Wed, 27 Apr 2016 09:57:21 GMT

Hi,

In props.conf you have to use this :

EXTRACT-user = \<username\>(?<user>[^\<]+)

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Wed, 27 Apr 2016 10:53:16 GMT

great thanks.

Re: How to configure Splunk to extract fields from XML data?

diogofgm — Wed, 27 Apr 2016 12:07:55 GMT

Can you try this?

props.conf
[your_sourcetype]
TIME_FORMAT=%b %d %Y %H:%M
TIME_PREFIX=<stamp>
SHOULD_LINEMERGE=false
BREAK_ONLY_BEFORE=\<CloneInfo\>
MUST_BREAK_AFTER=\<\/CloneInfo\>
LINE_BREAKER=(</CloneInfo>)
REPORT-xmlfields=xmlfields

transforms.conf
[xmlfields]
REGEX = <([^\>]*)>([^\<]*)
FORMAT = $1::$2

This will extract every field like this:
<field_name>value</field_name>

its also extracting the _timestamp from the field you already have in you data. Does you XML have multiple tags? With this configs every becomes a separate event making the data more readable.

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Fri, 29 Apr 2016 21:58:27 GMT

Thats great thanks. We have one issue that the username can be a multi value entry. This is because support may be remotely logged onto same host so could have 1 2 or 3 entries of username. The one we are interested in is the last value so could be:
username= a123456 b123456 c123456
username= b123456 c123456
username= c123456
Any way of exluding the others and ensuring the last one is always picked up in query?

Thanks

Re: How to configure Splunk to extract fields from XML data?

diogofgm — Mon, 02 May 2016 13:01:32 GMT

can you post an example?

Re: How to configure Splunk to extract fields from XML data?

smudge797 — Tue, 03 May 2016 11:13:24 GMT

username as shown in screen shot a123456 b123456 c123456" followed by /username as shown.

It wont let me post as written.