Splunk Search

How to configure Splunk to extract fields from XML data?

smudge797
Path Finder

Splunk is not recognizing the fields. What is the recommended method to extract these fields, especially username which can be upper or lower case letters followed by 6 digits:

alt text

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Can you try this?

props.conf
[your_sourcetype]
TIME_FORMAT=%b %d %Y %H:%M
TIME_PREFIX=<stamp>
SHOULD_LINEMERGE=false
BREAK_ONLY_BEFORE=\<CloneInfo\>
MUST_BREAK_AFTER=\<\/CloneInfo\>
LINE_BREAKER=(</CloneInfo>)
REPORT-xmlfields=xmlfields

transforms.conf
[xmlfields]
REGEX = <([^\>]*)>([^\<]*)
FORMAT = $1::$2

This will extract every field like this:
<field_name>value</field_name>

its also extracting the _timestamp from the field you already have in you data. Does you XML have multiple tags? With this configs every becomes a separate event making the data more readable.

------------
Hope I was able to help you. If so, some karma would be appreciated.

smudge797
Path Finder

Thats great thanks. We have one issue that the username can be a multi value entry. This is because support may be remotely logged onto same host so could have 1 2 or 3 entries of username. The one we are interested in is the last value so could be:
username= a123456 b123456 c123456
username= b123456 c123456
username= c123456
Any way of exluding the others and ensuring the last one is always picked up in query?

Thanks

0 Karma

diogofgm
SplunkTrust
SplunkTrust

can you post an example?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

smudge797
Path Finder

username as shown in screen shot a123456 b123456 c123456" followed by /username as shown.

It wont let me post as written.

0 Karma

jmallorquin
Builder

Hi,

You can use the rex command to try that the extraction is correct, and then add to props.conf

Use this command in search to create the field user

|rex "\<username\>(?<user>[^\<]+)"

Hope i help you

0 Karma

smudge797
Path Finder

Thanks. So in a distributed environment where would that sit? Forwarder IDX or SH?

0 Karma

jmallorquin
Builder

Hi,

Only in the SH

0 Karma

smudge797
Path Finder

Thanks but it does not look like a valid stanza for props.

0 Karma

jmallorquin
Builder

Hi,

In props.conf you have to use this :

EXTRACT-user = \<username\>(?<user>[^\<]+)
0 Karma

smudge797
Path Finder

great thanks.

0 Karma

diogofgm
SplunkTrust
SplunkTrust

Can you post the sample again?

------------
Hope I was able to help you. If so, some karma would be appreciated.
0 Karma

smudge797
Path Finder

The username field can contain sometimes the first character of the users ID is uppercase, and other times they are lower case. In other cases, an engineer logs in for remote assistance and two or three sets of IDs where the right most ID is the one we want.

0 Karma

smudge797
Path Finder

updated. thanks

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...