Splunk Search

How to extract multiple values on one field from XML structured data?

Buscatrufas
Path Finder

Hi,

I want to split data from this XML structure, but I cannot because the extracted field only gets the first element.

<vuln:vulnerable-software-list>
      <vuln:product>cpe:/o:novell:opensuse:13.1</vuln:product>
      <vuln:product>cpe:/a:samba:rsync:3.1.1</vuln:product>
      <vuln:product>cpe:/o:novell:opensuse:13.2</vuln:product>
  </vuln:vulnerable-software-list>

I edited the file fields.conf too with the following syntax:

[name]
TOKENIZER = (cpe(:\/[\:\w\.]+))

But nothing happens. Any help? Thanks!

Best regards, Buscatrufas

0 Karma
1 Solution

aljohnson_splun
Splunk Employee
Splunk Employee

Are you sure you want to extract these at index time? It is unlikely that you really want to do that:

From the docs here:

Caution: Do not add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time unless absolutely necessary. This includes fields such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see Index time versus search time in the Managing Indexers and Clusters manual.

Instead, you can use props.conf and transforms.conf to extract during search time,

Let's say your data has a sourcetype of vuln.

Props.conf:

[vuln]
REPORT = vuln_extractions

So whats going on here? The REPORT tells Splunk to look in transforms.conf for a stanza called vuln_extractions. It is simply linking the regular expression we will define to the vuln sourcetype.

Then we need Transfroms.conf:

[vuln_extractions]
REGEX = (?<_KEY_1>[\w\.]+):(?<_VAL_1>[\w\.]+)

Whats going on here? The REGEX line defines the regex we want to use. The _KEY_1 and _VAL_1 capture group names are special and Splunk already knows what to do with them. You can read more about them and this approach in transforms.conf.spec.

Here is an example of what the regex would extract: regex101

The transform should be applied by default, however, if you wanted to apply the same to a different sourcetype, ad-hoc, you could use the extract command to apply the transform, e.g.

sourcetype=some_other_sourcetype | extract vuln

View solution in original post

aljohnson_splun
Splunk Employee
Splunk Employee

Are you sure you want to extract these at index time? It is unlikely that you really want to do that:

From the docs here:

Caution: Do not add custom fields to the set of default fields that Splunk automatically extracts and indexes at index time unless absolutely necessary. This includes fields such as timestamp, punct, host, source, and sourcetype. Adding to this list of fields can negatively impact indexing performance and search times, because each indexed field increases the size of the searchable index. Indexed fields are also less flexible--whenever you make changes to your set of fields, you must re-index your entire dataset. For more information, see Index time versus search time in the Managing Indexers and Clusters manual.

Instead, you can use props.conf and transforms.conf to extract during search time,

Let's say your data has a sourcetype of vuln.

Props.conf:

[vuln]
REPORT = vuln_extractions

So whats going on here? The REPORT tells Splunk to look in transforms.conf for a stanza called vuln_extractions. It is simply linking the regular expression we will define to the vuln sourcetype.

Then we need Transfroms.conf:

[vuln_extractions]
REGEX = (?<_KEY_1>[\w\.]+):(?<_VAL_1>[\w\.]+)

Whats going on here? The REGEX line defines the regex we want to use. The _KEY_1 and _VAL_1 capture group names are special and Splunk already knows what to do with them. You can read more about them and this approach in transforms.conf.spec.

Here is an example of what the regex would extract: regex101

The transform should be applied by default, however, if you wanted to apply the same to a different sourcetype, ad-hoc, you could use the extract command to apply the transform, e.g.

sourcetype=some_other_sourcetype | extract vuln

Buscatrufas
Path Finder

I love you

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...