Splunk Search

Field extraction fails to extract all fields

bryancock
New Member

Hi All

I'm hoping someone might have an insight into a problem I'm having extracting fields from some XML based event data -- essentially I can only extract the first of 2 potential fields; the second field fails to be extracted and made available in the search app.

Example of the data I'm indexing:


<metrics>
<metric>
<name>diskavailablespaceC</name>
<value>1</value>
</metric>
<metric>
<name>diskavailablespaceE</name>
<value>7</value>
</metric>
</metrics>

My config files:

props.conf

[Power_test1]
TRANSFORMS-disk_available = disk_available

transforms.conf

[disk_available]
REGEX = (?i)<name>diskavailablespace(\w)</name>\s*?<value>(\d)</value>
FORMAT = $1::$2
WRITE_META = true

Some observations:

  • My expectation is that 2 fields should be extracted: C = 1 and E = 7.
  • Using the above, only a single field name 'C' is extracted from my indexed event data with value 1.
  • If I swap the order of the <metric> tags around, I extract a single field 'E' with value 7, suggesting to me that the regex is either failing or not being applied beyond the first match (??).
  • My regular expression, when tested in an external tool, seems to check out OK as it produces 2 matches of (2) groups --> one match for content of each <metric> tag.
  • I've added REPEAT_MATCH = true attribute with no change in behaviour.

I would appreciate any suggestions as to what may be going on, or what I might do to achieve the expected field extraction.

Many Thanks, B

Tags (1)
0 Karma

Ayn
Legend

Two things:

  1. You really shouldn't be using index-time extractions (TRANSFORMS) unless you really know what you're doing and have a good reason for it. Search-time extractions give better performance and flexibility.
  2. Only one value will be extracted because this is default behaviour when encountering more than one value for a field in an event. This behaviour can be toggled using the MV_ADD directive in transforms.conf. From the docs:

``

MV_ADD = [true|false]
* NOTE: This attribute is only valid for search-time field extractions.
* Optional. Controls what the extractor does when it finds a field which already exists.
* If set to true, the extractor makes the field a multivalued field and appends the 
* newly found value, otherwise the newly found value is discarded.
* Defaults to false
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...