Splunk Search

Extracting calculated fields from an XML input doesn't work

arkadyz1
Builder

I'm trying to extract some fields from an XML input. The sourcetype is set up correctly, and I get all kinds of extracted fields with long names such as objectdata.general.timestamp, objectdata.general.width{@unit} etc. My problem is this: calculated fields simply don't pick up the field names extracted from XML. For example, if I have an event looking like this:

<objectdata>
...
  <general>
  ...
    <width unit="inch">
      <value>17.3</value>
    </width>
    ...
  </general>
  ...
</objectdata>

In search, I see objectdata.general.width{@unit} as 'inch' and objectdata.general.width.value as 17.3. However, some of the objects have their width in 'mm', which, of course, needs to be divided by 25.4 to be converted to inches.

I wanted to calculate a new field and put this calculation in props.conf:

EVAL-objectWidth = objectdata.general.width.value / if(objectdata.general.width{@unit}=='mm',25.4,1)

but it doesn't work. No objectWidth field is created. I suspect it has something to do with the order of field extractions and evaluations in the pipeline. Is there a reasonable way to achieve what I want without putting the calculations into the search command each time?

0 Karma
1 Solution

masonmorales
Influencer

It's likely due to the field names containing special characters. You could use a fieldalias to rename the fields, and then do the EVAL with the renamed fields, or you could try this:

EVAL-objectWidth = 'objectdata.general.width.value' /  (if('objectdata.general.width{@unit}'="mm",25.4,1))

FYI I have not tested the above EVAL expression

View solution in original post

masonmorales
Influencer

It's likely due to the field names containing special characters. You could use a fieldalias to rename the fields, and then do the EVAL with the renamed fields, or you could try this:

EVAL-objectWidth = 'objectdata.general.width.value' /  (if('objectdata.general.width{@unit}'="mm",25.4,1))

FYI I have not tested the above EVAL expression

arkadyz1
Builder

I also tried to use FIELDALIAS. The fields are aliased quite well, but the EVAL (working in search strings) is not working in props.conf :(.

0 Karma

masonmorales
Influencer

Troubleshooting tip: Test the EVAL in search and get it to work there before putting it into props.conf 😉

0 Karma

arkadyz1
Builder

First of all, those single quotes around the names helped me in at least one other place - I had to distinguish between objectdata.timestamp and objectdata.general.timestamp, and EVAL-MyTimestamp='objectdata.general.timestamp' did solve this (I was not using single quotes previously).

However, the more complex eval with the if did not benefit enough from those - no field generated :(. And the eval does work from the search string! I tried both == and a single = as a comparison operator in props.conf to no avail. Search string accepts the C-style comparison (==) just fine.

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...