topic Re: Extracting calculated fields from an XML input doesn't work in Splunk Search

Extracting calculated fields from an XML input doesn't work

arkadyz1 — Thu, 08 Oct 2015 17:26:43 GMT

I'm trying to extract some fields from an XML input. The sourcetype is set up correctly, and I get all kinds of extracted fields with long names such as objectdata.general.timestamp, objectdata.general.width{@unit} etc. My problem is this: calculated fields simply don't pick up the field names extracted from XML. For example, if I have an event looking like this:

<objectdata>
...
  <general>
  ...
    <width unit="inch">
      <value>17.3</value>
    </width>
    ...
  </general>
  ...
</objectdata>

In search, I see objectdata.general.width{@unit} as 'inch' and objectdata.general.width.value as 17.3. However, some of the objects have their width in 'mm', which, of course, needs to be divided by 25.4 to be converted to inches.

I wanted to calculate a new field and put this calculation in props.conf:

EVAL-objectWidth = objectdata.general.width.value / if(objectdata.general.width{@unit}=='mm',25.4,1)

but it doesn't work. No objectWidth field is created. I suspect it has something to do with the order of field extractions and evaluations in the pipeline. Is there a reasonable way to achieve what I want without putting the calculations into the search command each time?

Re: Extracting calculated fields from an XML input doesn't work

masonmorales — Thu, 08 Oct 2015 18:26:07 GMT

It's likely due to the field names containing special characters. You could use a fieldalias to rename the fields, and then do the EVAL with the renamed fields, or you could try this:

EVAL-objectWidth = 'objectdata.general.width.value' /  (if('objectdata.general.width{@unit}'="mm",25.4,1))

FYI I have not tested the above EVAL expression

Re: Extracting calculated fields from an XML input doesn't work

masonmorales — Thu, 08 Oct 2015 18:27:15 GMT

Troubleshooting tip: Test the EVAL in search and get it to work there before putting it into props.conf 😉

Re: Extracting calculated fields from an XML input doesn't work

arkadyz1 — Thu, 08 Oct 2015 18:52:36 GMT

First of all, those single quotes around the names helped me in at least one other place - I had to distinguish between objectdata.timestamp and objectdata.general.timestamp, and EVAL-MyTimestamp='objectdata.general.timestamp' did solve this (I was not using single quotes previously).

However, the more complex eval with the if did not benefit enough from those - no field generated :(. And the eval does work from the search string! I tried both == and a single = as a comparison operator in props.conf to no avail. Search string accepts the C-style comparison (==) just fine.

Re: Extracting calculated fields from an XML input doesn't work

arkadyz1 — Fri, 09 Oct 2015 18:11:09 GMT

I also tried to use FIELDALIAS. The fields are aliased quite well, but the EVAL (working in search strings) is not working in props.conf :(.