Splunk Search

Invalid FORMAT when creating a field transformation

DUThibault
Contributor

I have these events that come with a source attribute something like source = /var/collectd/csv/sv3vm5b/cpu-0/cpu-idle-2018-01-10 and I need to extract the CPU number (the cpu-0 part, which can also be cpu-1, cpu-2, or cpu-3 ). So I tried to create (for my sourcetype) a transformation ( Fields: Field transformations: Add new ).

The destination app is search, the new field name is cpu, the type is regex-based with the regular expression ^.*/cpu-([0-9]+)/and the source key source. According to the form, the default format ( <transform_stanza_name>::$1 ) should do just fine so I leave the Format box blank. But it won't save, yielding this error message: Encountered the following error while trying to save: Invalid FORMAT: (I would add a screen capture but I don't have enough karma yet).

Help?

0 Karma
1 Solution

elliotproebstel
Champion

The recommended default isn't actually populated as a default value; it's just a suggestion. So try filling in the format box with cpu::$1 if that will work for you as a format.

View solution in original post

mayurr98
Super Champion

Hey edit your regex

^.*\/cpu-(?<cpu>[0-9]+)\/

Also in the format put

cpu::$1

Let me know if this works

0 Karma

DUThibault
Contributor

The slashes do not need escaping, and naming the capture group seems redundant (wouldn't the format then become "cpu::$cpu"?).

0 Karma

elliotproebstel
Champion

The recommended default isn't actually populated as a default value; it's just a suggestion. So try filling in the format box with cpu::$1 if that will work for you as a format.

micahkemp
Champion

And when configuring via the UI, it has to be in the form <fieldname>::<value>, you can't use just <value>.

0 Karma

DUThibault
Contributor

Having the Web interface state "default is" sounds like a lie, then.

Okay, this is starting to make sense. The process is:

1) Create a transformation ( Settings: (Knowledge) Fields: Field transformations: New )
2) Edit its permissions (if needed)
3) Create an extraction ( Settings: (Knowledge) Fields: Field extractions: New ) that uses the transformation
4) Edit its permissions (if needed)

The transformation:

destination app: search
name: TRANSFORM-COLLECTD-CSV-CPU-NUMBER
type: regex-based
regular expression: ^.*/cpu-([0-9]+)/
source key: source

The extraction:

destination app: search
name: COLLECTD-CSV-CPU-NUMBER (this will get a REPORT- prefix)
apply to: sourcetype
named: collectd_csv_cpu_idle
type: uses transform
extraction/transform: TRANSFORM-COLLECTD-CSV-CPU-NUMBER

The extraction will be listed as collectd_csv_cpu_idle : REPORT-COLLECTD-CSV-CPU-NUMBER . I can then create more extractions that use the same transform for other sourcetypes (e.g. collectd_csv_cpu_interrupt : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_nice : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_softirq : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_steal : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_system : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_user : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_wait : REPORT-COLLECTD-CSV-CPU-NUMBER )

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...