Splunk Search

Invalid FORMAT when creating a field transformation

DUThibault
Contributor

I have these events that come with a source attribute something like source = /var/collectd/csv/sv3vm5b/cpu-0/cpu-idle-2018-01-10 and I need to extract the CPU number (the cpu-0 part, which can also be cpu-1, cpu-2, or cpu-3 ). So I tried to create (for my sourcetype) a transformation ( Fields: Field transformations: Add new ).

The destination app is search, the new field name is cpu, the type is regex-based with the regular expression ^.*/cpu-([0-9]+)/and the source key source. According to the form, the default format ( <transform_stanza_name>::$1 ) should do just fine so I leave the Format box blank. But it won't save, yielding this error message: Encountered the following error while trying to save: Invalid FORMAT: (I would add a screen capture but I don't have enough karma yet).

Help?

0 Karma
1 Solution

elliotproebstel
Champion

The recommended default isn't actually populated as a default value; it's just a suggestion. So try filling in the format box with cpu::$1 if that will work for you as a format.

View solution in original post

mayurr98
Super Champion

Hey edit your regex

^.*\/cpu-(?<cpu>[0-9]+)\/

Also in the format put

cpu::$1

Let me know if this works

0 Karma

DUThibault
Contributor

The slashes do not need escaping, and naming the capture group seems redundant (wouldn't the format then become "cpu::$cpu"?).

0 Karma

elliotproebstel
Champion

The recommended default isn't actually populated as a default value; it's just a suggestion. So try filling in the format box with cpu::$1 if that will work for you as a format.

micahkemp
Champion

And when configuring via the UI, it has to be in the form <fieldname>::<value>, you can't use just <value>.

0 Karma

DUThibault
Contributor

Having the Web interface state "default is" sounds like a lie, then.

Okay, this is starting to make sense. The process is:

1) Create a transformation ( Settings: (Knowledge) Fields: Field transformations: New )
2) Edit its permissions (if needed)
3) Create an extraction ( Settings: (Knowledge) Fields: Field extractions: New ) that uses the transformation
4) Edit its permissions (if needed)

The transformation:

destination app: search
name: TRANSFORM-COLLECTD-CSV-CPU-NUMBER
type: regex-based
regular expression: ^.*/cpu-([0-9]+)/
source key: source

The extraction:

destination app: search
name: COLLECTD-CSV-CPU-NUMBER (this will get a REPORT- prefix)
apply to: sourcetype
named: collectd_csv_cpu_idle
type: uses transform
extraction/transform: TRANSFORM-COLLECTD-CSV-CPU-NUMBER

The extraction will be listed as collectd_csv_cpu_idle : REPORT-COLLECTD-CSV-CPU-NUMBER . I can then create more extractions that use the same transform for other sourcetypes (e.g. collectd_csv_cpu_interrupt : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_nice : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_softirq : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_steal : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_system : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_user : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_wait : REPORT-COLLECTD-CSV-CPU-NUMBER )

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...