Invalid FORMAT when creating a field transformation

[DUThibault]

I have these events that come with a source attribute something like source = /var/collectd/csv/sv3vm5b/cpu-0/cpu-idle-2018-01-10 and I need to extract the CPU number (the cpu-0 part, which can also be cpu-1, cpu-2, or cpu-3 ). So I tried to create (for my sourcetype) a transformation ( Fields: Field transformations: Add new ).

The destination app is search, the new field name is cpu, the type is regex-based with the regular expression ^.*/cpu-([0-9]+)/and the source key source. According to the form, the default format ( <transform_stanza_name>::$1 ) should do just fine so I leave the Format box blank. But it won't save, yielding this error message: Encountered the following error while trying to save: Invalid FORMAT: (I would add a screen capture but I don't have enough karma yet).

Help?

[mayurr98]

Hey edit your regex

^.*\/cpu-(?<cpu>[0-9]+)\/

Also in the format put

cpu::$1

Let me know if this works

[DUThibault]

The slashes do not need escaping, and naming the capture group seems redundant (wouldn't the format then become "cpu::$cpu"?).

[elliotproebstel · Accepted solution]

The recommended default isn't actually populated as a default value; it's just a suggestion. So try filling in the format box with cpu::$1 if that will work for you as a format.

[micahkemp]

And when configuring via the UI, it has to be in the form <fieldname>::<value>, you can't use just <value>.

[DUThibault]

Having the Web interface state "default is" sounds like a lie, then.

Okay, this is starting to make sense. The process is:

1) Create a transformation ( Settings: (Knowledge) Fields: Field transformations: New )
2) Edit its permissions (if needed)
3) Create an extraction ( Settings: (Knowledge) Fields: Field extractions: New ) that uses the transformation
4) Edit its permissions (if needed)

The transformation:

destination app: search
name: TRANSFORM-COLLECTD-CSV-CPU-NUMBER
type: regex-based
regular expression: ^.*/cpu-([0-9]+)/
source key: source

The extraction:

destination app: search
name: COLLECTD-CSV-CPU-NUMBER (this will get a REPORT- prefix)
apply to: sourcetype
named: collectd_csv_cpu_idle
type: uses transform
extraction/transform: TRANSFORM-COLLECTD-CSV-CPU-NUMBER

The extraction will be listed as collectd_csv_cpu_idle : REPORT-COLLECTD-CSV-CPU-NUMBER . I can then create more extractions that use the same transform for other sourcetypes (e.g. collectd_csv_cpu_interrupt : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_nice : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_softirq : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_steal : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_system : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_user : REPORT-COLLECTD-CSV-CPU-NUMBER , collectd_csv_cpu_wait : REPORT-COLLECTD-CSV-CPU-NUMBER )