Splunk Search

Dynamic field extraction name

phemmer
Path Finder

I am trying to extract some fields from some reporting data. The reporting data has a field name, and then a bunch of metrics related to that field. For example a log entry might look like

report=memory ram_used=123mb;500;600;0;700 swap_used=456mb;1100;1200;0;1300

Each of the stats is "{CURRENT}[UNIT];{WARNING};{CRITICAL};{MINIMUM};{MAXIMUM}". I want to end up with fields such as ram_used_current=123, ram_used_unit=mb, swap_used_max=1300, etc.

My first thought was to solve this by doing a transform such as

[reporting]
REGEX = ([^\s=]+)=([0-9\.\-\+_]+)([a-zA-Z]*);([^\s;]*);([^\s;]*);([^\s;]*);([^\s;]*)
FORMAT = $1_current::$2 $1_unit::3 $1_warn::$4 $1_crit::$5 $1_min::$6 $1_max::$7

But this doesn't appear to work. It looks like it's just setting fields such as current instead of the desired ram_used_current.

I know in the transforms.conf documentation it says you cannot concatenate fields, but I'm assuming this means values, and not the field names. As you can create whatever field names you want by doing named regexes, so this seems to indicate the engine is capable of handling it.

kristian_kolb
Ultra Champion

You could make that as an EXTRACT in props.conf only. It will not alter the events, like a transform would do. It will work on already indexed data.

Something like this (not extracting the unit);

props.conf

[your sourcetype]
EXTRACT-YYY = ram_used=(?<ram_used_current>\d+)\w+;(?<ram_used_warn>\d+);(?<ram_used_crit>\d+);(?<ram_used_min>\d+);(?<ram_used_max>\d+);
EXTRACT-XXX = swap_used=(?<swap_used_current>\d+) etc etc etc

/K

0 Karma

phemmer
Path Finder

This requires me to write a regex for every single group of statistics we have. There are hundreds of them, they change frequently, and some of them are even dynamic (they come and go automatically).

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...