topic Re: How to extract a field name and add it to a fixed string? in Splunk Search

How to extract a field name and add it to a fixed string?

edrivera3 — Tue, 29 Sep 2020 07:54:06 GMT

How can I extract these fieldnames and values from this event?

Step: 0345
Result: Valid
Step: 3345
Result: Valid

Field names and values:
1. Field Name: test_0345, Field Value: Valid
2. Field Name: test_3345, Field Value: Valid

I already know how to extract both the name and value, but I don't know how to add "test_" before all extracted field names.

Edit (more info):
The raw data has only one timestamp and it is located at the beginning of the file. This the reason why I didn't divide the file in multiple events.
I have events with thousands fields so renaming is not option.

At this moment, the fields look like this:

1350=Valid
2342=Valid
2345=Failed
2341=Valid

I'm looking for a solution using transforms.conf and props.conf so these fields will be available for all search in this sourcetype.

My current transforms.conf:

 REGEX=(?ms)^Step:\s*([^\r\n]+)[\r\n]+(.*?)(?=\Z|[\r\n]+Step:)
 CLEAN_KEYS = false
 FORMAT = $1::$2
 MV_ADD = 1

Re: How to extract a field name and add it to a fixed string?

esix_splunk — Thu, 12 Nov 2015 01:59:56 GMT

One quick way would be to re-name the fields after you have them extracted..

.. | rename 0345 AS test_0345

This isnt so efficient with large number of fields though. You can also do

.. | rename 0* AS test_0* | ..

You can wildcard match on field names. However, we cant match on regex patterns that Im aware off. So you would need to iterate through the number patterns of your fields to get this for all fields. Which again, is time consuming if you have a large number of fields..

Re: How to extract a field name and add it to a fixed string?

woodcock — Thu, 12 Nov 2015 16:36:37 GMT

Assuming that you have fields called Step and Result, this should work:

| eval Step = "test_" . Step | eval {Step} = Result | fields - Step Result

Re: How to extract a field name and add it to a fixed string?

edrivera3 — Thu, 12 Nov 2015 16:41:07 GMT

I should have included this in my question. Renaming is not an option because I have thousands of steps. The raw data only has one timestamp at the beginning of the file and that's the reason why I didn't divide the file in multiple events. Also I need these fieldnames to be a fix name for all search.

Re: How to extract a field name and add it to a fixed string?

edrivera3 — Thu, 12 Nov 2015 16:55:34 GMT

I updated the question with more information.

Re: How to extract a field name and add it to a fixed string?

woodcock — Thu, 12 Nov 2015 17:16:02 GMT

Because calculated fields happen last, you should be able to put this inside your props.conf like this:

EVAL-Step = "test_" . Step | eval {Step} = Result

Re: How to extract a field name and add it to a fixed string?

edrivera3 — Thu, 12 Nov 2015 17:30:20 GMT

But I don't have a field called result. I only have fields called 1234,1235,1236, etc. I can't divide the capturing in two fields because there is more information that just the "Valid or Failed" status. As you can see in the regex everything between "Step" words is captured. If I divide the fields I might end up with steps with wrong values or with multiple values.

Right now I am looking at the documentation and I see that there is way to concatenate a fix string using FORMAT.
http://docs.splunk.com/Documentation/Splunk/6.1/admin/Transformsconf

I have tried the following FORMAT but I haven't been successful:

FORMAT = step_$1::$2

Re: How to extract a field name and add it to a fixed string?

edrivera3 — Thu, 12 Nov 2015 17:38:27 GMT

It is not possible to concatenated fields with Format at search time:

NOTE: You cannot create concatenated fields with FORMAT at search time. That 
      functionality is only available at index time.

Re: How to extract a field name and add it to a fixed string?

woodcock — Thu, 12 Nov 2015 17:43:29 GMT

Right, change to this and then it should all work (but you will have "extra" fields as a result):

  FORMAT = Step::$1 Result::$2

Re: How to extract a field name and add it to a fixed string?

edrivera3 — Thu, 12 Nov 2015 21:05:46 GMT

I managed to capture the two fields with

FORMAT = Step::$1 Result::$2

but when I added:

EVAL-Step = "test_" . Step | eval {Step} = Result

I don't see any fields. If I changed the previous EVAL to REPORT like this

Report-step_num = step_num  #step_num is the name of the stanza

I see both fields, so there is a problem with the EVAL.

Re: How to extract a field name and add it to a fixed string?

woodcock — Fri, 13 Nov 2015 15:17:04 GMT

The syntax is special and needs the pipe, either one of which may not be supported by calculated fields so you will have to experiment. You may end up having to do the last steps at search-time.