Splunk Search

How to extract a field name and add it to a fixed string?

edrivera3
Builder

Hi

How can I extract these fieldnames and values from this event?

Step: 0345
Result: Valid
Step: 3345
Result: Valid

Field names and values:
1. Field Name: test_0345, Field Value: Valid
2. Field Name: test_3345, Field Value: Valid

I already know how to extract both the name and value, but I don't know how to add "test_" before all extracted field names.

Edit (more info):
The raw data has only one timestamp and it is located at the beginning of the file. This the reason why I didn't divide the file in multiple events.
I have events with thousands fields so renaming is not option.

At this moment, the fields look like this:

1350=Valid
2342=Valid
2345=Failed
2341=Valid

I'm looking for a solution using transforms.conf and props.conf so these fields will be available for all search in this sourcetype.

My current transforms.conf:

 REGEX=(?ms)^Step:\s*([^\r\n]+)[\r\n]+(.*?)(?=\Z|[\r\n]+Step:)
 CLEAN_KEYS = false
 FORMAT = $1::$2
 MV_ADD = 1
0 Karma

woodcock
Esteemed Legend

Assuming that you have fields called Step and Result, this should work:

| eval Step = "test_" . Step | eval {Step} = Result | fields - Step Result
0 Karma

woodcock
Esteemed Legend

Because calculated fields happen last, you should be able to put this inside your props.conf like this:

EVAL-Step = "test_" . Step | eval {Step} = Result
0 Karma

edrivera3
Builder

But I don't have a field called result. I only have fields called 1234,1235,1236, etc. I can't divide the capturing in two fields because there is more information that just the "Valid or Failed" status. As you can see in the regex everything between "Step" words is captured. If I divide the fields I might end up with steps with wrong values or with multiple values.

Right now I am looking at the documentation and I see that there is way to concatenate a fix string using FORMAT.
http://docs.splunk.com/Documentation/Splunk/6.1/admin/Transformsconf

I have tried the following FORMAT but I haven't been successful:

FORMAT = step_$1::$2
0 Karma

edrivera3
Builder

It is not possible to concatenated fields with Format at search time:

NOTE: You cannot create concatenated fields with FORMAT at search time. That 
      functionality is only available at index time.
0 Karma

woodcock
Esteemed Legend

Right, change to this and then it should all work (but you will have "extra" fields as a result):

  FORMAT = Step::$1 Result::$2
0 Karma

edrivera3
Builder

I managed to capture the two fields with

FORMAT = Step::$1 Result::$2

but when I added:

EVAL-Step = "test_" . Step | eval {Step} = Result

I don't see any fields. If I changed the previous EVAL to REPORT like this

Report-step_num = step_num  #step_num is the name of the stanza

I see both fields, so there is a problem with the EVAL.

0 Karma

woodcock
Esteemed Legend

The syntax is special and needs the pipe, either one of which may not be supported by calculated fields so you will have to experiment. You may end up having to do the last steps at search-time.

0 Karma

edrivera3
Builder

I updated the question with more information.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

One quick way would be to re-name the fields after you have them extracted..

.. | rename 0345 AS test_0345 

This isnt so efficient with large number of fields though. You can also do

.. | rename 0* AS test_0* | ..

You can wildcard match on field names. However, we cant match on regex patterns that Im aware off. So you would need to iterate through the number patterns of your fields to get this for all fields. Which again, is time consuming if you have a large number of fields..

0 Karma

edrivera3
Builder

I should have included this in my question. Renaming is not an option because I have thousands of steps. The raw data only has one timestamp at the beginning of the file and that's the reason why I didn't divide the file in multiple events. Also I need these fieldnames to be a fix name for all search.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.