Hi
I want to extract the field names and field values of my events.
My event looks like this:
Step: 1000
Result: blabkbk
Actual: blabblabl
Step: 1100
Result: blabkbk
Actual: blabblabl
I want the field name to be "1000" and "1100"and the respective field values to be everything below them. This is how I set up my props.conf and transforms.conf, but I am not extracting anything. I appreciate your help.
props.conf
[<mysourcetype>]
REPORT-step_num = step_num
transforms.conf
[step_num]
REGEX = STEP:\s+(?<_KEY_1>/d+)\n(?<_VAL_1>[\w\W\n]+?)\nSTEP
Like this in transforms.conf
:
REGEX=(?ms)^Step:\s*([^\r\n]+)[\r\n]+(.*?)(?=\Z|[\r\n]+Step:)
CLEAN_KEYS = false
FORMAT = $1::$2
MV_ADD = 1
You should be able to test it on directly in search like this:
... | rex max_match=0 "(?ms)^Step:\s*(?<_KEY_1>[^\r\n]+)[\r\n]+(?<_VAL_1>.*?)(?=\Z|[\r\n]+Step:)"
Like this in transforms.conf
:
REGEX=(?ms)^Step:\s*([^\r\n]+)[\r\n]+(.*?)(?=\Z|[\r\n]+Step:)
CLEAN_KEYS = false
FORMAT = $1::$2
MV_ADD = 1
You should be able to test it on directly in search like this:
... | rex max_match=0 "(?ms)^Step:\s*(?<_KEY_1>[^\r\n]+)[\r\n]+(?<_VAL_1>.*?)(?=\Z|[\r\n]+Step:)"
I tried the regex and it doesn't extract anything. Also I read that the name of the group must start with an alpha value.
I tried the following first regex inline and it worked. I tried the second one and Splunk displayed an error related with the alpha value issue.
... | rex "STEP:\s+\d+\n(?<myvalue>[\w\W\n]+?)\nSTEP"
... | rex "STEP:\s+\d+\n(?<1myvalue>[\w\W\n]+?)\nSTEP"
The _KEY_1
and _VAL_1
are special (valid) exceptions:
I was guessing that it could be tested inline but perhaps it cannot. In any case, the transforms.conf will work. The (ms)
part is critical.
Yes, but I was talking about the invalid field names which are "1000" and "1100" in my example.
Those names are valid for Search Time, provided that you tell splunk not to clean them. To do that, be sure to set this:
CLEAN_KEYS = false
http://www.splunk.com/base/Documentation/6.3.0/Admin/Transformsconf
How have you configured you LINE-BREAKER? One suggestion is to set your LINE-BREAKER as follows
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Step:
Then you can define you AUTO-EXTRACT to be something like this
REGEX = ([^\W]+):=\s+([^\W]+
FORMAT = $1::$2
Just a thought. Not sure I understand your questions. What do you expect your end result to look like? How have you set up your event line-breaking? From your example, would you like Step: 1000 Result: blabkbk Actual: blabbk to be one event and so on?
If that's how you would like to see the events, then try adding the following to your props.conf
SHOULD_LINEMERGE = True
BREAK_ONLY_BEFORE = Step:
And for field extraction (AUTO-EXTRACT) you could use something like
REGEX = ([^\W]+):=\s+([^\W]+)
FORMAT = $1::$2
The above is just a general idea to get you started. Hopefully this helps.
The event cannot be divided because there are relevant information at the beginning of the event related to all the steps. If I divided the steps of the event, then the information in this step would not have any reference of what's about and they would be useless.
Just a note: STEP does not equal Step.