Splunk Search

Is it possible to use an extracted field inside a regex?

Builder

Hi

I already extracted a field (block_num) in my event, but now I would like to use it as part of a new regex. I want to do something like this:
...| rex field=_raw " Block number block_num (? < block_info>\w{1,}?)" ---- where block_num is the field I already have.

0 Karma

Splunk Employee
Splunk Employee

It would really help if you gave an accurate picture of the event. To help you, we need to see what the interior of the "Block" looks like.

That said, this is a two step process and as someone mentioned you've done it backwards.

Extract the entire block as a field:
rex field=_raw "(?s)(?Pblock\snumber\s\d+.+)End"
field=_raw is the default so you don't need it on the first one.

second rex:
rex field=block "block\snumber\s\d+(?P.+)End"

If you show me what the inside of the block looks like... I can give you a more specific way to grab the inside.

you don't have to do two rex commands if you do an extraction for the bock field. you can still reference it in the rex command

so if it is inline it would look like:

...|rex field=_raw "(?s)(?Pblock\snumber\s\d+.+)End"|rex field=block "block\snumber\s\d+(?P.+)End"

then you can also extract the block_num field as another rex or extraction :

rex field=block "block\snumber\s(?P\d+)"

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

SplunkTrust
SplunkTrust

Try something like this:

...| rex field=_raw " Block number ".block_num." (?P<block_info>\w{1,}?)" | ...
---
If this reply helps you, an upvote would be appreciated.

Engager

This doesn't seem to work.

0 Karma

Explorer

Excellent. This did exactly what i needed (except i was doing this in a match)!

0 Karma

Builder

Well, there is no error but it doesn't match anything but thanks

0 Karma

Motivator

Are these things in the same event? What does an event look like?

0 Karma

Builder

Yes, they are in the same event. The event is quite a long and mostly a text. The structure of the events are like:
block number 500
... info (sometimes there are errors here)
.... info
End of block number
block number 501
....info
...info
End of block number 501
First I extract the error, then the block number where the error is, and finally I want to extract the whole block. Well... that's the idea.

0 Karma

Motivator

It looks like you're doing that inside out. Why not extract all the blocks first, and then filter your results based on whether there's an error in the block or not?

0 Karma

Builder

I extracted all blocks, but how I search for an error inside each block text.

0 Karma

Motivator

Use whatever error-finding regex you were using before on each block.

Alternatively, you can try to extract both at once:

rex field=_raw "Block number (?<block_num>\d+)(.*)[YOUR ERROR-EXTRACTING REGEX GOES HERE](.*)End of block number"
0 Karma

Builder

But you are only extracting the block_num which I already have.

About the alternative about using a error-finding regex after I extracted all blocks. How can I use a regex to look inside field values?

The following doesn't work because the block info is several lines, not only an error number:
...|search block_info="ERROR"

0 Karma

Builder

I think these are some of the options that I am looking for:

(1) a way to use a extracted field inside a regex

(2) If I extract all blocks in the event, is there is a way to look inside the field value to find if there is an error inside the block. Please consider that the field value is a string with several lines that include the word "ERROR".

(3) Is there is a way to use regex for looking for a word and extract some lines that are before it and after including the word?

0 Karma

SplunkTrust
SplunkTrust

For #1, see my answer.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Builder

I tried that but I encountered an error:
Error in 'rex' command: Encountered the following error while compiling the regex 'Block number block_num (?P<.block_info.>\w{1,}?)': Regex: syntax error in subpattern name (missing terminator)

This is my regex:
...| rex field=_raw "Block number block_num (?P<".block_info.">\w{1,}?)"

0 Karma

SplunkTrust
SplunkTrust

I got no errors when I used concatenation in my rex command. What version of Splunk are you using?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Builder

Version 6.2

0 Karma

SplunkTrust
SplunkTrust

I'm on 6.1. Perhaps that explains the difference.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Builder

I am not sure how that rex command is concatenating. I also don't get why you add the quotation mark and a point before and after the field that I want to extract. If I remove the points the rex command runs but without results.

Also remember that "block_num" is a field I already extracted.

0 Karma

SplunkTrust
SplunkTrust

Concatenation is done by the dot '.' (point) operator. The quotation marks close the string before the field and open the string after the field. The three pieces are then combined into a single string for rex to process.

I think I mis-read your question and gave you an incorrect answer. I've updated my answer to use block_num as the field name.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Builder

ok. I'm going to give it a try.

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!