Splunk Search

Is it possible to use an extracted field inside a regex?

edrivera3
Builder

Hi

I already extracted a field (block_num) in my event, but now I would like to use it as part of a new regex. I want to do something like this:
...| rex field=_raw " Block number block_num (? < block_info>\w{1,}?)" ---- where block_num is the field I already have.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

It would really help if you gave an accurate picture of the event. To help you, we need to see what the interior of the "Block" looks like.

That said, this is a two step process and as someone mentioned you've done it backwards.

Extract the entire block as a field:
rex field=_raw "(?s)(?Pblock\snumber\s\d+.+)End"
field=_raw is the default so you don't need it on the first one.

second rex:
rex field=block "block\snumber\s\d+(?P.+)End"

If you show me what the inside of the block looks like... I can give you a more specific way to grab the inside.

you don't have to do two rex commands if you do an extraction for the bock field. you can still reference it in the rex command

so if it is inline it would look like:

...|rex field=_raw "(?s)(?Pblock\snumber\s\d+.+)End"|rex field=block "block\snumber\s\d+(?P.+)End"

then you can also extract the block_num field as another rex or extraction :

rex field=block "block\snumber\s(?P\d+)"

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

richgalloway
SplunkTrust
SplunkTrust

Try something like this:

...| rex field=_raw " Block number ".block_num." (?P<block_info>\w{1,}?)" | ...
---
If this reply helps you, Karma would be appreciated.

davstone
Engager

This doesn't seem to work.

0 Karma

chris24747
Explorer

Excellent. This did exactly what i needed (except i was doing this in a match)!

0 Karma

edrivera3
Builder

Well, there is no error but it doesn't match anything but thanks

0 Karma

aweitzman
Motivator

Are these things in the same event? What does an event look like?

0 Karma

edrivera3
Builder

Yes, they are in the same event. The event is quite a long and mostly a text. The structure of the events are like:
block number 500
... info (sometimes there are errors here)
.... info
End of block number
block number 501
....info
...info
End of block number 501
First I extract the error, then the block number where the error is, and finally I want to extract the whole block. Well... that's the idea.

0 Karma

aweitzman
Motivator

It looks like you're doing that inside out. Why not extract all the blocks first, and then filter your results based on whether there's an error in the block or not?

0 Karma

edrivera3
Builder

I extracted all blocks, but how I search for an error inside each block text.

0 Karma

aweitzman
Motivator

Use whatever error-finding regex you were using before on each block.

Alternatively, you can try to extract both at once:

rex field=_raw "Block number (?<block_num>\d+)(.*)[YOUR ERROR-EXTRACTING REGEX GOES HERE](.*)End of block number"
0 Karma

edrivera3
Builder

But you are only extracting the block_num which I already have.

About the alternative about using a error-finding regex after I extracted all blocks. How can I use a regex to look inside field values?

The following doesn't work because the block info is several lines, not only an error number:
...|search block_info="ERROR"

0 Karma

edrivera3
Builder

I think these are some of the options that I am looking for:

(1) a way to use a extracted field inside a regex

(2) If I extract all blocks in the event, is there is a way to look inside the field value to find if there is an error inside the block. Please consider that the field value is a string with several lines that include the word "ERROR".

(3) Is there is a way to use regex for looking for a word and extract some lines that are before it and after including the word?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

For #1, see my answer.

---
If this reply helps you, Karma would be appreciated.
0 Karma

edrivera3
Builder

I tried that but I encountered an error:
Error in 'rex' command: Encountered the following error while compiling the regex 'Block number block_num (?P<.block_info.>\w{1,}?)': Regex: syntax error in subpattern name (missing terminator)

This is my regex:
...| rex field=_raw "Block number block_num (?P<".block_info.">\w{1,}?)"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I got no errors when I used concatenation in my rex command. What version of Splunk are you using?

---
If this reply helps you, Karma would be appreciated.
0 Karma

edrivera3
Builder

Version 6.2

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I'm on 6.1. Perhaps that explains the difference.

---
If this reply helps you, Karma would be appreciated.
0 Karma

edrivera3
Builder

I am not sure how that rex command is concatenating. I also don't get why you add the quotation mark and a point before and after the field that I want to extract. If I remove the points the rex command runs but without results.

Also remember that "block_num" is a field I already extracted.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Concatenation is done by the dot '.' (point) operator. The quotation marks close the string before the field and open the string after the field. The three pieces are then combined into a single string for rex to process.

I think I mis-read your question and gave you an incorrect answer. I've updated my answer to use block_num as the field name.

---
If this reply helps you, Karma would be appreciated.
0 Karma

edrivera3
Builder

ok. I'm going to give it a try.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...