Help with extracting specific symbols/text from raw log

[DanAlexander]

Hi All,

Can anyone help me create a regex to extract the bolded parts from the following _raw log, please?

some text - [action:"Accept"; some text ; origin:"10.111.10.111"; some text]"; dst:"192.168.11.01"; some text684"; layer_name:"Some text"; layer_nsome text"; src:"192.168.81.62"]

Thank you in advance!

[yeahnah]

Hi @DanAlexander 

I'd suggest using the extract command instead on rex as there is a regular pattern for the key value pairs.  Here's a run anywhere example...

index=dummy
| append [| makeresults | eval _raw="some text - [action:\"Accept\"; some text ; origin:\"10.111.10.111\"; some text]; dst:\"192.168.11.01\"; some text684\"; layer_name:\"Some text\"; layer_nsome text\"; src:\"192.168.81.62\"]" ]
| extract pairdelim="; ]" kvdelim=":"
| table _raw action origin dst src

Hope that helps

[yeahnah]

I see I missed layer_name, which extract fails to pull, probably due to the whitespace.  Here's a combination of extract and rex to get the desired results...

index=dummy
| append [| makeresults | eval _raw="some text - [action:\"Accept\"; some text ; origin:\"10.111.10.111\"; some text]; dst:\"192.168.11.01\"; some text684\"; layer_name:\"Some text\"; layer_nsome text\"; src:\"192.168.81.62\"]" ]
| extract pairdelim="; ]" kvdelim=":"
| rex "layer_name:\"(?<layer_name>[^\"]+)"
| table _raw action origin dst layer_name src

[DanAlexander]

@yeahnah thank you very much for trying to help. Your solution for sure works, but I am not advanced to implement it.

Your help much appreciated!

[enzomialich · Accepted solution]

Hey @DanAlexander , try this:

| rex field=_raw "action:"(?<Action>.*)";"

| rex field=_raw "origin:"(?<Origin>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})""

| rex field=_raw "dst:"(?<Dest>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})""

| rex field=_raw "layer_name:"(?<Text>\w+)"" ***i'm assuming some text doesnt contain whitespaces, if does, I would try to test with [\s\S]+)";<add data pattern here so it limits the expression

| rex field=_raw "src:"(?<Source>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})""

I tested on regxr and seems to be working and extracting the fields as it should.

input: action:"(?<Action>\w+)"|origin:"(?<Origin>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"|layer_name:"(?<Text>\w+)"|dst:"(?<dest>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"|src:"(?<Source>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"

test: (your log example)

notes:

*i can't remember if we are able to "bundle" all of these rex extractions into a single command, try to separate quoted expressions using a comma

*for text data, the best way would be using a specific number of characters such as \w{n,m} (a number of characters that could be between n and m. 

[DanAlexander]

@enzomialich thanks for the quick response. Yes, it all works. thank you very much!

I needed a single reg expression to extract the parts of the raw log as I want to discard all the rest

Kudos to you Sir!