topic Re: How to write the regular expression to extract these fields from my sample data? in Splunk Search

How to write the regular expression to extract these fields from my sample data?

kschmeling — Tue, 13 Dec 2016 12:37:44 GMT

Hello,

I'm trying to create a regex to extract the fields to the follow logs:

Example 1

msg=O equipamento marte (192.168.0.1) recuperou a conectividade SSH suid=7 sname=Script Assíncrono: 10 suser= spid=17025 dst=192.168.0.2 dhost=marte dport=22

Example 2

msg=Sessao iniciada para marte (192.168.0.1) - Usuario Privilegiado TPL - root pelo usuario Usuario Teste (usuario.teste) suid=297 sname=Usuario Teste suser=usuario.teste spid=14410 dst=192.168.0.1 dpt=22 duser=root

I need to extract the fields like that:

Example 1

msg=O equipamento marte (192.168.0.1) recuperou a conectividade SSH
suid=7
sname=Script Assíncrono: 10
suser=
spid=17025
dst=192.168.0.2
dhost=marte
dport=22

Example 2

msg=Sessao iniciada para marte (192.168.0.1) - Usuario Privilegiado TPL - root pelo usuario Usuario Teste (usuario.teste)
suid=297
sname=Usuario Teste
suser=usuario.teste
spid=14410
dst=192.168.0.1
dpt=22
duser=root

I'm using https://regex101.com/ to help me

Any help is appreciated

Re: How to write the regular expression to extract these fields from my sample data?

Richfez — Tue, 13 Dec 2016 13:27:55 GMT

I am surprised this isn't happening automatically. Under normal circustances, Splunk would find any "key=value" pairs in inputs.

Have you repeated your search in Verbose mode to make sure you can see the fields it's already found?

Re: How to write the regular expression to extract these fields from my sample data?

richgalloway — Tue, 13 Dec 2016 13:47:27 GMT

I believe the default parser uses space as a delimiter so he'd end up with 'msg=O' and 'msg=Sessao'.

Re: How to write the regular expression to extract these fields from my sample data?

richgalloway — Tue, 13 Dec 2016 13:57:15 GMT

The examples are little different, which makes finding a common regex string tricky, but I believe this will work.

"msg=(?<msg>.*?) suid=(?<suid>\d+) sname=(?<sname>.*?) suser=(?<suser>.*) spid=(?<spid>\d+) dst=(?<dst>[^ ]+) (dhost=(?<dhost>.*?) dport=(?<dport>\d+))?(dpt=(?<dpt>\d+) duser=(?<duser>.*))?"

Re: How to write the regular expression to extract these fields from my sample data?

kschmeling — Tue, 13 Dec 2016 15:15:56 GMT

Yes, Splunk give me this results:

msg=O
msh=Sessao
sname=Script
sname=Usuario

Re: How to write the regular expression to extract these fields from my sample data?

Richfez — Tue, 13 Dec 2016 15:17:46 GMT

Yes, I realized that would have been the case after I had another cup of coffee and the wonderfully talented richgalloway reminded me of same. 🙂

There should be an answer below - give it a try and if it works for you, please accept it!

Happy Splunking!

-Rich

Re: How to write the regular expression to extract these fields from my sample data?

kschmeling — Tue, 13 Dec 2016 15:37:14 GMT

Perfect, it's working. The trick is this expression:

(dhost=(?<dhost>.*?) dport=(?<dport>\d+))?(dpt=(?<dpt>\d+) duser=(?<duser>.*))?

Thanks a lot!

Re: How to write the regular expression to extract these fields from my sample data?

rahulrawlani — Wed, 14 Dec 2016 14:19:58 GMT

Hello Friend,

If you want to extract the fields for both the events with one regex, then try using the below regex

.*msg\=(?.*)?suid\=(?\d+)?\ssname\=(?.*)?suser\=(?.*)?\sspid\=(?\d+)\sdst\=(?\d+\.\d+\.\d+\.\d+)\s(?:dhost\=(?\w+)\sdport\=(?\d+)|(?:dpt\=(?\d+)\sduser\=(?\w+)))

Let me know if that works for you.