Splunk Search

universal rex for extract intersting fields like (url, uri, user, email, ip, etc)

indeed_2000
Motivator

Hi

is there any universal or general rex to extract every known intersting fields like  (url, uri, user, email, ip, etc) from logs?

Thanks

Labels (4)
0 Karma

PickleRick
Ultra Champion

Hey, if you know a way to universally extract fields from any log, give us a hint.

But seriously - there are so many log types out there that there's no possibility to parse them all the same way.

How would you want to parse let's say CSV files, json structures, CEF events and multiline events from a tk102 gps logger?

It's like asking if you know a universal way to talk to anyone, regardless of what language he speaks.

That's simply not possible.

0 Karma

indeed_2000
Motivator

simple log file (one event per line no json no csv just simple and unistructural one)

Here is the pattern:

TIMESTAMP    LOGLEVEL     SOMEMESSAGE

 

need to whereever find (url, uri, user, email, ip, etc) extract them.

Thanks,

0 Karma

gcusello
Esteemed Legend

Hi @indeed_2000,

I need to have some sample (three or four events) of your logs to create a regex for field extraction.

Ciao.

Giuseppe

0 Karma

PickleRick
Ultra Champion

If you have chunks of data that are easily distinguishable, you can make regex-based extractions that will find them wherever in the event they are. Mind you however, that not many types of data chunks like that can be reliably specified (so that they cover all possiblities and do not capture false positives).

 

0 Karma

gcusello
Esteemed Legend

Hi @indeed_2000,

Splunk by itself recognize the pairs field=value.

Then, if you installe some Add-On for the technologies you're monitoring (e.g. Splunk_TA_Windows for Windows logs), you already have many field extractions just ready for you.

At the end, you have the possibility to create your own field extractions using regexes, but they are custom and specified for the logs you're analyzing.

In addition there's the regex101.com site that's very helpful for your regexes extractions.

If you need an help for your logs, please share them and surely you'll have a very quick help.

Bye.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...