Getting Data In

How do I apply regex to filter the events in the logs?

deepthi5
Path Finder

Hi ,

I have logs with below format 

X.X.X.X. - - [02/Aug/2022:10:31:18 +0200] "GET /api/mc/v0.1/agendas/view/background-tasks?is-details-required=false HTTP/1.1" 200 20 "-" " "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X.X - - [02/Aug/2022:10:31:18 +0200] "GET /api/mc/v0.1/agendas/view/background-tasks?is-details-required=false HTTP/1.1" 200 20 "-" " "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X.- - [02/Aug/2022:10:31:33 +0200] "GET /api/mt/v0.1/tasks/view-count HTTP/1.1" 200 371 "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X. - - [02/Aug/2022:10:31:33 +0200] "GET /api/mt/v0.1/work-items?start-position=0&number-of-items=11 HTTP/1.1" 200 3084  "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

out of these logs i want to get only  events which has /api/mt in it and drop the remaining events 

My configurations: 

 [monitor:///aaa/yyy/xxxx/access_log]

disabled = false

sourcetype = mytask:access_log

index = temp

props.conf

[mytask:access_log]
TRANSFORMS-set = setnull
TRANSFORMS-set = setparsing 

 

Transforms.conf

 [setnull]
REGEX = 
^(.*)mc(.*)

DEST_KEY = queue
FORMAT = nullQueue

 [setparsing]
REGEX = ^(.*)mt(.*)
DEST_KEY = queue
FORMAT = indexQueue

Do we need set anything else in the configs 

Labels (2)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @deepthi5,

your configurations are correct, remember only that inputs.conf must be located on the Universal Forwarders and props.conf and transforms.conf must be located on Indexers or (if present) on Heavy Forwarders.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...