How do I apply regex to filter the events in the logs?

[deepthi5]

Hi ,

I have logs with below format 

X.X.X.X. - - [02/Aug/2022:10:31:18 +0200] "GET /api/mc/v0.1/agendas/view/background-tasks?is-details-required=false HTTP/1.1" 200 20 "-" " "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X.X - - [02/Aug/2022:10:31:18 +0200] "GET /api/mc/v0.1/agendas/view/background-tasks?is-details-required=false HTTP/1.1" 200 20 "-" " "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X.- - [02/Aug/2022:10:31:33 +0200] "GET /api/mt/v0.1/tasks/view-count HTTP/1.1" 200 371 "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

X.X.X.X. - - [02/Aug/2022:10:31:33 +0200] "GET /api/mt/v0.1/work-items?start-position=0&number-of-items=11 HTTP/1.1" 200 3084  "https://XXX.AAA.COM" "Mozilla/5.0 (Windows NT X.O; Win64; x64; rv:98.0)  Firefox/98.0"

out of these logs i want to get only  events which has /api/mt in it and drop the remaining events 

My configurations: 

 [monitor:///aaa/yyy/xxxx/access_log]

disabled = false

sourcetype = mytask:access_log

index = temp

props.conf

[mytask:access_log]
TRANSFORMS-set = setnull
TRANSFORMS-set = setparsing 

Transforms.conf

 [setnull]
REGEX = ^(.*)mc(.*)

DEST_KEY = queue
FORMAT = nullQueue

 [setparsing]
REGEX = ^(.*)mt(.*)
DEST_KEY = queue
FORMAT = indexQueue

Do we need set anything else in the configs 

[gcusello]

Hi @deepthi5,

your configurations are correct, remember only that inputs.conf must be located on the Universal Forwarders and props.conf and transforms.conf must be located on Indexers or (if present) on Heavy Forwarders.

Ciao.

Giuseppe