Getting Data In

Parsing of log using UF

mukhan1
Explorer

Hello Team,

I was trying to parse my data by updating props.conf file i have created this file in (C:\Program Files\SplunkUniversalForwarder\etc\system\local\props.conf) 

[t24protocollog]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=^@ID[\s\S]*?REMARK.*$
NO_BINARY_CHECK=true
disabled=false

by using above configuration file. 
When I'm using the regex it separate each line as a event instead of defining starting and endpoint of event. I checked this regex on regex 101 it is giving me proper result on regex101.com but not on Splunk. Attaching the screenshot how my logs are showing on SPLUNK GUI.

Below is the content of log file.
--------------------------------------

LIST F.PROTOCOL @ID PROTOCOL.ID PROCESS.DATE TIME.MSECS K.USER APPLICATION LEVEL.FUNCTION ID REMARK PAGE 1 11:34:02 23 NOV 2023

@ID............ 202403.16
@ID............ 202403.16
PROTOCOL.ID.... 202403.16
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:23:649
K.USER......... INPUTTER
APPLICATION.... DC.ENTRY
LEVEL.FUNCTION. 1
ID.............
REMARK......... QUIRY - AC.REPORT

@ID............ 202303.16
@ID............ 202303.16
PROTOCOL.ID.... 202303.16
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:23:649
K.USER......... INPUTTER
APPLICATION.... AC.ENTRY
LEVEL.FUNCTION. 1
ID.............
REMARK......... ENQUIRY - AC.REPORT

Kindly do let me know if im doing anything wrong, need to parse the log file.

Labels (1)
0 Karma

mukhan1
Explorer

Hello @gcusello ,

Thanks for your response, however I manually uploaded my sample logs the problem is when I'm providing the regex for event breaker it shows me result on each line making every line as event. I've uploaded the SS how it is showing me data you can see the attached file. 

Also you can take below sample logs 

LIST F.PROTOCOL @ID PROTOCOL.ID PROCESS.DATE TIME.MSECS K.USER APPLICATION LEVEL.FUNCTION ID REMARK PAGE 1 11:34:02 23 NOV 2023

@ID............ 202403.16
@ID............ 202403.16
PROTOCOL.ID.... 202403.16
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:23:649
K.USER......... INPUTTER
APPLICATION.... DC.ENTRY
LEVEL.FUNCTION. 1
ID.............
REMARK......... QUIRY - AC.REPORT

@ID............ 202303.16
@ID............ 202303.16
PROTOCOL.ID.... 202303.16
PROCESS.DATE... 20230926
TIME.MSECS..... 11:15:23:649
K.USER......... INPUTTER
APPLICATION.... AC.ENTRY
LEVEL.FUNCTION. 1
ID.............
REMARK......... ENQUIRY - AC.REPORT

by using this regex "^@ID[\s\S]*?REMARK.*$" i'm getting correct data on regex101. 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

at first parsing is done on Indexers or (when present) on Heavy Forwarders, never on Universal Forwarders.

the only exception are INDEXED_EXTRACTIONS (as e.g. csv) where parsing starts on UF.

Anyway, see if it already exists an add-on for the technology you want to parse.

Then my hint is to take a sample of your logs and manually uoload it in your Splunk (using the Add Data feature) so you can use the guided sourcetype creation.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...