We have logs , where first few lines start with "#" and we don't need to ingest these lines.
We tired to use different methods , that didn't work. Appreciated the help/ideas from splunkers:
1st idea: use PREAMBLE_REGEX = ^#.* in props.conf on Heavy Forwarders where data are being parsed
2nd idea : use TRANSFORMS-null = setnull in props.conf and transforms.conf
on Heavy Forwarders where data are being parsed
transforms.conf:
[setnull]
REGEX = ^#.*
DEST_KEY = queue
FORMAT = nullQueue
example of log:
#-----------------------------------------
#DATE CREATED: 11/02/2021@04:16
#SUBJECT: REPORT ON THE GENERAL STATUS OF AUTOSYS JOBS
#ENVIRONMENT: CBA
#-----------------------------------------
11/02/2021@04:16,CBA,OTHER,CBA_CLIENT_REPORT_BOX,OI
11/02/2021@04:16,CBA,OTHER,CBA_copy_file_job,OI
11/02/2021@04:16,CBA,OTHER,CBA_ABC_SCHEDULER_BOX,OI
11/02/2021@04:16,CBA,OTHER,CBA_ABC_REPORT_BOX,OI
Hi @mlevsh
Try this for preamble_regex. It works.
PREAMBLE_REGEX=#
If this helps, give thumbs-up 🙂
Happy Splunking!!
Hash is a character used for comments both in your logs and splunk config files. You might try escaping it.
Hi PickleRick,
Thank you for suggestion. Unfortunately , it didn't work