Remove unwanted events from the data.

[iamsplunker]

Hi I wanted to remove unwanted events from my data, To ingest as cleanest as possible and for better line breaking etc.

These events are not like regular logs. Since these are generated by a script which runs everyday at 12 am and log file rotates and gets renamed as logfile.log.<date>

Here is the pattern that I wanted to remove using SEDCMD and apply it in props.conf

[09/13/2023 00:00:00]       <Event was scheduled based on job definition.>

[10/12/2023 23:58:01]       <Executing at CA_AGENT>

[11/12/2023 23:58:01]        ----------------------------------------

[inventsekar]

Hi @iamsplunker ... pls suggest if you have HF or UF and indexer.

also pls suggest us if you have had a chance to look at this doc... pls copy paste your current props/transforms

https://docs.splunk.com/Documentation/Splunk/9.1.1/Data/Anonymizedata

[iamsplunker · Accepted solution]

@inventsekar Thanks for your response .We have UF and indexer, I developed a regex to stripe out the unwanted events. It's working as expected.

SEDCMD-Remove1=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Event was scheduled based on job definition.>//
SEDCMD-Remove2=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Executing at CA_AGENT>//
SEDCMD-Remove3=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+-+//

[inventsekar]

>>> I developed a regex to stripe out the unwanted events. It's working as expected.

so, may i know if your problem is resolved ah.. 

or do you some issues still, pls suggest, thanks. 

Karma points / upvotes are appreciated, thanks. 

[iamsplunker]

@inventsekar It has been resolved- Thanks