topic Re: Remove unwanted events from the data. in Getting Data In

Remove unwanted events from the data.

iamsplunker — Wed, 15 Nov 2023 21:16:41 GMT

Hi I wanted to remove unwanted events from my data, To ingest as cleanest as possible and for better line breaking etc.

These events are not like regular logs. Since these are generated by a script which runs everyday at 12 am and log file rotates and gets renamed as logfile.log.<date>

Here is the pattern that I wanted to remove using SEDCMD and apply it in props.conf

[09/13/2023 00:00:00] <Event was scheduled based on job definition.>

[10/12/2023 23:58:01] <Executing at CA_AGENT>

[11/12/2023 23:58:01] ----------------------------------------

Re: Remove unwanted events from the data.

inventsekar — Wed, 15 Nov 2023 23:35:18 GMT

Hi @iamsplunker ... pls suggest if you have HF or UF and indexer.

also pls suggest us if you have had a chance to look at this doc... pls copy paste your current props/transforms

https://docs.splunk.com/Documentation/Splunk/9.1.1/Data/Anonymizedata

Re: Remove unwanted events from the data.

iamsplunker — Thu, 16 Nov 2023 00:38:47 GMT

@inventsekar Thanks for your response .We have UF and indexer, I developed a regex to stripe out the unwanted events. It's working as expected.

SEDCMD-Remove1=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Event was scheduled based on job definition.>//
SEDCMD-Remove2=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Executing at CA_AGENT>//
SEDCMD-Remove3=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+-+//

Re: Remove unwanted events from the data.

inventsekar — Thu, 16 Nov 2023 00:41:01 GMT

>>> I developed a regex to stripe out the unwanted events. It's working as expected.

so, may i know if your problem is resolved ah..

or do you some issues still, pls suggest, thanks.

Karma points / upvotes are appreciated, thanks.

Re: Remove unwanted events from the data.

iamsplunker — Thu, 16 Nov 2023 00:44:23 GMT

@inventsekar It has been resolved- Thanks