Getting Data In

Remove unwanted events from the data.

iamsplunker
Communicator

Hi I wanted to remove unwanted events from my data, To ingest as cleanest as possible and for better line breaking etc.

These events are not like regular logs. Since these are generated by a script which runs everyday at 12 am and log file rotates and gets renamed as logfile.log.<date>

Here is the pattern that I wanted to remove using SEDCMD and apply it in props.conf

[09/13/2023 00:00:00]       <Event was scheduled based on job definition.>

[10/12/2023 23:58:01]       <Executing at CA_AGENT>

[11/12/2023 23:58:01]        ----------------------------------------

Labels (4)
Tags (3)
0 Karma
1 Solution

iamsplunker
Communicator

@inventsekar Thanks for your response .We have UF and indexer, I developed a regex to stripe out the unwanted events. It's working as expected.

SEDCMD-Remove1=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Event was scheduled based on job definition.>//
SEDCMD-Remove2=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Executing at CA_AGENT>//
SEDCMD-Remove3=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+-+//

View solution in original post

0 Karma

inventsekar
SplunkTrust
SplunkTrust

Hi @iamsplunker ... pls suggest if you have HF or UF and indexer.

also pls suggest us if you have had a chance to look at this doc... pls copy paste your current props/transforms

https://docs.splunk.com/Documentation/Splunk/9.1.1/Data/Anonymizedata

 

0 Karma

iamsplunker
Communicator

@inventsekar Thanks for your response .We have UF and indexer, I developed a regex to stripe out the unwanted events. It's working as expected.

SEDCMD-Remove1=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Event was scheduled based on job definition.>//
SEDCMD-Remove2=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+<Executing at CA_AGENT>//
SEDCMD-Remove3=s/^\[\d{2}\/\d{2}\/\d{4}\s\d{2}\:\d{2}:\d{2}]\s+-+//

0 Karma

inventsekar
SplunkTrust
SplunkTrust

>>> I developed a regex to stripe out the unwanted events. It's working as expected.

so, may i know if your problem is resolved ah.. 

or do you some issues still, pls suggest, thanks. 

 

Karma points / upvotes are appreciated, thanks. 

0 Karma

iamsplunker
Communicator

@inventsekar It has been resolved- Thanks

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...