topic Re: How to drop simular logs in Splunk Enterprise

How to drop simular logs

BoldKnowsNothin — Thu, 12 Oct 2023 04:55:15 GMT

Hello comrades!

I just wonder, does splunk detects logs similarity by it's pattern?

Many thanks.

Re: How to drop simular logs

inventsekar — Thu, 12 Oct 2023 07:01:03 GMT

Hi @BoldKnowsNothin .. Yes, Splunk detects the "pattern" in the log lines.

may we know more details from you.. from the subject.. it looks like you want to drop (dont want to ingest) a portion of the logs.. if its correct, pls provide us more details..

1) the sample logs..

2) do you have HF or no HF

3) the props.conf / transforms may be needed later. thanks.

Re: How to drop simular logs

BoldKnowsNothin — Thu, 12 Oct 2023 09:18:17 GMT

Hello inventsekar,

So let me try to explain this. We started to monitor our docker/containers on splunk. Unfortunately we cannot install splunk forwarders on container. So we using fluent bit to collect all logs from container and grabing logs via httpeventcollector.

So we cannot use splunk on this scenario.

When containers crushing they spamming exactly same logs(only timestamp different) like there is no tomorrow.