topic Re: Duplicate Data In Sourcetype in Getting Data In

Duplicate Data In Sourcetype

jbender72 — Tue, 12 Jan 2021 16:38:25 GMT

Hello,

I have two Domain Controllers that are producing a lot of data, pushing my daily usage over the limit. I saw that sourcetype WinNetMon and Perfmon:process have the most duplicate logs in them. How do I reduce the duplication before the indexing occurs saving me on my daily usage?

Thank You

Re: Duplicate Data In Sourcetype

General_Talos — Tue, 12 Jan 2021 16:44:45 GMT

Effectively this is not possible to identify/remove duplicate data before indexing.

Since Perfmon:process is always noisy, would recommend to filter only relevant/required data.

Re: Duplicate Data In Sourcetype

jbender72 — Tue, 12 Jan 2021 17:00:27 GMT

ok, thank you