topic Re: How to index duplicate files which has different name in Getting Data In

How to index duplicate files which has different name

snehalk — Mon, 19 Sep 2016 12:03:26 GMT

Hello All,

Is there any application or method in Splunk, where we can index the files(which has same contain) in Splunk but with different file name?

Or Can we identify the duplicate files in Splunk without indexing?

Note: one option is there where we can delete the fishbucket and reindex, but i dont want to go with this option.

Can any one help me on this.

Thanks in Advance.

gcusello — Mon, 19 Sep 2016 12:13:13 GMT

insert in your inputs.conf stanza the row

crcSalt = <SOURCE>

Bye.
Giuseppe

snehalk — Mon, 19 Sep 2016 12:30:14 GMT

Hello Cusello,

Thank you for quick response, am using below config as suggested by you, and its indexing duplicate files in splunk.

inputs.conf
[monitor://C:\sampleduplicate\duplicatefiles\*]
index=main
sourcetype=vendorduplicate
crcSalt = <SOURCE>

Thank you once again !!

Samir__ — Mon, 11 Dec 2017 11:16:04 GMT

Follow up question on the same topic. I also have files with different name but most of them have exact same contents.

After adding crcSalt = < SOURCE> in inputs.conf, does splunk automatically index previously excluded "duplicates"?