https://community.splunk.com/t5/Getting-Data-In/Is-there-any-way-to-NOT-duplicate-the-data-when-adding-CSV-file/m-p/317747#M59401 <P>Hello,</P> <P>Are there any way? In my dashboard the data comes from a CSV file and in order for the dashboard to update, I need to upload the fresh CSV file via Sharepoint, the problem is, when I upload the data, it is duplicated and I don't want to manually filter the old data through Excel and I don't want it duplicated. Any help will be appreciated</P> <P>Regards,<BR /> Jannsen</P> Thu, 19 Oct 2017 05:15:56 GMT jannsenagustin 2017-10-19T05:15:56Z https://community.splunk.com/t5/Getting-Data-In/Is-there-any-way-to-NOT-duplicate-the-data-when-adding-CSV-file/m-p/317747#M59401 <P>Hello,</P> <P>Are there any way? In my dashboard the data comes from a CSV file and in order for the dashboard to update, I need to upload the fresh CSV file via Sharepoint, the problem is, when I upload the data, it is duplicated and I don't want to manually filter the old data through Excel and I don't want it duplicated. Any help will be appreciated</P> <P>Regards,<BR /> Jannsen</P> Thu, 19 Oct 2017 05:15:56 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-any-way-to-NOT-duplicate-the-data-when-adding-CSV-file/m-p/317747#M59401 jannsenagustin 2017-10-19T05:15:56Z https://community.splunk.com/t5/Getting-Data-In/Is-there-any-way-to-NOT-duplicate-the-data-when-adding-CSV-file/m-p/317748#M59402 <P>If it is unique in _raw</P> <P>Search without searching for duplicates.<BR /> (your searcvh)|sort _raw| dedup _raw|(your searcvh)</P> <P>Periodically delete duplicates.<BR /> (your searcvh)|sort _raw|streamstats count by _raw|table count _raw|where count&gt;1|delete</P> Thu, 19 Oct 2017 06:23:23 GMT https://community.splunk.com/t5/Getting-Data-In/Is-there-any-way-to-NOT-duplicate-the-data-when-adding-CSV-file/m-p/317748#M59402 HiroshiSatoh 2017-10-19T06:23:23Z