Getting Data In

Is there any way to NOT duplicate the data when adding CSV file?

New Member


Are there any way? In my dashboard the data comes from a CSV file and in order for the dashboard to update, I need to upload the fresh CSV file via Sharepoint, the problem is, when I upload the data, it is duplicated and I don't want to manually filter the old data through Excel and I don't want it duplicated. Any help will be appreciated


0 Karma


If it is unique in _raw

Search without searching for duplicates.
(your searcvh)|sort _raw| dedup _raw|(your searcvh)

Periodically delete duplicates.
(your searcvh)|sort _raw|streamstats count by _raw|table count _raw|where count>1|delete

0 Karma
Get Updates on the Splunk Community!

Dashboard Studio Challenge - Learn New Tricks, Showcase Your Skills, and Win Prizes!

Reimagine what you can do with your dashboards. Dashboard Studio is Splunk’s newest dashboard builder to ...

Introducing Edge Processor: Next Gen Data Transformation

We get it - not only can it take a lot of time, money and resources to get data into Splunk, but it also takes ...

Take the 2021 Splunk Career Survey for $50 in Amazon Cash

Help us learn about how Splunk has impacted your career by taking the 2021 Splunk Career Survey. Last year’s ...