Getting Data In

Importing CSV

edgarsilva01
Path Finder

Hi everyone,

I am ingesting csv files that contain information about views of certain web pages,
These files are updated once a day.

I need that when the file is updated take only the new events.

Example: If the first file ends on day 20, for the second time it is updated I am no longer interested in seeing old events
I just want splunk to take the values ​​from day 21.

Will they have any idea how I can do it?

Thank you

Tags (1)
0 Karma

codebuilder
Influencer

Configure your forwarder to use "batch" instead of "monitor", and set "move_policy =sinkhole".

This will cause the forwarder to delete the file after it's been ingested. Once your process creates a new file, the forwarder will pick it up and you'll get only the new results.

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in...

Ignore the "Upload a File" portion in the documentation. Not sure why that is there, it's misleading.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is the entire file re-written every day or are the latest results appended to the file?
What are the inputs.conf settings for the file?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...