Getting Data In

Importing CSV

edgarsilva01
Path Finder

Hi everyone,

I am ingesting csv files that contain information about views of certain web pages,
These files are updated once a day.

I need that when the file is updated take only the new events.

Example: If the first file ends on day 20, for the second time it is updated I am no longer interested in seeing old events
I just want splunk to take the values ​​from day 21.

Will they have any idea how I can do it?

Thank you

Tags (1)
0 Karma

codebuilder
Influencer

Configure your forwarder to use "batch" instead of "monitor", and set "move_policy =sinkhole".

This will cause the forwarder to delete the file after it's been ingested. Once your process creates a new file, the forwarder will pick it up and you'll get only the new results.

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in...

Ignore the "Upload a File" portion in the documentation. Not sure why that is there, it's misleading.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is the entire file re-written every day or are the latest results appended to the file?
What are the inputs.conf settings for the file?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...