Getting Data In

Importing CSV

edgarsilva01
Path Finder

Hi everyone,

I am ingesting csv files that contain information about views of certain web pages,
These files are updated once a day.

I need that when the file is updated take only the new events.

Example: If the first file ends on day 20, for the second time it is updated I am no longer interested in seeing old events
I just want splunk to take the values ​​from day 21.

Will they have any idea how I can do it?

Thank you

Tags (1)
0 Karma

codebuilder
Influencer

Configure your forwarder to use "batch" instead of "monitor", and set "move_policy =sinkhole".

This will cause the forwarder to delete the file after it's been ingested. Once your process creates a new file, the forwarder will pick it up and you'll get only the new results.

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in...

Ignore the "Upload a File" portion in the documentation. Not sure why that is there, it's misleading.

----
An upvote would be appreciated and Accept Solution if it helps!
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Is the entire file re-written every day or are the latest results appended to the file?
What are the inputs.conf settings for the file?

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...