Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- static csv file (uploaded only once in a day) issu...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a csv file kept in a central path which is only uploaded once in a day.
The moment i search the data on my search head, the data is not completely found. As in, if I choose the exact path of the source and select All Time in the time range picker, only then i am able to see the exact count from the csv. Any idea why am I not able see the entire data if I keep the "source=* and time range=Today". time range must show all data when selected for "Today" because I upload the csv only once in a day.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
... that would be true if you delete the csv file from splunk before you upload it. Do you?
If not, then splunk is probably only setting new timestamps on any "new" or "changed" records that it uploads, or records after the first new/changed one.
There's a fairly complete description of how timestamps work in this post -
https://answers.splunk.com/answers/148926/wrong-timestamp-of-csv.html
But the other part, not talked about there, is the old thing about fishbuckets and splunk already knowing what's in that file and where it left off.
If you set up the file with some kind of junk comment header record at the top, which changes daily, then when splunk ingests the file, it should mark all the records as new-ish, assuming I am interpreting the docs correctly.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
... that would be true if you delete the csv file from splunk before you upload it. Do you?
If not, then splunk is probably only setting new timestamps on any "new" or "changed" records that it uploads, or records after the first new/changed one.
There's a fairly complete description of how timestamps work in this post -
https://answers.splunk.com/answers/148926/wrong-timestamp-of-csv.html
But the other part, not talked about there, is the old thing about fishbuckets and splunk already knowing what's in that file and where it left off.
If you set up the file with some kind of junk comment header record at the top, which changes daily, then when splunk ingests the file, it should mark all the records as new-ish, assuming I am interpreting the docs correctly.
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
Automating Threat Operations and Threat Hunting with Recorded Future
Keep the Learning Going with the New Best of .conf Hub
Splunk Community Badges!
