Are you a member of the Splunk Community?
- Find Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- Re: static csv file (uploaded only once in a day) ...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a csv file kept in a central path which is only uploaded once in a day.
The moment i search the data on my search head, the data is not completely found. As in, if I choose the exact path of the source and select All Time in the time range picker, only then i am able to see the exact count from the csv. Any idea why am I not able see the entire data if I keep the "source=* and time range=Today". time range must show all data when selected for "Today" because I upload the csv only once in a day.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
... that would be true if you delete the csv file from splunk before you upload it. Do you?
If not, then splunk is probably only setting new timestamps on any "new" or "changed" records that it uploads, or records after the first new/changed one.
There's a fairly complete description of how timestamps work in this post -
https://answers.splunk.com/answers/148926/wrong-timestamp-of-csv.html
But the other part, not talked about there, is the old thing about fishbuckets and splunk already knowing what's in that file and where it left off.
If you set up the file with some kind of junk comment header record at the top, which changes daily, then when splunk ingests the file, it should mark all the records as new-ish, assuming I am interpreting the docs correctly.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
... that would be true if you delete the csv file from splunk before you upload it. Do you?
If not, then splunk is probably only setting new timestamps on any "new" or "changed" records that it uploads, or records after the first new/changed one.
There's a fairly complete description of how timestamps work in this post -
https://answers.splunk.com/answers/148926/wrong-timestamp-of-csv.html
But the other part, not talked about there, is the old thing about fishbuckets and splunk already knowing what's in that file and where it left off.
If you set up the file with some kind of junk comment header record at the top, which changes daily, then when splunk ingests the file, it should mark all the records as new-ish, assuming I am interpreting the docs correctly.
Thanks for the Memories! Splunk University, .conf25, and our Community
Data Persistence in the OpenTelemetry Collector
Introducing Splunk 10.0: Smarter, Faster, and More Powerful Than Ever
