static csv file (uploaded only once in a day) issue

simon21 — Tue, 31 Jan 2017 12:22:00 GMT

I have a csv file kept in a central path which is only uploaded once in a day.
The moment i search the data on my search head, the data is not completely found. As in, if I choose the exact path of the source and select All Time in the time range picker, only then i am able to see the exact count from the csv. Any idea why am I not able see the entire data if I keep the "source=* and time range=Today". time range must show all data when selected for "Today" because I upload the csv only once in a day.

Re: static csv file (uploaded only once in a day) issue

DalJeanis — Wed, 01 Feb 2017 03:41:29 GMT

... that would be true if you delete the csv file from splunk before you upload it. Do you?

If not, then splunk is probably only setting new timestamps on any "new" or "changed" records that it uploads, or records after the first new/changed one.

There's a fairly complete description of how timestamps work in this post -
https://answers.splunk.com/answers/148926/wrong-timestamp-of-csv.html

But the other part, not talked about there, is the old thing about fishbuckets and splunk already knowing what's in that file and where it left off.

If you set up the file with some kind of junk comment header record at the top, which changes daily, then when splunk ingests the file, it should mark all the records as new-ish, assuming I am interpreting the docs correctly.

topic static csv file (uploaded only once in a day) issue in Getting Data In

static csv file (uploaded only once in a day) issue

Re: static csv file (uploaded only once in a day) issue