Getting Data In

Is it possible to add a crcSalt value that is not a static string or the source information of the file?

Communicator

Is it possible to add a crcSalt value that is not a static string or the source information of the file?

I have a log that is overwritten each day that is often identical to the previous days and therefore is sometimes not indexed. crcSalt= doesn't work because the file is written in the same location each day. A date / time stamp is what i am looking for.

Thanks.

SplunkTrust
SplunkTrust

If your log files are getting overwritten every day OR every time there is an update and will get created if file is not there, then you can use [batch:// input instead of [monitor://, along with move_policy=sink_hole. The batch input with move_policy will delete the files and will not keep track of indexed files.

SplunkTrust
SplunkTrust

smart ! thank you, ill try this out!

0 Karma

SplunkTrust
SplunkTrust

Did you find an answer to that ?

0 Karma

Communicator

I did not find a direct solution to my problem. Instead I redesigned my application to include the date and timestamp in the file name, this ensured that the filename was always different and in turn guaranteed that Splunk would read the file using crcSalt=SOURCE.

Hope this helps...

0 Karma

SplunkTrust
SplunkTrust

so you didn't find a crcsalr = or something ? 😞

0 Karma

SplunkTrust
SplunkTrust

Philosophical question: If you overwrite a logfile with identical content, did you really overwrite it? 😄
More serious question: Why do you need data you already have re-indexed?

You could have a scripted input call a oneshot once per day, Splunk should read that no matter what.

0 Karma

SplunkTrust
SplunkTrust

It'd be best if your log had a timestamp somewhere near the start of the file.

0 Karma

Communicator

I cannot rely on what is inside the log file because it is often identical to the file it is replacing.

Is SOURCE the only dynamic input the crcSalt setting will accept?

Is there another way altogether to force splunk to index a new file no matter what splunk thinks it should do?

0 Karma