topic Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file? in Getting Data In

Is it possible to add a crcSalt value that is not a static string or the source information of the file?

JWBailey — Mon, 22 Dec 2014 20:16:18 GMT

Is it possible to add a crcSalt value that is not a static string or the source information of the file?

I have a log that is overwritten each day that is often identical to the previous days and therefore is sometimes not indexed. crcSalt= doesn't work because the file is written in the same location each day. A date / time stamp is what i am looking for.

Thanks.

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

martin_mueller — Mon, 22 Dec 2014 21:23:52 GMT

It'd be best if your log had a timestamp somewhere near the start of the file.

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

JWBailey — Mon, 22 Dec 2014 21:34:29 GMT

I cannot rely on what is inside the log file because it is often identical to the file it is replacing.

Is SOURCE the only dynamic input the crcSalt setting will accept?

Is there another way altogether to force splunk to index a new file no matter what splunk thinks it should do?

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

martin_mueller — Mon, 22 Dec 2014 21:37:24 GMT

Philosophical question: If you overwrite a logfile with identical content, did you really overwrite it? 😄
More serious question: Why do you need data you already have re-indexed?

You could have a scripted input call a oneshot once per day, Splunk should read that no matter what.

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

DavidHourani — Fri, 17 Jun 2016 15:24:19 GMT

Did you find an answer to that ?

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

JWBailey — Fri, 17 Jun 2016 16:06:38 GMT

I did not find a direct solution to my problem. Instead I redesigned my application to include the date and timestamp in the file name, this ensured that the filename was always different and in turn guaranteed that Splunk would read the file using crcSalt=SOURCE.

Hope this helps...

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

DavidHourani — Fri, 17 Jun 2016 16:34:00 GMT

so you didn't find a crcsalr = or something ? 😞

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

somesoni2 — Fri, 17 Jun 2016 18:01:15 GMT

If your log files are getting overwritten every day OR every time there is an update and will get created if file is not there, then you can use [batch:// input instead of [monitor://, along with move_policy=sink_hole. The batch input with move_policy will delete the files and will not keep track of indexed files.

Re: Is it possible to add a crcSalt value that is not a static string or the source information of the file?

DavidHourani — Fri, 17 Jun 2016 23:12:32 GMT

smart ! thank you, ill try this out!