How to Index Rolling Log Files

[_gkollias]

I've come across a use case where a log file roles once every 24h.

e.g: logname.log.2014-01-01
logname.log.2014-01-02
logname.log.2014-01-03 ...etc

I've edited the inputs.conf to monitor ...///logpath/LogName.log*], however I am unable to see any of the log data.

Is there something I need to add in order to start seeing this data (i.e. regex via props.conf)?

Your help would be much appreciated.

Thank You!

[Drainy]

Bonjour!

Have you already indexed these logs previously and not realised? Generally logs will be filled with all sorts of wonderful data throughout the day which Splunk gobbles up, then at midnight a script rolls the logs and starts afresh.
Splunk recognises this by taking a CRC of the start of each file, this way it stops it re-indexing old data and producing duplicates;
http://docs.splunk.com/Documentation/Splunk/6.0.1/Data/Howlogfilerotationishandled

[kristian_kolb]

And you are sure that your [monitor] stanza is correct? If you are not seeing any log files, that might be the issue.

[Drainy]

ok so time configuration in props is about how it pulls the timestamp from an event. Its not related to how Splunk handles rolled logs, thats done by the CRC of the first 256 bytes of a file (or something like that. Try crcSalt = in your inputs definition for that file, it will force it to use the filename as the crcsalt which will always be unique as long as its a unique filename

[_gkollias]

I believe timestamps in correlation with rolling based logs is the problem, right?

I don't think Splunk will index rolling based logs without specifying its timestamps, but thats essentially what I'm asking.

I need a way for Splunk to receive rolling based log data, and I'm pretty sure I need to specify that somehow, but I'm not exactly sure except that CRC attributes look like its more for headers within the log, and not being able to specify the log files timestamps.

Sorry for any confusion 🙂

[Drainy]

hmm, I'm not sure if you're confusing problems here? Setting the timestamp stuff may be useful if you're having issues with timestamps. But is that the problem? I thought the issue is indexing/reindexing files? You don't need to configure a CRC value as there is a default, if you want it to read every file you can set the crcsalt to to use the filename as the CRC but you would need to clear your data everywhere otherwise you'll end up with duplicates

[_gkollias]

It seems like the CRC attributes would be good to use if a log had headers.

I have an easy name=value format that Splunk can auto-extract the data from..so what about using something like a props.conf and configuring time parameters (TIME_PREFIX, TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD) that may be useful in my case.

If I set up the props.conf, would I still need to set up a CRC attribute in the inputs.conf?

Thanks!

[Drainy]

Ah, once you've consumed a file Splunk won't re-read it, even if you add a new output. On the forwarder you'll need to run a command, splunk clean eventdata. This will delete all its internal "trackers" of what it has and has't read. This will cause it to re-send everything. So if you do this remember to run the same command on your indexers so you don't end up with duplicate data... its a bit of a clean everything and start again approach. Also bear in mind that on a UF that command will reset the admin password to default, thats not the case with an indexer

[_gkollias]

Thanks for the quick response!

I've just recently installed a forwarder and configured this server to start sending its data to Splunk. I've gotten the dev server to index its data, but prod seems to be a different story.

I'll take a look at this documentation,
Thanks