I have file called console.log. When its size reaches to 512MB, another file gets created with the name consoleserver01.log. When this reaches to 512 MB, another file is created with name consoleserver02.log and so on.
I would like to configure inputs.conf for all source files like
console* so that Splunk automatically fetches data from the latest file.
How to do that?
If you have control over the process thats writing the logs I would recommend that you have the most recent file be called console.log, and call the rolled files console.log.1 etc. Have a look at the logrotate command.
Otherwise, set up your inputs.conf stanza like so:
This will ensure that all the console.logs are monitored. Note that if you have a header in the log file you may run into problems with the crc checking - ie Splunk won't know that the files are different. If that happens, and you will never reuse the filenames you could set
All you need to do is monitor the directory those files are located in and whitelist the files
You can do all this through the SplunkWeb GUI or through conf files. Here's an example of what your inputs.conf should look like:
[monitor:///your_directory] host = hostname sourcetype = your_sourcetype index = your_index whitelist = console.*\.log
Heres the latest inputs.conf doc: