I think you'd need the application to be able to log to one of 2 separate log files, on demand.
Then, set up an alert (based on indexed volume for the day) to switch logging (via an alert-triggered script) to the log file that is not being indexed by Splunk.
At midnight daily, fire another alert that will switch logging back to the log file that is being indexed.
Some Java app servers can switch log4j logging levels on demand (via API) without a need to stop/start the app.
I do this for DEBUG-level logging to a logfile (6 of them) that is not being indexed, that rolls over every hour.
So I have the last 6 hours of logs, if I ever need to load any of them manually into Splunk for on-demand analysis of what just happened, without exhausting my daily indexing limit.
Now, I'm tempted to write/automate the script that will do the switch...
... View more
