How to avoid blowing the license at the first start.

[mataharry]

I have 5 GB /day license, and I am setting up plenty of new forwarders with new inputs. When I will turn them on, I expect about 50 GB of historical logs to be collected.

I have windows forwarders with WinEventLogs, I have linux /var/log/ folders etc...
Some of them goes several month in the past.

How to avoid getting license violation ?

[yannK · Accepted solution]

You have 2 approaches for this problem :

1) index all quickly and run

You can turn on all your inputs at once, and index all the historical logs, get a warning for the first day
(it can be more than 2 day or more if the volume is too important for the indexing speed, or if new inputs/forwarders are added over several days)

If you do not reach 5 days of warnings (3 for a free license) you are not in violation and the warning counter will reset after 30 days without new warning. This is usually a good method to index all your historical data at once. And If you reach a violation, you can always get a reset key from splunk support if you have a support contract.

see http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Aboutlicenseviolations

PS : if you are using forwarders, you may want to remove the default thruput limit of 256KBps to fasten the first monitoring.
see http://docs.splunk.com/Documentation/Splunk/5.0.1/Admin/Limitsconf

2) exclude the old events from the initial indexing

You can limit the volume of the historical logs (by example if you want to save space on the indexers or if you do not need the old events)

• With a monitor on files, in inputs.conf you can add a setting named ignoreOlderThan = 7d that will disregard logs files based on their mod time. (by example older than 7 days )
  see http://docs.splunk.com/Documentation/Splunk/5.0.1/admin/Inputsconf

• With the windows events logs you can exclude the old events with the option current_only=1.
  see http://docs.splunk.com/Documentation/Splunk/5.0.1/admin/Inputsconf

• monitor only the last versions of the files, and force a log rotation to roll the old events. (configure your monitor with specific path or blacklists)