Installation

Indexing a "Backlog", or "Old" data, without breaching license

jamesmcgee
Explorer

Is there any recommended mechanism to bring a new server, or new "source" online, where there may be historic data, thus avoiding a temporary "breach" of licensing?

i.e., I bring in, a 2 year old server, with data that's never been indexed, and say it's been logging 50MB per day.

50MB x 365Days x 2Years = 36500MB or 36GB.

Going forwards, it would only ever log 50MB per day, so shouldn't be an issue, but indexing that initial backlog would put me well over, albeit temporary.

Likewise, if a "Noisy" forwarder was offline, or down for a while, is there any way to bring that back online without going over?

Advice/guidance appreciated....

mcbradford
Contributor

Download splunk (free) to another machine - index the data, move the buckets to the production.

mikelanghorst
Motivator

You can also limit the amount of data the forwarder can send in limits.conf
http://docs.splunk.com/Documentation/Splunk/latest/Admin/Limitsconf

`[thruput]

maxKBps =
* If specified and not zero, this limits the speed through the thruput processor to the specified
rate in kilobytes per second.
* To control the CPU load while indexing, use this to throttle the number of events this indexer
processes to the rate (in KBps) you specify. `

While it specifically mentions CPU load, I've seen several people using this to limit data for just this purpose.

David
Splunk Employee
Splunk Employee

Basically, just bring it back online, exceed your license for the day, and move on. You can exceed a free license 3 times in 30 days without losing your ability to search, and you can do the same 5 times on an enterprise license. So the only real recommendation is to start the process at 12:01 AM server time, to make the best use of your license exceptions (or at least, don't start it at 11 PM).

Check this out for more: http://splunk-base.splunk.com/answers/322/what-happens-when-i-exceed-my-licensed-limit

Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...