All Apps and Add-ons

Can you help me calculate my Splunk license usage?

prathapkcsc
Explorer

Hi one and all,

I need to calculate Splunk license usage for the last 30 days as I am going for a Splunk license renewal. I found two queries for the above and both were giving different results. The below are the two queries

  1. index="_internal" source="*.log" per_index_thruput earliest=-90d@d | eval GB=kb/(1024*1024) | timechart span=1d sum(GB) | convert ctime(_time) as timestamp

  2. index=_internal source=*license_usage.log type=RolloverSummary earliest=-30d@d latest=now
    | eval GB = round(b/1024/1024/1024,2)
    | eval _time = _time - 43200
    | timechart span=1d sum(GB) AS "Total GB used

Query- 1 showing almost double usage to query-2.

Can some one please guide me which one i need to follow??

Thanks

0 Karma
1 Solution

FrankVl
Ultra Champion

Query 2 looks a bit like what DMC does, so I'd trust that much more than per index thruput approach. I guess, per index, thruput will also include metrics from your forwarders. Maybe that explains the difference? Also, note the metrics documentation: https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Aboutmetricslog#Thruput_messages

It states the following regarding per_x_thruput data:

Note: The per_x_thruput categories are not complete. Remember that by default metrics.log shows the 10 busiest of each type, for each sampling window. If you have 2000 active forwarders, you cannot expect to see the majority of them in this data. You can adjust the sampling quantity, but this will increase the chattiness of metrics.log and the resulting indexing load and _internal index size. The sampling quantity is adjustable in limits.conf, [metrics] maxseries = num.

In general, if you have a DMC set up, just use the license usage dashboard built into that, if you don't have DMC set up, this might be a good moment to do that 🙂

View solution in original post

joshd
Builder

You can also simply use the extremely popular MetaWoot! App for Splunk. It provides a ton of useful reporting, including license reports...

https://splunkbase.splunk.com/app/2949/

FrankVl
Ultra Champion

Query 2 looks a bit like what DMC does, so I'd trust that much more than per index thruput approach. I guess, per index, thruput will also include metrics from your forwarders. Maybe that explains the difference? Also, note the metrics documentation: https://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Aboutmetricslog#Thruput_messages

It states the following regarding per_x_thruput data:

Note: The per_x_thruput categories are not complete. Remember that by default metrics.log shows the 10 busiest of each type, for each sampling window. If you have 2000 active forwarders, you cannot expect to see the majority of them in this data. You can adjust the sampling quantity, but this will increase the chattiness of metrics.log and the resulting indexing load and _internal index size. The sampling quantity is adjustable in limits.conf, [metrics] maxseries = num.

In general, if you have a DMC set up, just use the license usage dashboard built into that, if you don't have DMC set up, this might be a good moment to do that 🙂

prathapkcsc
Explorer

Hello,

Thanks for your reply. While doing the DMC setup there is a warning coming up , you can refer the same in attached pic.
I would like to know which is the best query for splunk license usage calculation. So that i can go with the Renewal according to the usage.alt text

0 Karma

FrankVl
Ultra Champion

Yeah, in a distributed environment you want to setup the DMC on a separate system, not on one of your production search heads.

I recently used the following in a place where I didn't have DMC access (also derived from what DMC does):

index="_internal" source="*license_usage.log" type=RolloverSummary earliest=-30d@d latest=now
| eval _time = _time - 43200
| bin _time span=1d
| stats latest(b) AS b by slave,pool,_time
| eval DailyGB=round(bytes/1024/1024/1024,2)
| timechart sum(DailyGB) as "volume (GB)" span=1d

Another way to do it is to sum over the usage type data in the license usage log, but the above is much faster. The advantage of using the type=Usage is that you can split the statistics by index (as well as some other things like source / sourcetype / host).

index="_internal" source="*license_usage.log" type=Usage earliest=-30d@d latest=now
| bin _time span=1d
| stats sum(b) AS bytes by _time,idx
| eval DailyGB=bytes/1024/1024/1024
| timechart sum(DailyGB) as "volume (GB)" by idx span=1d
0 Karma

prathapkcsc
Explorer

HI,
I understood that peek_index_throughput means the amount of data does travel through the index pipeline. This changes time to time depends on the system performance, network etc etc.
Thanks for your quick help on this. I used query-2 for the splunk license usage.

0 Karma

FrankVl
Ultra Champion

Glad to have helped. If the answer worked for you, please mark it as accepted, so others can also see this question was answered successfully and which answer was correct.

0 Karma

prathapkcsc
Explorer

HI,

One final question. Can you just elaborate what is mean by per_index_throughput?
Is it the amount data passed the index pipeline ? If yes how it is different with query-2.

Thanks

0 Karma