How determine which source/host/sourcetype has comsumed large amount of license for past 24 hours?

[Hemnaath]

Hi All, Can any one guide us on how to search a which source/host/source type are consuming very high licenses, as we had seen sudden increase in license usage it seems it was due to one of syslog source has spiked high, but not sure how zero down which source had consumed the licenses. Kindly provide me the query to capture the source which had consumed large amount of license for past 24 hours.

thanks in advance.

[dbcase]

Here is what I use to get usage by index

index=_internal source="*license_usage.log*"|where idx!="NULL"| timechart limit=0 span=1d sum(eval(round(b/1024/1024/1024,5))) by idx

Then this one to get usage by host

index=_internal sourcetype=splunkd group=per_host_thruput earliest=-1mon@mon latest=@mon NOT series=idx* NOT series=lm1* NOT series=c0m1* NOT series=sh1*| bucket _time span=1d | stats sum(kb) as total by series,_time | stats avg(total) as average by series |eval averageMB=round(average/1024,2) |fields - average |sort -averageMB| rename series as "Host Server",averageMB as "Average size per day in MB"

[rjthibod · Accepted solution]

Here is query that breaks it down by host, source, sourcetype, and index where you see the individual values and the totals for each dimension. Not the most efficient, but it gives you all degrees of freedom to play with.

index=_internal source=*license_usage.log* type="Usage" idx=* earliest=-24h@d
| stats sum(b) as bytes by h s st idx
| eventstats sum(bytes) as TotalSourcetype by st
| eventstats sum(bytes) as TotalSource by s
| eventstats sum(bytes) as TotalIndex by idx
| eventstats sum(bytes) as TotalHost by h
| sort -TotalSourcetype

[Hemnaath]

Hi Rjthibod,

thanks for providing some inputs to capture the source which had consumed large amount of license for past 24 hours. I have modified as per our requirement and got the expected output.

index=_internal source=license_usage.log type="Usage" idx=*
| stats sum(b) as bytes by host source sourcetype idx
| eventstats sum(bytes) as TotalSourcetype by sourcetype
| eventstats sum(bytes) as TotalSource by source
| eventstats sum(bytes) as TotalIndex by idx
| eventstats sum(bytes) as TotalHost by host | eval Mbytes=bytes/(1024*1024) | sort -TotalSource Mbytes | eventstats sum(Mbytes) as Totalcount by sourcetype | eval P=round((Mbytes/Totalcount*100),2)|eval Percentage=P+" "+"%" | table host source sourcetype idx Mbytes Percentage | sort - Percentage

[muebel]

Hi Hemnaath, you can get a report on license utilization as described more here : http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/AboutSplunksLicenseUsageReportView

In particular, you can split by the various meta fields (index / host / source / sourcetype) in order to get a better idea what is blowing up your license.

In particular, the search used to drive this report is:

index=_internal host=yourLicenseServer source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time, pool, s, st, h, idx   | timechart span=1d sum(b) AS volumeB by s fixedrange=false  | join type=outer _time [search index=_internal host=yourLicenseServer source=*license_usage.log* type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | stats latest(stacksz) AS "stack size" by _time] | fields - _timediff  | foreach * [eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

you'd have to have access to the _internal index of course, and adjust the host= line to be what the host value is for you license server.

Additionally, you can see the span here is set as span=1d at a couple points. Feel free to adjust that along with the overall search time frame to suit your needs.

Please let me know if this answers your question! 😄

[Hemnaath]

Is there any query which I can execute and get the required result, as I dnt have an access to that license manager. so could you please share me the query to find out the source which had consumed large amount of license for past 24 hours.

thanks in advance.

[Hemnaath]

Hi All can any one provide a query to find out the source which had consumed large amount of license for past 24 hours.

thanks in advance.

[somesoni2]

Try this

index=_internal host=yourLicenseServer source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | stats sum(b) as usage by idx st s h

To get top license usage indexes

above query | stats sum(usage) as usage by idx | sort 10 -usage

To get top license usage sourcetype

above query | stats sum(usage) as usage by st| sort 10 -usage

To get top license usage sources

above query | stats sum(usage) as usage by s| sort 10 -usage

Run it for whatever timerange you want to run. For this query to work, you should've License Master's splunkd logs coming to your Indexers OR run this from License master server's UI.