topic Re: Data ingestion statistics in Splunk Search

Which applications are ingesting more data and violating the license?

blbr123 — Thu, 10 Feb 2022 01:18:31 GMT

Hi All,

I would like to know which applications are ingesting more data and violating the license.

I tried the below query but I am not sure if it gives correct results.

index=_internal source=*license_usage.log type=”Usage” splunk_server=*
| eval Date=strftime(_time, “%Y/%m/%d”)
| streamstats sum(b) as volume
| eval MB=round(volume/1024/1024,5)
| timechart span=1w avg(MB) by idx

index=_internal source=*license_usage.log type=Usage 
| stats sum(b) as bytes by h 
| eval MB = round(bytes/1024/1024,1)
| fields h MB
| rename h as host

Re: Data ingestion statistics

gcusello — Tue, 08 Feb 2022 15:14:33 GMT

Hi @blbr123,

you already have this search in License consuption by index [Settings -- Licensing -- Usage Report -- Previous 60 days -- Split by index (or surcetype)].

Remember that anyway, you could have two problems:

usually retention on _internal is few days, so you could not have 60 days of logs,
you could have on the same index (or the same sourcetype) more applications.

Anyway, the search is:

index=_internal [`set_local_host`] source=*license_usage.log* type="Usage" | eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) | eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) | eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) | bin _time span=1d | stats sum(b) as b by _time, pool, s, st, h, idx | timechart span=1d sum(b) AS volumeB by idx fixedrange=false | join type=outer _time [search index=_internal [`set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d | eval _time=_time - 43200 | bin _time span=1d | dedup _time stack | stats sum(stacksz) AS "stack size" by _time] | fields - _timediff | foreach * [eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

You have similar searches also in the Monitoring Console App or installing the License consuption App.

Ciao.

Re: Data ingestion statistics

blbr123 — Tue, 08 Feb 2022 23:15:37 GMT

Why do we need to use set_local_host and what it does?

Re: Data ingestion statistics

isoutamo — Wed, 09 Feb 2022 07:11:20 GMT

it should be "host=<your LM host>".

r. Ismo