topic Re: Count Sourcetype by Day in Splunk Search

Count Sourcetype by Day

laberthelemy — Thu, 28 Jul 2016 14:02:04 GMT

In 6.4.2 version,
when i try to count the integrated volume by sourcetype last day for example with this search :

earliest=-1d@d latest=@d  index=_internal source=*license_usage.log* type=Usage 
 | stats sum(b) AS Bytes by st | eval GB = (Bytes/1024/10241024/) | sort -GB

The volume seems to be wrong.

Sourcetype --------Bytes ---------------------GB
opsec---------4693489783100---------- 4371.152989
f5:hsl----------4472278291965---------- 4165.133733

The real volume is near 43 Go but not 4371 Go

In version 6.3.2, the results were correct.

Have you one explanation please ?

Re: Count Sourcetype by Day

inventsekar — Tue, 29 Sep 2020 10:25:11 GMT

4693489783100 bytes is 4693.4897831 GB.
so, something wrong with "Bytes" calculation only. (earliest=-1d@d latest=@d index=_internal source=license_usage.log type=Usage | stats sum(b) AS Bytes by st)

Re: Count Sourcetype by Day

axl88 — Thu, 28 Jul 2016 16:23:45 GMT

When I google byte to GB and enter information you have in your question,
4693489783100 bytes is equal to 4693.4897831 gb
Do you think that first number is calculated wrong? in 6.32, does it give you 46934897831 in bytes?

Re: Count Sourcetype by Day

Raghav2384 — Thu, 28 Jul 2016 16:28:15 GMT

Hello,

The search you posted, is that what you are running exactly or did you just type it?Reason i ask, I see a typo in there
Highlighted in Bold.

earliest=-1d@d latest=@d  index=_internal source=*license_usage.log* type=Usage 
  | stats sum(b) AS Bytes by st | eval GB = (**Bytes/1024/10241024/**) | sort -GB

As far as the Bytes to GB calculation, that is absolutely right. I used my 6.4 splunk as well as a calculator and 4693489783100 Bytes is 4371.152989GB

I just ran the search on my LURV by sourcetype and GB calculation is correct. I suspect the Bytes value. It's almost like something is forcing the timerange to be much higher than the -1d@d ,@d . Again, mine is 6.4 and not 6.4.2.May be 6.4.2 has a bug to report the Bytes value some extra. Try not putting the earliest and latest in the search, instead use the Time picker and see if it changes (Guess).

Hope this helps!

Thanks,
Raghav

Re: Count Sourcetype by Day

laberthelemy — Fri, 29 Jul 2016 09:47:16 GMT

Hello,
I tried not putting the earliest and latest in the search but using Time picker. The result is the same .
When I use Deployment Monitor /All Sourcetypes , the result is the same.
May be , as you say, the 6.4.2 has a bug to report the Bytes.

Thanks for your Help
laberthelemy

Re: Count Sourcetype by Day

inventsekar — Tue, 29 Sep 2020 10:26:02 GMT

Check this one.. from a similar issue post -
The only reliable source for license usage is the $SPLUNK_HOME/var/log/splunk/license_usage.log file on your license master instance, and unfortunately it does not split usage by index.

There is, however, a sampled record of kilobytes indexed for the top 10 most active indexes every 30s in metrics.log. Of course if you have less than 10 active indexes, this is not an issue.

So, for a report showing daily license usage over the past 1 day, you would run:

index=_internal source=*license_usage.log type=RolloverSummary earliest=-1d

|eval GB = b/1024/1024/1024 | eval _time = _time - 43200

| timechart span=1d sum(GB) AS "Total GB used"

For a report showing estimated daily volume indexed (whether it counted against your license quota or not) over the past 7 days, you would run:

index=_internal (host=indexer1 OR host=indexer2 OR host=license_master) source=*metrics.log group=per_index_thruput earliest=-7d
| timechart span=1d sum(eval(kb/1024)) AS "MB indexed" by series

Re: Count Sourcetype by Day

laberthelemy — Fri, 29 Jul 2016 12:11:05 GMT

With this search, the result shows a daily licence usage and unfortunately it does not split usage by sourcetype.
But the result shows also a problem of conversion Bytes to Go
Whe have a licence of 300 Go and the total is around of 21012 Go

_time -------------------------------------Total GB used

2016-07-28 ---------------------------------21012.868955

In 6.4.2, may be, the search has to be adapted

Thanks for your Help

Re: Count Sourcetype by Day

laberthelemy — Fri, 29 Jul 2016 15:54:52 GMT

In 6.3.2, the calcul was right. Now, in 6.4.2, the result is not the reality. Something has been changed . The search, certainly, has to be changed