topic Re: How to find number of events, size (in GB), and frequencies of ingestion? in Splunk Search

How to find number of events, size (in GB), and frequencies of ingestion?

SplunkDash — Fri, 25 Feb 2022 18:20:56 GMT

Hello,

are there any queries we can use to find the Total Number of Events, Total Size/Volume (in GB) of Data, Frequencies of data coming into SPLUNK by index and sourcetype. Any help will be highly appreciated, thank you!

Re: How to find number of events, size (in GB), and frequencies of ingestion?

richgalloway — Fri, 25 Feb 2022 20:36:36 GMT

Have you looked at the Monitoring Console? It should have most of what you seek.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

SplunkDash — Fri, 25 Feb 2022 21:07:22 GMT

Hello,

Thank you so much for your quick response, truly appreciated. Yes, I saw that, it is in the monitor console. But, is the any way I can get same data using queries and make a report to send as a csv file. Or is there any way I can export that data as csv from monitor console. Thank you again.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

richgalloway — Sat, 26 Feb 2022 00:31:35 GMT

In the lower-right corner of most of the MC panels you should find a magnifying glass icon. It will only appear when your cursor is in the area. Click the icon to open the panel in a search window. Then you will have the query which you can modify or copy.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

SplunkDash — Sat, 26 Feb 2022 16:08:48 GMT

Hello,

Thank you so much. Yes, it's working as expected for counting events by host/source/sourcetype. Here are the codes what I got:

Codes for counting event by host:

| metadata type=hosts index=test_prod splunk_server=<servername> datatype=event

| fields host, totalCount

| sort - totalCount

| rename host as Host, totalCount as "Event Count"

But most likely I don't have access to view/have volume/size (in MB) by host/source/sourcetype. are there any ways to tweak these above codes to get the volume/size of data (in MB). Thank you again and any help will be highly appreciated.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

richgalloway — Sat, 26 Feb 2022 17:51:24 GMT

"Most likely"? Have you tried? If you don't have access then no tweaks to a search will get you access.

The data you seek should be available on the MC of your license manager instance. Go to Indexing->License Usage->Historic License Usage. There are dropdowns to select the data will be grouped or you can copy the queries to customize.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

SplunkDash — Sun, 27 Feb 2022 00:23:07 GMT

Hello,

Thank you so much again! But still, I am not getting expected output from that option, however, I am getting result from this query:

index=_internal(test_prod) source=*license_usage.log type=Usage|stats sum(b) as Bytes by idx st s h |eval MB=round(Bytes/1024/1024,3)

But, missing some of the sourcetypes, there should be 12 sourcetypes under that index, however, getting only 8 sourcetypes (using "All time") when I use this query.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

richgalloway — Wed, 02 Mar 2022 15:25:36 GMT

I'm confused by use of the term "index=_internal(test_prod)". That doesn't look like a valid index name.

Re: How to find number of events, size (in GB), and frequencies of ingestion?

SplunkDash — Wed, 02 Mar 2022 15:40:19 GMT

Hello,

index=_internal is the actual term and "test_prod" is the user defined index. So, index=_internal gives us all the index names and info; index=_internal(test_prod) gives info only for test_prod. Does it make sense to you? Please feel free to let me know if you have any questions. Thank you!

Re: How to find number of events, size (in GB), and frequencies of ingestion?

richgalloway — Thu, 03 Mar 2022 01:07:55 GMT

Have you compared the sourcetypes returned by tstats with those returned by metadata?

| metadata type=sourcetypes index=*

If that finds 12 sourcetypes then we'll have pick apart the tstats query to find the problem.