Getting Data In

How do you calculate the size of indexed data as per source type?

rakesh44
Communicator

Hi friends,

I have two different source types, each with the same Index...

| dbinspect index=myindex | eval GB=sizeOnDiskMB/1024 | stat sum(GB)     

( It is giving over all indexed size )

...but, I am looking size as per source type , have type and payabal source type. I don't have a monitoring console.

Thanks

0 Karma
1 Solution

Vijeta
Influencer

@rakesh44 - you cannot find the usage data by searching on index=myindex, the index _internal stores the usage for each index and sourcetype. You can use below search , given that your role has permission to search on _internal index, if this search doesn't work for you ask someone with admin role to run it.

index=_internal source=*license_usage.log*  type=Usage idx=<yourindexname> | eval GB=b/1024/1024/1024 | stats sum(GB) by st

View solution in original post

0 Karma

jvishwak
Path Finder

Check if this helps:
index=_internal| eval size = len(_raw) | stats sum(size) as rawSize by sourcetype | eval mbSize = round(rawSize / 1024 / 1024, 2)

0 Karma

Vijeta
Influencer

@rakesh44 - you cannot find the usage data by searching on index=myindex, the index _internal stores the usage for each index and sourcetype. You can use below search , given that your role has permission to search on _internal index, if this search doesn't work for you ask someone with admin role to run it.

index=_internal source=*license_usage.log*  type=Usage idx=<yourindexname> | eval GB=b/1024/1024/1024 | stats sum(GB) by st
0 Karma

rakesh44
Communicator

index=_internal source=license_usage.log type=Usage idx= | eval GB=b/1024/1024/1024 | stats sum(GB) by st

when i running above command in local machine getting below error

Error in 'search' command: Unable to parse the search: Comparator '>' is missing a term on the right hand side.
The search job has failed due to an error. You may be able view the job in the Job Inspector.

0 Karma

Vijeta
Influencer

Hi @rakesh44 you need to write idx = the name of your index , no need to put <> sign

0 Karma

rakesh44
Communicator

index=_internal source=license_usage.log type=Usage idx= | eval GB=b/1024/1024/1024 | stats sum(GB) by st

It worked for me Thanks

0 Karma

Vijeta
Influencer

@rakesh44 Please accept the answer if it worked .

0 Karma

rakesh44
Communicator

Kindly ignore previous comment , tried running below command in local machine which has splunk

index=_internal source=license_usage.log type=Usage idx=test | eval GB=b/1024/1024/1024 | stats sum(GB) by st

0 Karma

mayurr98
Super Champion

Hi
can you try this :

index=_internal 
    [ `set_local_host`] source=*license_usage.log* type="Usage" 
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) 
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) 
| eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time, pool, s, st, h, idx 
| timechart span=1d sum(b) AS volumeB by st fixedrange=false 
| join type=outer _time 
    [ search index=_internal 
        [ `set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d 
    | eval _time=_time - 43200 
    | bin _time span=1d 
    | stats latest(stacksz) AS "stack size" by _time] 
| fields - _timediff 
| foreach * 
    [ eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

You can find the same on license master :

go to Licensing > Usage Report > Previous 30 days > Split by Sourcetype

let me know if this helps!

mayurr98
Super Champion

NOTE: You need to run this query on the license master if you have not forwarded internal logs to indexer.
are you running it on license master?

0 Karma

rakesh44
Communicator

no am not running on license master

0 Karma

rakesh44
Communicator

Thanks for quick reply, I tried above command but did not work. I dot have monitoring console.Thanks

0 Karma

vishaltaneja070
Motivator

Hello @rakesh44

try this:

index=_internal source=*license_usage.log*  type=Usage pool=*  | eval GB=b/1024/1024/1024 | stats sum(GB) by st
0 Karma

rakesh44
Communicator

am sorry not able to run above command, but i tried one thing, correct me is this correct

index=myindex sourcetype=type ( then i checked in activity tab , under job section and find no of events, size , is size is Indexed size ?) Thanks

0 Karma

vishaltaneja070
Motivator

@rakesh44

No i think that is event size on disk no the licensed size.

What is the error you are getting while running this command?

0 Karma

rakesh44
Communicator

no error no result found

index=_internal source=license_usage.log type=Usage pool=* | eval GB=b/1024/1024/1024 | stats sum(GB) by st

0 Karma

vishaltaneja070
Motivator

You can either run this search on License master or on SH if you have forwarded the _internal logs to Indexer layer.

As you need details with sourcetype, so metrices.log will also not help you.

0 Karma

Vijeta
Influencer

@rakesh44 you probably don't have the permissions to search on _internal index that is why seeing no results.

0 Karma

rakesh44
Communicator

If my Index name is MyIndex and have source type is Payable ten below is command.

index=MyIndex source=license_usage.log type=Usage pool=* | eval GB=b/1024/1024/1024 | stats sum(GB) by Payable

Can you confirm above command is correct.Thanks

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...