How do you calculate the size of indexed data as per source type?

[rakesh44]

Hi friends,

I have two different source types, each with the same Index...

| dbinspect index=myindex | eval GB=sizeOnDiskMB/1024 | stat sum(GB)     

( It is giving over all indexed size )

...but, I am looking size as per source type , have type and payabal source type. I don't have a monitoring console.

Thanks

[jvishwak]

Check if this helps:
index=_internal| eval size = len(_raw) | stats sum(size) as rawSize by sourcetype | eval mbSize = round(rawSize / 1024 / 1024, 2)

[Vijeta · Accepted solution]

@rakesh44 - you cannot find the usage data by searching on index=myindex, the index _internal stores the usage for each index and sourcetype. You can use below search , given that your role has permission to search on _internal index, if this search doesn't work for you ask someone with admin role to run it.

index=_internal source=*license_usage.log*  type=Usage idx=<yourindexname> | eval GB=b/1024/1024/1024 | stats sum(GB) by st

[rakesh44]

index=_internal source=license_usage.log type=Usage idx= | eval GB=b/1024/1024/1024 | stats sum(GB) by st

when i running above command in local machine getting below error

Error in 'search' command: Unable to parse the search: Comparator '>' is missing a term on the right hand side.
The search job has failed due to an error. You may be able view the job in the Job Inspector.

[Vijeta]

Hi @rakesh44 you need to write idx = the name of your index , no need to put <> sign

[rakesh44]

index=_internal source=license_usage.log type=Usage idx= | eval GB=b/1024/1024/1024 | stats sum(GB) by st

It worked for me Thanks

[Vijeta]

@rakesh44 Please accept the answer if it worked .

[rakesh44]

Kindly ignore previous comment , tried running below command in local machine which has splunk

index=_internal source=license_usage.log type=Usage idx=test | eval GB=b/1024/1024/1024 | stats sum(GB) by st

[mayurr98]

Hi
can you try this :

index=_internal 
    [ `set_local_host`] source=*license_usage.log* type="Usage" 
| eval h=if(len(h)=0 OR isnull(h),"(SQUASHED)",h) 
| eval s=if(len(s)=0 OR isnull(s),"(SQUASHED)",s) 
| eval idx=if(len(idx)=0 OR isnull(idx),"(UNKNOWN)",idx) 
| bin _time span=1d 
| stats sum(b) as b by _time, pool, s, st, h, idx 
| timechart span=1d sum(b) AS volumeB by st fixedrange=false 
| join type=outer _time 
    [ search index=_internal 
        [ `set_local_host`] source=*license_usage.log* type="RolloverSummary" earliest=-30d@d 
    | eval _time=_time - 43200 
    | bin _time span=1d 
    | stats latest(stacksz) AS "stack size" by _time] 
| fields - _timediff 
| foreach * 
    [ eval <<FIELD>>=round('<<FIELD>>'/1024/1024/1024, 3)]

You can find the same on license master :

go to Licensing > Usage Report > Previous 30 days > Split by Sourcetype

let me know if this helps!

[mayurr98]

NOTE: You need to run this query on the license master if you have not forwarded internal logs to indexer.
are you running it on license master?

[rakesh44]

no am not running on license master

[rakesh44]

Thanks for quick reply, I tried above command but did not work. I dot have monitoring console.Thanks

[vishaltaneja070]

Hello @rakesh44

try this:

index=_internal source=*license_usage.log*  type=Usage pool=*  | eval GB=b/1024/1024/1024 | stats sum(GB) by st

[rakesh44]

am sorry not able to run above command, but i tried one thing, correct me is this correct

index=myindex sourcetype=type ( then i checked in activity tab , under job section and find no of events, size , is size is Indexed size ?) Thanks

[vishaltaneja070]

@rakesh44

No i think that is event size on disk no the licensed size.

What is the error you are getting while running this command?

[rakesh44]

no error no result found

index=_internal source=license_usage.log type=Usage pool=* | eval GB=b/1024/1024/1024 | stats sum(GB) by st

[vishaltaneja070]

You can either run this search on License master or on SH if you have forwarded the _internal logs to Indexer layer.

As you need details with sourcetype, so metrices.log will also not help you.

[Vijeta]

@rakesh44 you probably don't have the permissions to search on _internal index that is why seeing no results.

[rakesh44]

If my Index name is MyIndex and have source type is Payable ten below is command.

index=MyIndex source=license_usage.log type=Usage pool=* | eval GB=b/1024/1024/1024 | stats sum(GB) by Payable

Can you confirm above command is correct.Thanks