How can we get usage related data from index ?

[arjun]

How can we locate usage related data from splunk, I have onpremise splunk instance and looking for usage and billing related data grouped by day.
I am not able to locate data in any index.

[arjun]

Hi @gcusello , We have many client who uses splunk and we need to get some data from those splunk server

I am trying to get a way with SPL to get those data. 

Basic Data that we need from those splunk system are 

1 )  detailed information about resources, their usage, and associated costs.

But i am not sure which index will have this data ? does _telemetry index will have all required data to know how much utilisation has been done day by day ?  

I hope this define my requirement clearly.

[PickleRick]

Your description is still way incomplete. But whatever your exact use case is, I agree with @gcusello that it's something that you should work with your local Splunk Partner on - have an experienced Architect or Consultant go through your use case and see what can be done and how.

[gcusello]

Hi @arjun ,

multi tenency  implementation isn't a Community job and it requires an analysis and a design by a Splunk Architect.

You should define rules to identify customers and assign to each of them an index overriding the default.

So first job is to identify rules (regexes) and then apply on your Heavy Forwarders (if present) or on your Indexers something like this:

# transforms.conf 
[overrideindex_customer1]
DEST_KEY =_MetaData:Index
REGEX = .
FORMAT = customer1_index

# props.conf 
[host::customer1_host]
TRANSFORMS-index = overrideindex_customer1

Ciao.

Giuseppe

[arjun]

Hi @gcusello  i am trying to get data related to usage and billing from splunk, here is query i am using for that

index=_telemetry source=*license_usage_summary.log*
| bin _time span=1d
| stats sum(b) as TotalBytes by _time
| eval GB=round(TotalBytes / (1024 * 1024 * 1024), 2)
| timechart span=1d values(GB) as "Daily Indexed GB"

And per my research spulnk has few more such index like _internal and _audit 

I just want to know if this is correct approach or not 

[gcusello]

Hi @arjun ,

you can calculate the License consuption per day using the [Settings > License > License Consuption > Past days > by index ].

using your search you have all the license consuption, you cannot divide them for customer, as I already said: multitenency isn't a Community topic, it requires a Splunk PS or a Certified Architect that already did this job (like me).

Ciao.

Giuseppe

[PickleRick]

Do you mean you want to monitor your Splunk infrastructure usage or do you ingest some data regarding "external" hosts? For the former as @dural_yyz mentioned, check Monitoring Console. You can also gather metrics from the _metrics index. For the latter - it depends on your environment. Splunk "just" happily gets the data you throw at it and can manipulate and search it. But it's up to your architects and admins to tell you where they set up the data and what it's made of.

[arjun]

HI @gcusello , 

In splunk we monitor devices or Host and we get logs from them what i need to know how much memory (in GB) has been utilised by those host or log source where does splunk store such data in case of Onpremise instance ?

[gcusello]

Hi @arjun ,

to monitor windows or Linux machines having a Universal Forwarder installed, you have to install on these UFs the related add on (Linux https://splunkbase.splunk.com/app/833 or windows https://splunkbase.splunk.com/app/742 ) enabling the input stanza for memory monitoring.

In this way you'll have the logs to use in your searches.

Ciao.

Giuseppe

[dural_yyz]

Start with the DMC (Distributed Monitoring Console) to review the License usage broken down by index.  This will share with you the daily ingest records for the last 30 days broken down by index.  This is only a starting point as depending on how your environment was setup you may have very specific indexes or things may have been aggregated into only a few indexes.

From there you can start decided what questions come next.

[gcusello]

Hi @arjun ,

what's your requirement: to know the volume for each customer? or what else?

Could you better describe your environment and your situation?

E.g.: have you a multi-tenant environment or not?

Ciao.

Giuseppe