topic Re: Calculating _internal log volume for a particular host in Splunk Search

Calculating _internal log volume for a particular host

dky — Tue, 08 Mar 2016 20:42:47 GMT

Hello, I'm trying to determine how much traffic gb/mb/kb that a particular forwarder is sending in daily. I'm using the current command:

index=_internal* host="somehost.mydomain.com" | timechart span=1h avg(kb)

This is giving me values like:

5,832.547626 per hour. I'm assuming to get GB I would divide by 1024/1024?

Thanks

Re: Calculating _internal log volume for a particular host

vasildavid — Tue, 08 Mar 2016 21:54:07 GMT

The license_usage.log provides data in bytes. To get the base2 representation of GiB you would indeed divide by bytes/1024/1024/1024 like below:

index=_internal host=license_manager source=*license_usage.log type="Usage" 
 | stats sum(b) as b by h
 | eval gb=round(b/1024/1024/1024, 3)

Re: Calculating _internal log volume for a particular host

martin_mueller — Tue, 08 Mar 2016 21:54:15 GMT

Are you looking for the volume of _internal data sent by that host, or are you trying to search events in _internal to determine overall volume sent by that host?

Re: Calculating _internal log volume for a particular host

dky — Tue, 08 Mar 2016 22:09:20 GMT

Correct, I'm estimating how much network traffic received by a single host so I can estimate how much it would cost us to send this volume into the cloud.

Re: Calculating _internal log volume for a particular host

dky — Wed, 09 Mar 2016 13:50:53 GMT

What is host=license_manager? I don't have that host so the search returns null.

Re: Calculating _internal log volume for a particular host

vasildavid — Wed, 09 Mar 2016 13:54:02 GMT

Replace "license_manager" with the host that is acting as your license manager.

Re: Calculating _internal log volume for a particular host

dky — Wed, 09 Mar 2016 14:26:36 GMT

Thanks, I did that, but now there is no field called "type"?

Re: Calculating _internal log volume for a particular host

somesoni2 — Wed, 09 Mar 2016 15:45:38 GMT

This data is generated in the License Server only, so unless you're forwarding _internal data from your License server to Indexers, you need to run this from License server Web UI.

Re: Calculating _internal log volume for a particular host

dky — Wed, 09 Mar 2016 18:49:35 GMT

Thanks, worked on the license server. Let me try to apply timechart to it.

Re: Calculating _internal log volume for a particular host

martin_mueller — Wed, 09 Mar 2016 20:05:00 GMT

While you're on the license master, hit up Settings -> Licensing -> Usage Report -> Last 30 days - that should come pretty close to what you're looking for out of the box.

Re: Calculating _internal log volume for a particular host

dky — Wed, 09 Mar 2016 21:49:50 GMT

I wanted to get the events hourly so I came up with this:

index=_internal host=license.example.com source=*license_usage.log type="Usage" h=someforwarder.example.com | eval megabytes=b/1024/1024 | timechart sum(megabytes)

Does this look correct to you guys? Goal is to get MB transferred each hour thorough the day.

Re: Calculating _internal log volume for a particular host

martin_mueller — Wed, 09 Mar 2016 23:06:05 GMT

Looks okay.

Keep in mind that this is licensed volume, not transferred volume - doesn't include internal logs or filtered events.

Re: Calculating _internal log volume for a particular host

dky — Wed, 09 Mar 2016 23:47:33 GMT

Thanks for all the assistance with this.