topic Re: Volume of SYSLOG traffic coming into splunk? in Getting Data In

Volume of SYSLOG traffic coming into splunk?

leonardw — Thu, 12 Aug 2010 04:55:17 GMT

Does anyone know how to determine the volume of SYSLOG traffic coming into Splunk over a 30, 60, and 90 day period?

Re: Volume of SYSLOG traffic coming into splunk?

Lamar — Thu, 12 Aug 2010 05:09:58 GMT

Something like this will give you what you need.

index=_internal (source=*/metrics.log* OR source=*\\metrics.log*) group=per_sourcetype_thruput series=syslog | timechart sum(kb) by series

Then, just change your timeframe to 30, 60 or 90 days appropriately.

Re: Volume of SYSLOG traffic coming into splunk?

Simeon — Thu, 12 Aug 2010 06:47:18 GMT

If syslog is one of the top ten sourcetypes or sources, then you can use the metrics log search below:

index=_internal source=*metrics.log* per_sourcetype_thruput series=syslog | eval gb=kb/1024/1024 | stats sum(gb) as Total

If you want to see a charted breakdown, you can substitute the timechart command for stats:

... | timechart span=1d sum(gb) as Total | addtotals

For the above searches, use the TimeRange picker to select the 30, 60, or 90 day duration. If syslog is not in the top ten sourcetypes, you'll have to do a somewhat expensive query that measures the raw size of the events. So something like:

sourcetype=syslog | eval size=len(_raw) | stats sum(size)

Re: Volume of SYSLOG traffic coming into splunk?

leonardw — Thu, 12 Aug 2010 10:45:50 GMT

The above query works; however anything after 30 days there is no data showing up. This system has been in production for several months. Could it be with logs being rotated? If so, how to proceed?

Re: Volume of SYSLOG traffic coming into splunk?

leonardw — Thu, 12 Aug 2010 10:46:35 GMT

BTW - Thank you for your help with this. It's greatly appreciated.

Re: Volume of SYSLOG traffic coming into splunk?

BunnyHop — Thu, 12 Aug 2010 21:33:51 GMT

This might also help.

http://www.splunkbase.com/apps/All/4.x/App/app:Splunk+License+Usage

You can tweak the xml to your needs.

Re: Volume of SYSLOG traffic coming into splunk?

Lamar — Sat, 14 Aug 2010 00:34:40 GMT

Sadly, I believe the default setting for _internal is 28 days or 2419200 seconds. You can raise that setting 'frozenTimePeriodInSecs' to whatever fits for you so that you can get a greater field of view.

If there is no coldToFrozenScript set, it simply erases the data.