Getting Data In

How to calculate volume of syslog traffic on syslog-ng server


I want to calculate the volume of Syslog data coming into Syslog-ng server.
For example, I am forwarding proxy logs to Syslog-ng server on port say 514.
I want to calculate the data coming in.
Can someone please help me with this?


You could check the directory size on the syslog-ng server, your /etc/syslog-ng/syslog-ng.conf will tell you where you are storing the incoming syslog data. du -h <directory> will provide you the size of directory in typical linux environment.

0 Karma

Ultra Champion

What exactly do you want to do? Have you already ingested this data into Splunk and want to now calculate the volume of it? Or do you need to calculate (estimate?) the volume of data coming in in preparation for onboarding this data into Splunk?

In the first case, the best place to look would be the splunk internal logs on license usage for this particular source / sourcetype / host / index (depending what is easiest to distinguish data coming from this particular syslog server). For example to get the license usage for sourcetype syslog over past 14 days:

index="_internal" source="*license_usage.log" type=Usage earliest=-14d@d latest=@d st=syslog | timechart span=1d sum(eval(b/1024/1024)) AS DailyMB

If the data is not in Splunk yet, you would have to look at the size of the files syslog-ng has written and divide that by the time span those files cover.

.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!