The way we do our logging is to a centralized syslog server, using the remote syslog protocol. All our Cisco gear is configured to log to this server, all Linux/Unix boxes have a *.* @syslog-server line in their syslog config. Even the Windows machines have an app which exports events into a stream of syslog messages that go here. There is very little granularity on the syslog server itself. Basically all incoming messages gets dumped into a single file (a huge file, rotated daily, stored on a netapp).
This huge log file is mounted on the Splunk server as the main input source. We have a couple other sources, but mainly it is just this one file with log data from all sorts of disparate sources (including the BIND/named logging from all our name servers..)
I am currently trying to get the Splunk Add-on for ISC Bind working using this architecture. The instructions say to do the following:
From http://docs.splunk.com/Documentation/AddOns/latest/ISCBIND/Configureinputs
Configure inputs.conf
You can create an inputs.conf file and configure the monitor inputs in this file instead of using Splunk Web.
1. Using a text editor, create a file named inputs.conf in the $SPLUNK_HOME/etc/apps/Splunk_TA_isc-bind/local folder.
2. Add the following stanzas and lines, and save the file:
[monitor:///var/log/named/queries.log]
sourcetype = isc:bind:query
disabled = 0
[monitor:///var/log/named/query-errors.log]
sourcetype = isc:bind:queryerror
disabled = 0
[monitor:///var/log/named/network.log]
sourcetype = isc:bind:network
disabled = 0
[monitor:///var/log/named/notify.log]
sourcetype = isc:bind:transfer
disabled = 0
[monitor:///var/log/lame-servers.log]
sourcetype = isc:bind:lameserver
disabled = 0
This obviously will not work for me... I have sample regular expressions for each of these sourcetypes that needs to be set. My question is: What is the best/correct way to do this? Edit the system/local/transforms.conf with a stanza for each of the isc:bind:XXX sourcetypes... but what would that look like? Do I use DEST_KEY = isc:bind:XXXX, something like:
[iscBindQuery]
DEST_KEY = isc:bind:query
REGEX = query:\s+\(?([\w\-\.:]{1,100})\)?\s+([^\s]+)\s+([^\s]+)\s+[\+\-]([^\s]*)\s+\(([\w\-\.:]{1,100})\)/)
... and so on, with another stanza using DEST_KEY to set the various isc:bind:XXX types... Will that work for setting a "sourcetype" ? Or should I use SOURCE_KEY = isc:bind:XXXX in these stanzas...
Or is there another approach here that I am missing?
Thanks so much for any insight!
