Splunk Add-on for ISC BIND: How to index a single data source and apply multiple sourcetypes in Hunk?

[jwalzerpitt]

We use BlueCat for DNS/DHCP and we are forwarding the DNS/DHCP logs via CEF format to HDFS. I am trying to reverse engineer the Splunk Add-on for ISC BIND for Hunk, specifically the assigning of multiple sourcetypes to one index (assign events to their proper sourcetype - isc:bind:query, isc:bind:lameserver, isc:bind:network, isc:bind:transfer).

How would I go about configuring the props.conf file to assigning multiple sourcetypes to my source of BlueCat?

Thx

[kschon_splunk]

You should be able to do this with regular expressions. This page should help:
http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Advancedsourcetypeoverrides

[jwalzerpitt]

I have my regexes ready to go, but do I list each sourcetype as follows in props.conf as:

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:query

[isc:bind:query]
REPORT-1_extract_field = isc_bind_query_extract_field_0
EVAL-message_type = "Query"
EVAL-query_type = "Query"
EVAL-vendor_product = "ISC:Bind"
LOOKUP-2_look_up_extract = isc_bind_severities_lookup vendor_severity OUTPUT severity
LOOKUP-3_look_up_extract = isc_bind_category_lookup sourcetype OUTPUT vendor_category

transforms.conf
[isc_bind_query_extract_field_0]
REGEX = \s+client\s+([\w-.:]{1,100})#(\d{1,5})(?:\s+([^)]+))?:(?:\s+view\s+[^:]+:)?\s+query:\s+(?([\w-.:]{1,100}))?\s+([^\s]+)\s+([^\s]+)\s++-\s+(([\w-.:]{1,100}))$
FORMAT = vendor_severity::$1 src::$2 src_port::$3 query::$4 record_class::$5 record_type::$6 flag::$7 dest::$8

and so on?

[isc:bind:lameserver]
REPORT-1_extract_field = isc_bind_lameserver_extract_field_0
EVAL-app = "ISC:Bind"
EVAL-type = "alert"
LOOKUP-2_look_up_extract = isc_bind_severities_lookup vendor_severity OUTPUT severity
LOOKUP-3_look_up_extract = isc_bind_category_lookup sourcetype OUTPUT vendor_category

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:lameserver

transforms.conf
[isc_bind_lameserver_extract_field_0]
REGEX = (?:\s+lame-servers:)?(?:\s+([^:]+):)?\s+(error\s+(([^)]+))\s+resolving\s+'([\w-.:]{1,100})/([^/]+)/([^']+)':\s+([\w-.:]{1,100})#(\d{1,5}))$
FORMAT = vendor_severity::$1 body::$2 error_type::$3 query::$4 record_type::$5 record_class::$6 dest::$7 dest_port::$8

Thx

[kschon_splunk]

(Take the following with a big grain of salt, as I haven't had a chance to test it myself)
I think what you want to do is something like the following. Choose one sourcetype to be the default for the index, i.e.:

[source::/xyz/dnslogs/...]
sourcetype = isc:bind:lameserver

Then for every other sourcetype you wish to assign, do the following:

transforms.conf
[disambiguate_isc_bind_query]
REGEX = \s+client\s+([\w-.:]{1,100})#(\d{1,5})(?:\s+([^)]+))?:(?:\s+view\s+[^:]+:)?\s+query:\s+(?([\w-.:]{1,100}))?\s+([^\s]+)\s+([^\s]+)\s++-\s+(([\w-.:]{1,100}))$
FORMAT = sourcetype:isc:bind:query
DEST_KEY = MetaData:Sourcetype

props.conf
[source::/xyz/dnslogs/...]

TRANSFORMS-isc_bind_query_transform = disambiguate_isc_bind_query

After combining all stanzas with the same header, your stanza in props should looks like:
[source::/xyz/dnslogs/...]

TRANSFORMS-isc_bind_query_transform = disambiguate_isc_bind_query
TRANSFORMS-isc_bind_network = disambiguate_isc_bind_network
......
......

[ddrillic]

In Hunk one assigns the sourcetype by the association to the file location in the HDFS.

Hunk - assigning sourcetype

Gives the example of -

[source::/user/xyz/ciscologfiles/...]
sourcetype = cisco_syslog

[source::/user/xyz/iislogfiles/...]
sourcetype = iis

Do you need to associate multiple sourcetypes for one source?

[jwalzerpitt]

Yes, I would like to associate multiple sourcetypes to one source.

Thx