I have my DNS and DHCP logs in one file and I would like to set "TZ = UTC" on the sourcetype. My problem is what would the sourcetype be since the file has both DNS and DHCP in the file.
Here is an example of the logs:
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: DHCP_RenewLease: Host= IP=x.x.x.x MAC=001b786eb865 Domain=ndc.nasa.gov ClientID=01001b786eb865
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Sent DHCPACK to Client MAC= 001b786eb865 ciaddr= x.x.x.x yiaddr= x.x.x.x client ID= 01001b786eb865
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.161 notify: zone 154.146.in-addr.arpa/IN: sending notifies (serial 98815)
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR started
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR ended
So I thought maybe i can create two sourcetypes with two monitors useing REGEX to monitor the one file for each of the diferent sourctypes.
Can I do this? What would the REGEX in the [monitor:///myfile] stanza in the inputs.conf look like to run the regex?
Example:
[monitor:////myfile/location]
TZ = UTC
regex="/opt/qip/usr/bin/dhcpd"
sourcetype=dhcp
host=nameserver.mydomain
index=network
Example:
[monitor:////myfile/location]
TZ = UTC
regex="named["
sourcetype=dns
host=nameserver.mydomain
index=network
Overlapping [monitor]
stanzas in this way won't work. What you can do is to perform an index-time transform of the sourcetype. Assuming that you currently have one [monitor]
that specifies one sourcetype, like this
[monitor:///path/to/file]
sourcetype=dns
other params here
Then on the indexer (or where your parsing takes place) you can do like so;
props.conf
[dns]
TRANSFORMS-blah = split_dhcp
other params here
transforms.conf
[split_dhcp]
REGEX = dhcpd\[
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::dhcp
For more info see;
http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides
Hope this helps,
K
If the data passes through a Heavy Forwarder, that is where you must do this configuration. This operation is done during the parsing phase, which takes place only once.
See http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings
Thanks so much for the help. I will try it out right now.
One question if you don't mind
Can I do this on the heavy forwarder before it is sent to the indexer? I have 3 indexers do I have to setup the props and transforms on all three or can i just do this on the Heavy Forwarder