Splunk Search

One file with two sourcetype Need Regex

hartfoml
Motivator

I have my DNS and DHCP logs in one file and I would like to set "TZ = UTC" on the sourcetype. My problem is what would the sourcetype be since the file has both DNS and DHCP in the file.

Here is an example of the logs:

Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: DHCP_RenewLease: Host= IP=x.x.x.x MAC=001b786eb865 Domain=ndc.nasa.gov ClientID=01001b786eb865
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Sent DHCPACK to Client MAC= 001b786eb865 ciaddr= x.x.x.x yiaddr= x.x.x.x client ID= 01001b786eb865
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.161 notify: zone 154.146.in-addr.arpa/IN: sending notifies (serial 98815)
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 /opt/qip/usr/bin/dhcpd[25911]: Received DHCPINFORM from Client MAC= 00237d6f67f7 ciaddr= x.x.x.x requestedIP= client ID= giaddr= x.x.x.x
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR started
Sep 11 12:01:42 ns1 named[15517]: 11-Sep-2013 12:01:42.232 xfer-out: client x.x.x.x#50462: transfer of '154.146.in-addr.arpa/IN': IXFR ended

So I thought maybe i can create two sourcetypes with two monitors useing REGEX to monitor the one file for each of the diferent sourctypes.

Can I do this? What would the REGEX in the [monitor:///myfile] stanza in the inputs.conf look like to run the regex?

Example:

[monitor:////myfile/location]
TZ = UTC
regex="/opt/qip/usr/bin/dhcpd"
sourcetype=dhcp
host=nameserver.mydomain
index=network

Example:
[monitor:////myfile/location]
TZ = UTC
regex="named["
sourcetype=dns
host=nameserver.mydomain
index=network

Tags (2)
0 Karma

kristian_kolb
Ultra Champion

Overlapping [monitor] stanzas in this way won't work. What you can do is to perform an index-time transform of the sourcetype. Assuming that you currently have one [monitor] that specifies one sourcetype, like this

[monitor:///path/to/file]
sourcetype=dns
other params here

Then on the indexer (or where your parsing takes place) you can do like so;

props.conf

[dns]
TRANSFORMS-blah = split_dhcp
other params here

transforms.conf

[split_dhcp]
REGEX = dhcpd\[
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::dhcp

For more info see;

http://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides

Hope this helps,

K

0 Karma

kristian_kolb
Ultra Champion

If the data passes through a Heavy Forwarder, that is where you must do this configuration. This operation is done during the parsing phase, which takes place only once.

See http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings

0 Karma

hartfoml
Motivator

Thanks so much for the help. I will try it out right now.

One question if you don't mind

Can I do this on the heavy forwarder before it is sent to the indexer? I have 3 indexers do I have to setup the props and transforms on all three or can i just do this on the Heavy Forwarder

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...