I can't for the life of me get Splunk to recognize any form of host for UDP connections. As an example, I have a listener set up via inputs.conf on udp/514:
[udp://514] sourcetype = syslog connection_host = dns
That seems to work fine. Now I'd like, for one host, to move to a different index and modify the sourcetype. The splunk index recognizes the host as "dhcpserver" (so DNS lookups are working - though names changed to protect the innocent).
[host::dhcpserver] MAX_TIMESTAMP_LOOKAHEAD = 16 BREAK_ONLY_BEFORE_DATE = True SHOULD_LINEMERGE = False sourcetype = syslog_dhcp TRANSFORMS-index = dhcp-index REPORT-syslog = syslog-extractions
and in transforms.conf:
[dhcp-index] DEST_KEY = _MetaData:Index FORMAT = net_dhcp
I've tried IP address, the FQDN, using wildcards, etc. No matter what I choose, this stanza is not processed. The sourcetype is still set to syslog, and the data is still indexed into main. I've run into other issues trying to use host-based criteria with UDP data sources. Am I not recognizing some inherent limitation of UDP or is there some other issue going on?
extract from btool --debug props list:
system [host::dhcpserver] system BREAK_ONLY_BEFORE = system BREAK_ONLY_BEFORE_DATE = True system CHARSET = UTF-8 system DATETIME_CONFIG = /etc/datetime.xml system LEARN_SOURCETYPE = true system MAX_DAYS_AGO = 2000 system MAX_DAYS_HENCE = 2 system MAX_DIFF_SECS_AGO = 3600 system MAX_DIFF_SECS_HENCE = 604800 system MAX_EVENTS = 256 system MAX_TIMESTAMP_LOOKAHEAD = 16 system MUST_BREAK_AFTER = system MUST_NOT_BREAK_AFTER = system MUST_NOT_BREAK_BEFORE = system REPORT-syslog = syslog-extractions system SEGMENTATION = indexing system SEGMENTATION-all = full system SEGMENTATION-inner = inner system SEGMENTATION-outer = outer system SEGMENTATION-raw = none system SEGMENTATION-standard = standard system SHOULD_LINEMERGE = False system TRANSFORMS = system TRANSFORMS-index = dhcp-index system TRUNCATE = 10000 system maxDist = 100 system sourcetype = syslog_dhcp
I just had this exact same problem. This is what solved it for me:
I didn't use the
WRITE_META=true line though. Here's what I ended up putting in transforms.conf:
# send data from the vmware servers to the vmware index [vmware_set_index] SOURCE_KEY=MetaData:Host DEST_KEY=_MetaData:Index REGEX=^host::vmware-(\d+).example.com$ FORMAT=vmware # set data from vmware servers to vmware_syslog sourcetype [vmware_set_sourcetype] SOURCE_KEY=MetaData:Host DEST_KEY=MetaData:Sourcetype REGEX=^host::vmware-(\d+).example.com$ FORMAT=vmware_syslog
syslog sourcetype modified the
host value before it is indexed (based on the value of the host field in the syslog data). It would be nice if the Splunk UDP input processor actually added the "host::" key to the incoming event, but instead it relies on the transform (if you look in
props.conf for the
[syslog] stanza, you'll see
TRANSFORMS = syslog-host) to set the host.
The problem is this happens at the same time you are trying to make your own index-time settings, so they can't be applied, since at that time, the host has not yet been changed. (Search time configurations are fine.)
What can you do about it? Pretty much, you'll have to base it on a source stanza, i.e., put your configs in
[source::udp:514]. Note this will apply to all data coming in that port, so you may need a bunch of different transforms that use a regex on the raw data to set things right.
I was under the impression that setting sourcetype in inputs.conf takes precedence over all others. Following this line of though, if it is correct, then doesnt the sourcetype = syslog in your inputs.conf take precedence over the props.conf sourcetype = syslog_dhcp
Since splunk seems to be good at understanding that the data is syslog, could you try to remove the line sourcetype = syslog and perform a restart.
What does that lead to?
PS. The index that you are trying to send the data in, already exists, correct? You cannot try to have data go to a non existing index.
This is not the case. All matching stanzas in
props.conf are applied,
host::, and sourcetype, and if there are conflicts,
source:: generally wins, followed by
host::, and sourcetype is after those. setting
priority on a stanza can change this order. But that's not his problem. His problem is probably that
host:: is not what he thinks (and hopes) it is. It's probably just set to the indexer's default host.