Syslog-ng logs show as host=localServer, rather than remote?

[norfleetj]

Hello,

System type: Linux

We have splunk running on our centralized syslog-ng server. We then have other servers forwarding syslog traffic to it. Those logs are then stored in their own folder based on their hostname (i.e. /var/log/syslog-ng/remoteHost/logfile.)

We have splunk setup to see the syslog-ng folder and it reads everything fine. But in splunk, the output of all the logs say host=localServerName, what I would like them to do is say host=remoteServerName, is this possible?

Thanks in advance for any suggestions.

[mayler]

We have this working so let's start with your Data Inputs. How are they configured? Syslog by port or Files and Directories? What version of splunk are you running?

I am using syslog-ng as well, splunk 4.1, using UDP 514 as my data input. Set host is DNS, Sourcetype is Manual. If that all looks the same, have you made any changes to your syslog-ng.conf file?

If you've made a change to your syslog-ng file, Global Options should be:

options {
sync (0);
time_reopen (10);
log_fifo_size (1000);
long_hostnames (off);
use_dns (yes);
use_fqdn (yes);
use_time_recvd (yes);
create_dirs (yes);
keep_hostname (yes);
};

[Dan · Accepted solution]

You can configure Splunk to assign the hostname in a few different ways. They're all documented here: http://docs.splunk.com/Documentation/Splunk/5.0/Data/Aboutdefaultfields

In your case it makes sense to specify a host for the input looking at the directory, and to use a segment in path value of 4.

How Splunk assigns the host value

If no other host rules are specified for a source, Splunk assigns host a default value that applies to all data coming from inputs on a given Splunk server. The default host value is the hostname or IP address of the network host. When Splunk is running on the server where the event occurred (which is the most common case) this is correct and no manual intervention is required.

For more information, see "Set a default host for a Splunk server" in this manual.

Set a default host for a file or

directory input

If you are running Splunk on a central log archive, or you are working with files forwarded from other hosts in your environment, you may need to override the default host assignment for events coming from particular inputs.

There are two methods for assigning a host value to data received through a particular input. You can define a static host value for all data coming through a specific input, or you can have Splunk dynamically assign a host value to a portion of the path or filename of the source. The latter method can be helpful when you have a directory structure that segregates each host's log archive in a different subdirectory.

For more information, see "Set a default host for a file or directory input" in this manual.

Override default host values based on event data

You may have a situation that requires you to override host values based on event data. For example, if you work in a centralized log server environment, you may have several host servers that feed into that main log server. The central log server is called the reporting host. The system where the event occurred is called the originating host (or just the host). In these cases you need to define rules that override the automatic host assignments for events received from that centralized log host and replace them with distinct originating host values.

For more information, see "Override default host values based on event data" in this manual.

Tag host values

Tag host values to aid in the execution of robust searches. Tags enable you to cluster groups of hosts into useful, searchable categories.

[norfleetj]

Thanks for the direct link to instructions. The Regex, based on segment in path, worked perfect.

Our environment is dynamic in nature. The syslog folder is archived and flushed nightly. What is in there today, might not be in there tomorrow, and vice-versa. This is exactly what I needed.

Thanks again.

[the_wolverine]

You should use the host_segment setting in your inputs.conf. It looks like this might be the same issue addressed in the following answers post.