double host name and timestamps

[bbear]

Hi All,

I have been trying to get Splunk to strip off the timestamp and host of forwarded events but do not understand how to do this.

I have syslog-ng and Splunk on the same server. Syslog-ng is forwarding the events to udp port 1514 for Splunk to read and index. This all works fine. But the data Splunk receives has the timestamp and hostname of the syslog server in addition to the evendata host. I.E.

Jul 14 14:15:56 10.128.213.50 Jul 14 14:15:56 my-host-int02 snmpd[7777]: Received SNMP packet(s) from UDP: [10.128.30.20]:54900

Notice the additional timestamp above. I see in the docs and in the answers forum that this is possible to have Splunk strip this out before indexing, but I have not been able to get all the pieces worked out.

Does someone know the answer that has worked this out already? I would like Splunk to strip the event to look like:

Jul 14 14:15:56 my-host-int02 snmpd[7777]: Received SNMP packet(s) from UDP: [10.128.30.20]:54900

[gkanapathy · Accepted solution]

It would be best to avoid having the double added in the first place. In inputs.conf, under where you define the input port, add:

no_appending_timestamp = true

From inputs.conf.spec documentation file:

no_appending_timestamp = true
* If this attribute is set to true, then Splunk does NOT append a timestamp and host to received events.
* NOTE: Do NOT include this key if you want to append timestamp and host to received events.

[bbear]

I guess I should just keep trying. I figured out how to get it to work.

in $SPLUNK_HOME/etc/system/local create props.conf

props.conf

[syslog] TRANSFORMS = syslog-header-stripper-ts-host REPORT-syslog = syslog-extractions

In $SPLUNK_HOME/etc/system/local create transforms.conf

transforms.conf

[syslog-header-stripper-ts-host] REGEX = ^[A-Z][a-z]+\s+\d+\s\d+:\d+:\d+\s[^\s]\s(.)$ FORMAT = $1 DEST_KEY = _raw

Restart Splunk and it works

[ziegfried]

You could do this by modifying the raw event text with a SEDCMD your props.conf (eg. $SPLUNK_HOME/etc/system/local/props.conf)

[source::tcp:1514]
SEDCMD-remove-dbl-date=s/^[A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s+([A-Z][a-z]+\s+\d+\s+\d+:\d+:\d+\s+)/\1/g