Getting Data In
Highlighted

How do I add the right time stamp and correct server name to the following syslog messages?

New Member

Hi guys,

Please pardon my ignorance here as i am new to Splunk. I am using Splunk 7.1 on a Windows server and forwarding syslog messages from Linux server (j01ftc) , now as you can see below example

<0>Oct 18 14:49:51 j01ftc Oct 19 02:02:23 akl-ftc-sbc3b.vfnz-ipsn akl-ftc-sbc3b sipd05[4f] ERROR could not identify psipcontact

There are 2 issues. One is that the wrong time stamp is in beginning of the message and 2nd server name added , i want to exclude both of these from my logs in Splunk or at least fix the time in the event.

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Contributor

You can overwrite those using props.conf and transform.conf

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

New Member

would you be able to point me to ,how to remove "Oct 18 14:49:51 j01ftc " from my log messages

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Motivator

Hi @saadi381,

That is the syslog header added by syslog server for every event.

If you want to extract the second timestamp (Oct 19 02:02:23), configure the props.conf as below:

[sourcetype]
TIME_PREFIX = j01ftc\s
TIME_FORMAT = %b %d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 15

If you want to completely remove the syslog header from events then, configure the props.conf as below:

    [sourcetype]
    SEDCMD-dropSyslogHeader = s/(^[\w\s\:]+j01ftc\s)//g
    TIME_PREFIX = ^
    TIME_FORMAT = %b %d %H:%M:%S
    MAX_TIMESTAMP_LOOKAHEAD = 15

I would suggest using 1st props.conf as it is much simpler. HTH!

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

New Member

under D:\Splunk\etc\apps\search\local props.conf

[saadtest]
NO
BINARYCHECK = true
TZ = Antarctica/South
Pole
category = Custom
pulldowntype = 1
disabled = false
SHOULD
LINEMERGE = false
SEDCMD-dropSyslogHeader = s/(^[\w\s:]+j01ftc\s)//g
TIMEPREFIX = ^
MAX
TIMESTAMPLOOKAHEAD = 15
INDEXED
EXTRACTIONS =
DATETIME_CONFIG =

but still no luck 😞

looking like

<0>Oct 18 23:38:53 j01ftc Oct 19 12:38:46 akl-ftc-sbc3b.ipsn akl-ftc-sbc3b sipd01[4b] ERROR could not identify psipcontact

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Motivator

Few questions please,
Can you tell me if <0> is part of the event or is it something that got added while posting the question?
Can you tell more about your architecture?

UF (on syslog server) --> Indexer then, these configs should be on indexer.
HF (on syslog server) --> Indexer then, these configs should be on HF.

Make sure that these configuration are present on parsing layer (HF and Indexer).

Finally, did you try the 1st props.conf. It will fix the timestamp but won't discard syslog header.

Also, as @FrankVl suggested, you should have a look at syslog configuration to troubleshoot further.

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Path Finder

sudosplunk's answer is the solution that worked for me using your example string:
(Oct 18 23:38:53 j01ftc Oct 19 12:38:46 akl-ftc-sbc3b.ipsn akl-ftc-sbc3b sipd01[4b] ERROR could not identify psipcontact)

[sourcetype]
TIMEPREFIX = j01ftc\s
TIME
FORMAT = %b %d %H:%M:%S
MAXTIMESTAMPLOOKAHEAD = 15

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Ultra Champion

I would very much suggest fixing this on syslog side, instead of pulling all kinds of tricks on Splunk side to fix problems introduced by the syslog setup.

What does your syslog setup look like? Because the duplicate headers seem to indicate there is some intermediate syslog server involved or so? And also: how exactly are you feeding this into Splunk?

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

New Member

The feed is like
device-->(tcp514)syslog server -->TCP9998 splunk.

i highly doubt the duplicate time stamp is added by Syslog server , i believe its added on splunk server

0 Karma
Highlighted

Re: How do I add the right time stamp and correct server name to the following syslog messages?

Ultra Champion

Unless you specifically configured Splunk to add that header, I don't think so. Splunk only does that by default for UDP inputs and even then, it does not add that <xx> part, only timestamp and sending host.

You can validate that by running a networkcapture (e.g. using tcpdump) on either the outgoing traffic of the syslog server, or the incoming traffic on the splunk server. If the header is there already, it isn't Splunk who is adding it.

Feel free to share your relevant Splunk configs (inputs, possible props/transforms) and syslog daemon configs here for further troubleshooting.

0 Karma