Getting Data In
Highlighted

Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Contributor

Hi

I am receiving a syslog feed from a server.I am trying to index that data. In syslog feed no milliseconds are being sent in feed.So event datetiime is of format 8/31/12 11:44:18.000 AM
i.e milliseconds is always .000 for all events indexed.

So while calculating duration after using transaction command if transaction happens in less than a second then the duration shows as O, which is claerly wrong.
So is there a way i can append current date time with milliseconds when getting indexed, so that i get proper milliseconds feild also.

Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Influencer

As far as I'm aware, this would be something that you have to try and configure at the source (i.e. the syslog source), as adding milliseconds at Splunk's end would give inaccurate values as it would clearly not be the same time (even if you had the worlds fastest networks and systems it would still be different).

It might be better to look at the data source for options, perhaps this is something that can configured if you ask the questions to the vendor or those in the know with the product. If not a possible solution for having some difference at the millisecond level...

You could possible write the syslog to file and have a date inserted into the file before/after each log. Although this would probably be different to the actual syslog time, it may give you the accurate difference... you could then use this as you index time, and extract the actual time as field for additional information when troubleshooting, etc.

Just some thoughts,

MHibbin

Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Contributor

Hi MHibbin,

Thnx for that. It is confirmed at datasource that milliseconds cannot be sent in syslog.

So only other should be writing it in file, and then reading it.

But is there anyway i could append the time of receipt at indexer as the event time, instaed of writing and reading from file.

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Influencer

So you are indexing syslog straight to Splunk (i.e. allow Splunk to read on UDP 514)?

Perhaps try looking at props.conf for your source/sourctype/host (whichever is more specific to your syslog data) use that as a stanza in props.conf and then use the "DATETIMECONFIG = CURRENT" parameter, this may force that data type to index with the system time when being indexed. You can also use the "TIMEFORMAT" option to configure specific time format. See the link below for reference and examples.

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Influencer

I would add that if you are monitoring that port directly with Splunk, best practise is to record to file first... ref: http://wiki.splunk.com/Community:Best_Practice_For_Configuring_Syslog_Input

NOTE Option 1

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Ultra Champion

Damn! I realize that I'm more than a little bit late... spend too much time editing. 😞

Well, at least it seems that we have the same thoughts.

/k

Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Influencer

Well, your answer has better content... I didn't know about the _indextime field. So I've learnt something new... so I'll upvote for that!

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Contributor

Hi MHibbin,
Using TIME_FORMAT hw can i pick milliseconds, can you please elaborate on that...

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Influencer

Apologies I thought you could use milliseconds in strptime

0 Karma
Highlighted

Re: Is there anyway to append current time along with milliseconds to incoming syslog feed to indexer

Motivator

You can use milliseconds in TIMEFORMAT - use %xN, where x is the number of digits of precision below second level. For example, %H:%M:%S.%3N for 10:23:45.447 - use of TIMEFORMAT of course requires the data to be in your event to begin with.

0 Karma