TIME not getting milliseconds

[MERBAG]

Hi all,

I do have a log which does look like this:

Jul  6 09:31:18.729: %SYS-5-CONFIG_I: Configured from console by username on vty1 (ip-address)

This data is received by syslog, but for any reason the internal _time variable does not contain the milliseconds, in this example .729 the _time variable in splunk does look like this: 2018-07-06T09:31:18.000+02:00

So I did some research and started to edit the props.conf for this sourcetyp

TIME_PREFIX=^
TIME_FORMAT=%b %d %H:%M:%S.%3N

But for any reason this did not impact the _time variable in splunk, can someone may tell me what I do wrong here?

Thanks a lot.

[MERBAG]

Does someone have any other ideas?

[MERBAG]

@poete , @FrankVI
Since there came up to similar questions, I want to explain a bit more how the setup does look like. To be honest it is the first time I am doing this, so my answer to the question "do you modify the right props.conf file?" is: I think so, but I do not know 🙂

We did create a new app, in the app the inputs.conf does look like this:

[udp://515]
connection_host = dns
index = cisco
sourcetype = switch

in the same app the props.conf does look like this:

[switch]
category = Network & Security
TIME_PREFIX=^
TIME_FORMAT=%b %e %H:%M:%S.%3Q
pulldown_type = 1
EXTRACT-ciscoaudit = ^(?:[^ \n]* ){4}(?P<hostname>[^ ]+)(?:[^ \n]* ){2}(?P<iostime>\w+\s+\d+\s+\d+:\d+:\d+\.\d+)[^%\n]*%(?P<facility>\w+)\-(?P<severity>[^\-]+)\-(?P<mnemonic>[^:]+):\s+\w+:(?P<user>[a-z]+)[^:\n]*:(?P<command>.+)
EXTRACT-cisco = ^(?:[^ \n]* ){4}(?P<hostname>[^ ]+)[^:\n]*:\s+(?P<iostime>\w+\s+\d+\s+\d+:\d+:\d+\.\d+)[^%\n]*%(?P<facility>\w+)\-(?P<severity>[^\-]+)\-(?P<mnemonic>\w+)[^ \n]* (?P<message>.+)

I also already tried to remove the EXTRACT-* part from the props.conf since I though the may have any negativ impact, but that also did not change anything.

[poete]

@MERBAG,

depending if you are running a single instance of Splunk, or an instance with separate indexer and search head, the steps for the props.conf to be taken into account are different.

If you are running on a single instance, in what directory is located the props.conf file you modified?

[MERBAG]

It is a single instance, the directory for the inputs and props file I posted here is

C:\Program Files\Splunk\etc\apps\merbag_it\local

[FrankVl]

Have you restarted Splunk after making the adjustments?

[MERBAG]

yes, after every change I did a restart uf splunk and generated new logs to test

[poete]

Hello @MERBAG,

it seems that the fomat for the millisesonds is not OK.

According to the documentation (https://docs.splunk.com/Documentation/Splunk/7.1.1/SearchReference/Commontimeformatvariables),
replace %3Nwith %3Q

[FrankVl]

I've always used %3N for that. Documentation is a bit confusing in that sense, I don't understand the difference between %xN and %xQ:

%N  Subseconds with width. (%3N = milliseconds, %6N = microseconds, %9N = nanoseconds)
 %Q The subsecond component of 2017-11-30 23:59:59.999 UTC. %3Q = milliseconds, with values of 000-999. %6Q = microseconds, with values of 000000-999999. %9Q = nanoseconds, with values of 000000000-999999999.

[MERBAG]

Hey,

also just tried that, still did not change anything, time in the log does look like this:

Jul 6 10:30:21.987

the _time variable now contains this:

2018-07-06T10:30:22.000+02:00

[poete]

| makeresults
| eval _time=strptime("Jul  6 09:31:18.729: %SYS-5-CONFIG_I: Configured from console by username on vty1 (ip-address)","%b %e %H:%M:%S.%3Q")

works like a charm. Are you sure you are modifying the right props.conf file?

[poete]

Well, that's good to know. I always used %Q up to now, and it always worked so far. May be worth a try

[FrankVl]

From how I read the docs they would be the same anyway, so not surprising %Q worked for you. Why they have 2 ways of doing the same is beyond me 🙂

[FrankVl]

Since the day of month is preceded with a space when single digit instead of 06, you need to use %e instead of %d.

[MERBAG]

Thanks for the quick feedback. I just did the change you said, but unfortunatly that didnt help - the _time does still look like this:

2018-07-06T10:25:34.000+02:00   

The props.conf now looks like this:

TIME_PREFIX=^
TIME_FORMAT=%b %e %H:%M:%S.%3N

[FrankVl]

In that case, issue may be with how you've set up your props.conf and where you deployed it.

Can you show a bit more of the config files used for this data input? You set the time config based on sourcetype, right? Is that sourcetype determined in inputs.conf, or is it overridden using props and transforms?
What does your architecture look like and where in the architecture did you deploy the time config props.conf?