Getting Data In

TIME not getting milliseconds

Explorer

Hi all,

I do have a log which does look like this:

Jul  6 09:31:18.729: %SYS-5-CONFIG_I: Configured from console by username on vty1 (ip-address)

This data is received by syslog, but for any reason the internal _time variable does not contain the milliseconds, in this example .729 the _time variable in splunk does look like this: 2018-07-06T09:31:18.000+02:00

So I did some research and started to edit the props.conf for this sourcetyp

TIME_PREFIX=^
TIME_FORMAT=%b %d %H:%M:%S.%3N

But for any reason this did not impact the _time variable in splunk, can someone may tell me what I do wrong here?

Thanks a lot.

0 Karma

Explorer

Does someone have any other ideas?

0 Karma

Explorer

@poete , @FrankVI
Since there came up to similar questions, I want to explain a bit more how the setup does look like. To be honest it is the first time I am doing this, so my answer to the question "do you modify the right props.conf file?" is: I think so, but I do not know 🙂

We did create a new app, in the app the inputs.conf does look like this:

[udp://515]
connection_host = dns
index = cisco
sourcetype = switch

in the same app the props.conf does look like this:

[switch]
category = Network & Security
TIME_PREFIX=^
TIME_FORMAT=%b %e %H:%M:%S.%3Q
pulldown_type = 1
EXTRACT-ciscoaudit = ^(?:[^ \n]* ){4}(?P<hostname>[^ ]+)(?:[^ \n]* ){2}(?P<iostime>\w+\s+\d+\s+\d+:\d+:\d+\.\d+)[^%\n]*%(?P<facility>\w+)\-(?P<severity>[^\-]+)\-(?P<mnemonic>[^:]+):\s+\w+:(?P<user>[a-z]+)[^:\n]*:(?P<command>.+)
EXTRACT-cisco = ^(?:[^ \n]* ){4}(?P<hostname>[^ ]+)[^:\n]*:\s+(?P<iostime>\w+\s+\d+\s+\d+:\d+:\d+\.\d+)[^%\n]*%(?P<facility>\w+)\-(?P<severity>[^\-]+)\-(?P<mnemonic>\w+)[^ \n]* (?P<message>.+)

I also already tried to remove the EXTRACT-* part from the props.conf since I though the may have any negativ impact, but that also did not change anything.

0 Karma

Builder

@MERBAG,

depending if you are running a single instance of Splunk, or an instance with separate indexer and search head, the steps for the props.conf to be taken into account are different.

If you are running on a single instance, in what directory is located the props.conf file you modified?

0 Karma

Explorer

It is a single instance, the directory for the inputs and props file I posted here is

C:\Program Files\Splunk\etc\apps\merbag_it\local

0 Karma

Ultra Champion

Have you restarted Splunk after making the adjustments?

0 Karma

Explorer

yes, after every change I did a restart uf splunk and generated new logs to test

0 Karma

Builder

Hello @MERBAG,

it seems that the fomat for the millisesonds is not OK.

According to the documentation (https://docs.splunk.com/Documentation/Splunk/7.1.1/SearchReference/Commontimeformatvariables),
replace %3Nwith %3Q

0 Karma

Ultra Champion

I've always used %3N for that. Documentation is a bit confusing in that sense, I don't understand the difference between %xN and %xQ:

%N  Subseconds with width. (%3N = milliseconds, %6N = microseconds, %9N = nanoseconds)
 %Q The subsecond component of 2017-11-30 23:59:59.999 UTC. %3Q = milliseconds, with values of 000-999. %6Q = microseconds, with values of 000000-999999. %9Q = nanoseconds, with values of 000000000-999999999.
0 Karma

Explorer

Hey,

also just tried that, still did not change anything, time in the log does look like this:

Jul 6 10:30:21.987

the _time variable now contains this:

2018-07-06T10:30:22.000+02:00
0 Karma

Builder
| makeresults
| eval _time=strptime("Jul  6 09:31:18.729: %SYS-5-CONFIG_I: Configured from console by username on vty1 (ip-address)","%b %e %H:%M:%S.%3Q")

works like a charm. Are you sure you are modifying the right props.conf file?

0 Karma

Builder

Well, that's good to know. I always used %Q up to now, and it always worked so far. May be worth a try

0 Karma

Ultra Champion

From how I read the docs they would be the same anyway, so not surprising %Q worked for you. Why they have 2 ways of doing the same is beyond me 🙂

0 Karma

Ultra Champion

Since the day of month is preceded with a space when single digit instead of 06, you need to use %e instead of %d.

0 Karma

Explorer

Thanks for the quick feedback. I just did the change you said, but unfortunatly that didnt help - the _time does still look like this:

2018-07-06T10:25:34.000+02:00   

The props.conf now looks like this:

TIME_PREFIX=^
TIME_FORMAT=%b %e %H:%M:%S.%3N
0 Karma

Ultra Champion

In that case, issue may be with how you've set up your props.conf and where you deployed it.

Can you show a bit more of the config files used for this data input? You set the time config based on sourcetype, right? Is that sourcetype determined in inputs.conf, or is it overridden using props and transforms?
What does your architecture look like and where in the architecture did you deploy the time config props.conf?

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!