Milliseconds timestamp for syslog udp 514 is not working with no_appending_timestamp=false (default)

[deodion]

May 25 15:21:44 192.168.1.1 2097826: T2-D1-BDS LC/0/0/CPU0:May 25 15:21:44.362 GMT: npu_driver[273]: %L2-PLIM_ETHER-2-RX_LF : Interface HundredGigE0/0/0/3, Detected Local Fault
May 11 20:37:50 192.168.1.2 38897734: May 11 20:37:49 WIB: %GT96K_FE-5-LATECOLL: Late Collision on int FastEthernet0/0
May 11 20:37:50 192.168.1.3 196640143: May 11 20:37:50.789 WIB: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/22, changed state to down
May 11 20:37:50 192.168.1.4 196640142: May 11 20:37:48.868 WIB: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface GigabitEthernet3/22, changed state to up
May 11 20:37:50 192.168.1.5 38897733: May 11 20:37:48 WIB: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
May 11 20:37:50 192.168.1.6 34430720: 30340540: May 11 20:37:49.622 WIB: %BGP-5-ADJCHANGE: neighbor 192.168.1.1 active Down BGP Notification sent
May 11 20:37:50 192.168.1.7 14180869: .May 11 20:02:22.000 WIB: %CDP-4-NATIVE_VLAN_MISMATCH: Native VLAN mismatch discovered on GigabitEthernet1/33 (3600), with SNMPTN-JKT FastEthernet0/5 (707).

I have event like above,
Trying to get milliseconds indexed,

Tried 1:
props.conf
[mysourcetype]
TIME_PREFIX = ^.*\d{4}:\s
TIME_FORMAT = %b %d %H:%M:%S.%3N

[source::udp:514]

mysourcetype comes from this udp 514 ....

index = my-index
queueSize = 50MB
persistentQueueSize = 100MB

no_appending_timestamp = true

Tried 2:
props.conf
[mysourcetype]
TRANSFORMS-set_time = set-time

transforms.conf
[set-time]
SOURCE_KEY = _raw
REGEX = (\d{2}:\d{2}:\d{2}.\d{3})
DEST_KEY = _time

Tried 3:
Using command: splunk train dates to get the custom datetime code.
props.conf
[mysourcetype]
DATETIME_CONFIG = $SPLUNK_HOME/etc/system/local/datetime_custom.xml

datetime_custom.xml

<!--   Version 4.0 -->

<!-- datetime.xml -->
<!-- This file contains the general formulas for parsing date/time formats. -->

<datetime>

<define name="sample-cisco_1_date" extract="litmonth,day,">
        <text><![CDATA[\d+:\s(\w+)\s(\d+)]]></text>
</define>
<define name="sample-cisco_1_time" extract="hour,minute,second,subsecond,zone,">
        <text><![CDATA[\w+\s\d+\s(\d+):(\d+):(\d+)\.(\d+)\s(\w+)]]></text>
</define>

<define name="sample-cisco_4_time" extract="hour,minute,second,zone,">
        <text><![CDATA[\w+\s\d+\s(\d+):(\d+):(\d+)\s(\w+)]]></text>
</define>


<timePatterns>
      <use name="sample-cisco_1_time"/>
      <use name="sample-cisco_4_time"/>
</timePatterns>

<datePatterns>
      <use name="sample-cisco_1_date"/> 
</datePatterns>

</datetime>

None of above tries are working.

but i see I can do in search-time:

sourcetype=mysourcetype
| rex field=_raw "^.*\d{4}:(\s|\s\.|\.|)*?(?.*):\s"
| eval _time = strptime(TIME, "%b %d %H:%M:%S.%3N")

It looks like strptime working as expected, but that is search time, I need to make it index time parsing.

Now I'm getting feeling is this because of no_appending_timestamp=false in default inputs.conf?
but that is default behavior, if I change it it is working, but will make other sourcetype parsing a mess.
with no_appending_timestamp=true, there is no additional timestamp at the leftside.

Any idea?

Thank you!

[darrenfuller]

Hi There.. what about fixing the timestamp of the events that are missing the milliseconds using transforms to detect the space after seconds... then append .000 to it.

props.conf

[test_date_fix]
disabled = false
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3n
MAX_TIMESTAMP_LOOKAHEAD = 30
TRUNCATE = 10000

TRANSFORMS-01-FIXTIMESTAMP = fix_timestamp_missing_miliseconds

transforms.conf

[fix_timestamp_missing_miliseconds]
disabled = false
SOURCE_KEY = _raw
DEST_KEY = _raw
REGEX = ^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})(\s.+)
FORMAT = $1.000$2

C:\opt\test.log

2018-05-30 00:00:00.007 INFO Platea elementum vivamus scelerisque curabitur te
2018-05-30 00:00:00.144 INFO Nulla tellus blandit erat semper turpis aliquet m
2018-05-30 00:00:00.277 INFO Ipsum ultrices cras dictumst aenean proin, litora
2018-05-30 00:00:00.301 INFO At nam tempor nec quis sem augue placerat suspend
2018-05-30 00:00:00.560 INFO Nec massa auctor nibh tellus conubia a lacus, nam
2018-05-30 00:00:00.701 INFO id pulvinar tempus massa cubilia curabitur at pri
2018-05-30 00:00:00.801 INFO Platea elementum vivamus scelerisque curabitur te
2018-05-30 00:00:00 INFO Nulla tellus blandit erat semper turpis aliquet m
2018-05-30 00:00:01.008 INFO Ipsum ultrices cras dictumst aenean proin, litora
2018-05-30 00:00:01.012 INFO At nam tempor nec quis sem augue placerat suspend
2018-05-30 00:00:01.210 INFO Nec massa auctor nibh tellus conubia a lacus, nam
2018-05-30 00:00:01.240 INFO id pulvinar tempus massa cubilia curabitur at pri
2018-05-30 00:00:01.360 INFO Platea elementum vivamus scelerisque curabitur te
2018-05-30 00:00:01.373 INFO Nulla tellus blandit erat semper turpis aliquet m
2018-05-30 00:00:01 INFO Ipsum ultrices cras dictumst aenean proin, litora

search: index=main sourcetype=test_date_fix | table _raw

_raw

2018-05-30 00:00:01.000 INFO Ipsum ultrices cras dictumst aenean proin, li
2018-05-30 00:00:01.373 INFO Nulla tellus blandit erat semper turpis aliquet m
2018-05-30 00:00:01.360 INFO Platea elementum vivamus scelerisque curabitur te
2018-05-30 00:00:01.240 INFO id pulvinar tempus massa cubilia curabitur at pri
2018-05-30 00:00:01.210 INFO Nec massa auctor nibh tellus conubia a lacus, nam
2018-05-30 00:00:01.012 INFO At nam tempor nec quis sem augue placerat suspend
2018-05-30 00:00:01.008 INFO Ipsum ultrices cras dictumst aenean proin, litora
2018-05-30 00:00:00.000 INFO Nulla tellus blandit erat semper turpis aliqu
2018-05-30 00:00:00.801 INFO Platea elementum vivamus scelerisque curabitur te
2018-05-30 00:00:00.701 INFO id pulvinar tempus massa cubilia curabitur at pri
2018-05-30 00:00:00.560 INFO Nec massa auctor nibh tellus conubia a lacus, nam
2018-05-30 00:00:00.301 INFO At nam tempor nec quis sem augue placerat suspend
2018-05-30 00:00:00.277 INFO Ipsum ultrices cras dictumst aenean proin, litora
2018-05-30 00:00:00.144 INFO Nulla tellus blandit erat semper turpis aliquet m
2018-05-30 00:00:00.007 INFO Platea elementum vivamus scelerisque curabitur te