Getting Data In

How to configure int64 epoch nanosecond timestamp as _time

tincupchalice
Path Finder

So I tried pattern as \d{18} for events looking like:



1351623403000225565 Type=VARIABLE, blah blah

1351623403000225568 Type=VARIABLE, blah blah

1351623403000225570 Type=VARIABLE, blah blah




when trying to add a data file from the manager, but splunk will not recognize this as a timestamp.

I've also tried ^\d, ^\d{18}.


With \d{18}, it generates a props.conf:


# your settings

NO_BINARY_CHECK=1

SHOULD_LINEMERGE=false

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%9N

TIME_PREFIX=\d{18}

TZ=America/Chicago

0 Karma
1 Solution

sowings
Splunk Employee
Splunk Employee

TIME_PREFIX is to indicate any static (or even variable) text which might appear before the time stamp that exists.

TIME_FORMAT indicates the format that the stamp takes. Your config, cited above, would skip 18 (or 19) digits, then look for a stamp like 2012-11-07 21:44:23.623560235. That's not what you've got. You'll want to capture the first 10 digits as an epoch time, followed by a nanosecond.

TIME_FORMAT=%s%9N

TIME_PREFIX=^

View solution in original post

0 Karma

sowings
Splunk Employee
Splunk Employee

TIME_PREFIX is to indicate any static (or even variable) text which might appear before the time stamp that exists.

TIME_FORMAT indicates the format that the stamp takes. Your config, cited above, would skip 18 (or 19) digits, then look for a stamp like 2012-11-07 21:44:23.623560235. That's not what you've got. You'll want to capture the first 10 digits as an epoch time, followed by a nanosecond.

TIME_FORMAT=%s%9N

TIME_PREFIX=^

0 Karma

tincupchalice
Path Finder

however what is the point in using a nanosecond timestamp for _time if I cant get the precision out of Splunk???

sourcetype=OrderGW-RJ | eval tm=_time | eval stm=strftime(_time, "%Y-%m-%d %H:%M:%S.%9N") | stats count by _time tm stm | head 1 | table _time tm stm

_time tm stm
10/30/12 1:56:43.000 PM 1351623403.000225 2012-10-30 13:56:43.000225000

0 Karma

tincupchalice
Path Finder

That worked, thanks for the clarification.

0 Karma

tincupchalice
Path Finder

should be {19}, but still not working...

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...