Getting Data In

Timestamp extraction problem, not pulling timestamp from field

markucsb
Explorer

I have the following events that I am trying to pull the timestamp out of the Time field, seems pretty straightforward however I am unable to get it working properly. The data is coming in over the http-simple REST endpoint and the data looks like the following :
Time=03-18-2014 18:25:04.775 UTC, ip=0.0.0.0, MajVer=0, MinVer=0, BuildNum=69180, UAModel=BR100, SnsId=0000000, HHId=Sonos_000000000, SN=00-00-00-00-00-00:G, Model=000, Ver=2, fq=2462, phyErr=1126, latThreshold=50, pktsAboveThreshold=0, pktsBelowThreshold=1082, link0=[ mac=000E58762555, tx=299, pktErrRate=1, sigStrength=30],

And the props.conf looks like this:

[source::http-simple]

TZ = UTC

TIME_PREFIX = ^Time=

TIME_FORMAT = %m-%d-%y%t%H:%M:%S.%3N

This is not working correctly and the indexing time is used instead of the event's Time field.

Tags (1)
0 Karma

linu1988
Champion

try this

NO_BINARY_CHECK=1
TIME_FORMAT=%m-%d-%Y %H:%M:%S.%3N %Z
TIME_PREFIX==

linu1988
Champion

what exactly are you getting there then? where is you props.conf placed? in indexer or search head? You need to place in indexer and restart splunk then log new events , this will affect only new logs

0 Karma

markucsb
Explorer

Still not working, any other ideas?

0 Karma

linu1988
Champion

it is not a mandatory field. But Time_prefix only indicate where the time starts. So we need not go to complexity where it only follows "=" sign. So it should work perfectly, right? For more you can refer props.conf specs in docs.splunk.com

markucsb
Explorer

I don't see why I'd need the NO_BINARY_CHECK, can you explain that a little bit more please. And for the TIME_PREFIX why would I make it less explicit?

0 Karma

lukejadamec
Super Champion

Try:

TIME_FORMAT=%m-%d-%Y %H:%M:%S.%3N %Z

0 Karma

lukejadamec
Super Champion

For these index time configs Splunk will look in etc/system/local on the indexer or forwarder. Indexer configs will take priority over the forwarder configs if the forwarder is Not a heavy forwarder. Placing in the app folder can work, but only if there are no conflicting configs in the system local folder. The best place is on the indexer, if there are no heavy forwarders involved. That way you can manage them from one place. Like linu1998 said, the system that holds the configs must be restarted, and it will only affect new events.

0 Karma

markucsb
Explorer

Events are broken correctly, the thing I pasted above is from the _raw field, however I went in and changed the numbers so I'm not showing real people's data, the events are being broken up correctly. This can be in a props.conf under an app correct? I don't need to put this in a props.conf at the $SPLUNK_HOME\etc\system\local\ level right?

0 Karma

lukejadamec
Super Champion

There was an error in the second field, I fixed it.
These are pretty much all the same answer. The basic problem with your time format was a y instead of a Y.
Are your events getting broken correctly?
Can you post a 'source=yoursource |table _raw` example for this source?

0 Karma

markucsb
Explorer

Tried this one, it is not working still.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Use '%Y' for 4-digit years. Also '%t' is not a time_format meta-character.

---
If this reply helps you, Karma would be appreciated.
0 Karma

markucsb
Explorer

I thought it says to use %t for all white space characters, but I will make the changes.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...