Getting Data In
Highlighted

Configuring TimeStamp Format

Communicator

Hello everyone!

Currently, we're running into a problem configuring our timestamp for a source type. After browsing a few questions on Splunk Answers, we are still not able to correctly implement the timestamp.

When we leave the extraction on "Auto", it does not extract the timezone from our logs:

2018-05-23 08:49:59.266814 - Toaster 375790 - INFO - Boot up sequence begin
2018-05-23 08:46:23.836273 - Toaster 450623 - INFO - Loading: Network Lib

TIME_FORMAT =
%Y-%m-%d %H:%M:%S.%6N

We left the TIME_PREFIX empty because the timestamp is at the beginning of our log.

We have also attempted to use TIME_PREFIX = ^

Remaining Configuration:
MAXTIMESTAMPLOOKAHEAD = 50
NOBINARYCHECK = true
SHOULDLINEMERGE = true
TZ = America/New
York
category = Application
disabled = false
pulldown_type = true

Tags (2)
0 Karma
Highlighted

Re: Configuring TimeStamp Format

Ultra Champion

Can you provide some details on how you have deployed that configuration? What does the rest of your props.conf look like? On which instance is it deployed (and what is your architecture)?

Is that sourcetype the original sourecetype from inputs.conf, or are you overriding the sourcetype using transforms,conf?

0 Karma
Highlighted

Re: Configuring TimeStamp Format

Communicator

We are currently using the cloud and cannot access the props.conf. We have created our own source type named Kitchen Appliances.

Remaining Configuration:

MAXTIMESTAMPLOOKAHEAD = 1000
NOBINARYCHECK = true
SHOULDLINEMERGE = true
TZ = America/New
York
category = Application
disabled = false
pulldown_type = true

0 Karma
Highlighted

Re: Configuring TimeStamp Format

SplunkTrust
SplunkTrust

where is the timezone in the data?
your format looks ok,
can you share the full relevant props.conf?
will recommend to create a sourcetype regardless and not leave on auto

0 Karma
Highlighted

Re: Configuring TimeStamp Format

Communicator

We have actually already created the source type named "Kitchen Appliances". We have updated the post with the full props.conf that we can view from the cloud client.

0 Karma
Highlighted

Re: Configuring TimeStamp Format

SplunkTrust
SplunkTrust

You should include TIME_PREFIX=^ and you should change your MAX_TIMESTAMP_LOOKAHEAD to something more reasonable like 28 rather than 1000

0 Karma
Highlighted

Re: Configuring TimeStamp Format

Communicator

We have tried this as well. We just tried it again to make sure and it still did not yield the correct result.

0 Karma
Highlighted

Re: Configuring TimeStamp Format

SplunkTrust
SplunkTrust

The info I provided wont fix the TZ issue, but rather take some of the load off your indexers.. As for TZ, are the servers with the UF's installed in the same TZ as your indexers? What's your end goal? Have you tried change the TZ in the Splunk UI?

0 Karma
Highlighted

Re: Configuring TimeStamp Format

Communicator

Thanks for the response!

Right now, we are using the Splunk Cloud client as a proof of concept. We have not figured out a way to configure Splunk to pull the timestamp correctly from our log. We have changed the timezone in the UI to match the timezone from our log source.

The timestamp that shows on the Splunk search is the time that the log was received from the server, but not the time when the log was generated.

0 Karma
Highlighted

Re: Configuring TimeStamp Format

Communicator

Thank you to everyone who answered this question! I should have included more details, but I now know what the solution was. This event was received by the HTTP Event Collector. I was trying to parse the raw event that was sent.

By referencing this question, https://answers.splunk.com/answers/312321/can-we-process-the-timestamp-in-an-event-sent-to-t.html, you can see that, for time to be extracted properly, the "time" keyword must be placed in the post call.

View solution in original post

0 Karma