Getting Data In

Why is Splunk indexing data for 12 hours instead of 24 hours?

New Member

After 12:59 PM slpunk is indexing data to 1:AM. It should index data for 24 hours but it is indexing for 12 hours only however 1:PM data are getting indexed in 1Am so I have two events in 1 am time stamp Below is my props.conf file.

DATETIME_CONFIG=CURRENT
NO_BINARY_CHECK = 1
pulldown_type = 1
TIME_FORMAT = %H:%M
TZ = US/Eastern
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = \d\d:\d\d+\s*$
MAX_TIMESTAMP_LOOKAHEAD = 50
0 Karma

Contributor

Hi,
Can you give some sample of your time format. So if you also have seconds and milliseconds in your events, then you will have to change your TIMEFORMAT in props.conf as below:
TIME
FORMAT=%H:%M:%S.%N

0 Karma

Splunk Employee
Splunk Employee

Just to expand on previous comments - it is indexing for 24 hours, but the lack of AM/PM data is resulting in everything being in AM.

If you source data cannot be adjusted to include more time information, then as @thomast_splunk suggests one option would be to just use the whatever the current time and date is when splunk receives the event for processing.

DATETIME_CONFIG = NONE is another option:

  * "NONE" will leave the event time set to whatever time was selected by
    the input layer
    * For data sent by splunk forwarders over the splunk protocol, the input
      layer will be the time that was selected on the forwarder by its input
      behavior (as below).
    * For file-based inputs (monitor, batch) the time chosen will be the
      modification timestamp on the file being read.
    * For other inputs, the time chosen will be the current system time when
      the event is read from the pipe/socket/etc.

This page is a good primer on how Splunk assigns timestamps if you want more details:
https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/HowSplunkextractstimestamps#How_Splunk_softw...

0 Karma

Splunk Employee
Splunk Employee

May want to just use index time if in the same timezone - or keep that in mind for this particular sourcetype

Props.conf

[sourcetypeName]
DATETIME_CONFIG = CURRENT

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf?utm_source=answers&utm_medium=in...

0 Karma

Champion

What does your event text look like? If it includes AM/PM your TIME_FORMAT won't handle that.

If your event looks like:

03:45 PM

Your TIME_FORMAT would need to be:

TIME_FORMAT = %H:%M %p
0 Karma

New Member

right without AM and PM

0 Karma

New Member

Hi Micahkemp thank you for your reply

I dont have am pm on my event logs

this is my logs generated at 12:01 AM---> 12:01 Info [tasksadvancemediaaspx]
and this was generated at 12:01 PM --> 12:01 Info [WorkerService] RTAEncode acm status

Also I wanted to break event according to time I have another log at same time ---> 12:01 Error [lambdamethod] Unable how would I break event with same time but different logs, I tried BREAKONLY_BEFORE = ^\d\d:\d\d+\s but it did not work.

0 Karma

Champion

So your events look like:

12:01     <-- 12:01 AM
01:01     <-- 1:01 AM
...
12:01    <-- 12:01 PM
01:01    <-- 1:01 PM

?

Which would mean you don't have AM/PM or 24-hour format. That sounds less than ideal to say the least.

0 Karma