Getting Data In

Why is Splunk indexing data for 12 hours instead of 24 hours?

gautamr103
New Member

After 12:59 PM slpunk is indexing data to 1:AM. It should index data for 24 hours but it is indexing for 12 hours only however 1:PM data are getting indexed in 1Am so I have two events in 1 am time stamp Below is my props.conf file.

DATETIME_CONFIG=CURRENT
NO_BINARY_CHECK = 1
pulldown_type = 1
TIME_FORMAT = %H:%M
TZ = US/Eastern
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE = \d\d:\d\d+\s*$
MAX_TIMESTAMP_LOOKAHEAD = 50
0 Karma

nikita_p
Contributor

Hi,
Can you give some sample of your time format. So if you also have seconds and milliseconds in your events, then you will have to change your TIME_FORMAT in props.conf as below:
TIME_FORMAT=%H:%M:%S.%N

0 Karma

eavent_splunk
Splunk Employee
Splunk Employee

Just to expand on previous comments - it is indexing for 24 hours, but the lack of AM/PM data is resulting in everything being in AM.

If you source data cannot be adjusted to include more time information, then as @thomast_splunk suggests one option would be to just use the whatever the current time and date is when splunk receives the event for processing.

DATETIME_CONFIG = NONE is another option:

  * "NONE" will leave the event time set to whatever time was selected by
    the input layer
    * For data sent by splunk forwarders over the splunk protocol, the input
      layer will be the time that was selected on the forwarder by its input
      behavior (as below).
    * For file-based inputs (monitor, batch) the time chosen will be the
      modification timestamp on the file being read.
    * For other inputs, the time chosen will be the current system time when
      the event is read from the pipe/socket/etc.

This page is a good primer on how Splunk assigns timestamps if you want more details:
https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/HowSplunkextractstimestamps#How_Splunk_softw...

0 Karma

thomast_splunk
Splunk Employee
Splunk Employee

May want to just use index time if in the same timezone - or keep that in mind for this particular sourcetype

Props.conf

[sourcetypeName]
DATETIME_CONFIG = CURRENT

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Propsconf?utm_source=answers&utm_medium=in...

0 Karma

micahkemp
Champion

What does your event text look like? If it includes AM/PM your TIME_FORMAT won't handle that.

If your event looks like:

03:45 PM

Your TIME_FORMAT would need to be:

TIME_FORMAT = %H:%M %p
0 Karma

gautamr103
New Member

right without AM and PM

0 Karma

gautamr103
New Member

Hi Micahkemp thank you for your reply

I dont have am pm on my event logs

this is my logs generated at 12:01 AM---> 12:01 Info [tasks_advancemedia_aspx]
and this was generated at 12:01 PM --> 12:01 Info [WorkerService] RTAEncode acm status

Also I wanted to break event according to time I have another log at same time ---> 12:01 Error [lambda_method] Unable how would I break event with same time but different logs, I tried BREAK_ONLY_BEFORE = ^\d\d:\d\d+\s but it did not work.

0 Karma

micahkemp
Champion

So your events look like:

12:01     <-- 12:01 AM
01:01     <-- 1:01 AM
...
12:01    <-- 12:01 PM
01:01    <-- 1:01 PM

?

Which would mean you don't have AM/PM or 24-hour format. That sounds less than ideal to say the least.

0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Hi Splunky people! We are excited to share the newest updates in Splunk Enterprise 9.3!Admins and Analyst can ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...