Getting Data In

datetime.xml problem with a word "hour"

davecroto
Splunk Employee
Splunk Employee

I have a non - standard, Adobe / Omniture log standard timestamp that I want to extract. The value after the word Hour is the actual hour of the day in military time so 0 is between 12 and 1am...and so on and so forth til hour 23. Don't care about minutes or seconds.

Error logs are not spitting out any useful info, but it is not pulling out the timestamp.

Example 1: "August 13, 2013", Hour 0 foobar:A 1

Example 2: "August 13, 2013", Hour 1 300:A general

here is my complete datetime.xml


<![CDATA[\"(\w+)\s+(\d{1,2})\,\s+(\d{4})\,\s+Hour\s+(\d{1,2})\"\,\s+]>







Why is this not working? I hard coded the actual Month "August" to test make sure it was not my regex, but timestamp is still wrong.

Here is my props.conf:

[timestamp]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD = 2
DATETIME_CONFIG = /etc/system/local/datetime.xml

TIME_FORMAT = %B %d, %Y, Hour %H doesn't like the hour either.

0 Karma

Jon_Webster
Splunk Employee
Splunk Employee

I don't think we need datetime.xml for this. I think the "Hour" string needs to be specified just as you are.

I'd open a support case. As you can see, the H M gets recognized, while just the H alone does not.

Interestingly, this:

your settings

MAX_TIMESTAMP_LOOKAHEAD=50
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TIME_FORMAT= %B %d, %Y, Hour %H %M

Does get recognized, but without the %M, the %H does not get recognized:

With %M:

Timestamp
Event

1 8/13/13 12:30:00.000 AM
August 13, 2013, Hour 0 300:A general ]

2 8/13/13 12:30:00.000 AM August 13, 2013, Hour 0 300:A general ]

3 8/13/13 12:30:00.000 AM
August 13, 2013, Hour 0 300:A general ]

4 8/13/13 12:30:00.000 AM
August 13, 2013, Hour 0 300:A general ]

5 8/13/13 12:30:00.000 AM
August 13, 2013, Hour 0 300:A general ]

6 8/13/13 1:30:00.000 AM
August 13, 2013, Hour 1 300:A general ]

7 8/13/13 1:30:00.000 AM
August 13, 2013, Hour 1 300:A general ]

8 8/13/13 1:30:00.000 AM
August 13, 2013, Hour 1 300:A general ]

9 8/13/13 1:30:00.000 AM
August 13, 2013, Hour 1 300:A general ]

10 8/13/13 1:30:00.000 AM
August 13, 2013, Hour 1 300:A general ]

11 8/13/13 2:30:00.000 AM
August 13, 2013, Hour 2 300:A general ]

Without %M:

12 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 2 300:A general ]

13 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 2 300:A general ]

14 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 2 300:A general ]

18 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 9 300:A general ]

19 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 9 300:A general ]

20 8/13/13 12:00:00.000 AM
August 13, 2013, Hour 9 300:A general ]

0 Karma

davecroto
Splunk Employee
Splunk Employee

Sorry Jon, This did not work

The sample log does not always have a %M for example:

August 13, 2013, Hour 1 foobar:A general

I will be opening a case.

0 Karma

tgow
Splunk Employee
Splunk Employee

Instead of having the datetime.xml file in the /etc/system/default/ directory I am wondering if you put it in the local directory and it will work:

DATETIME_CONFIG = /etc/system/local/datetime.xml

0 Karma

davecroto
Splunk Employee
Splunk Employee

Thanks Tgow, but I tried it in both.

0 Karma

davecroto
Splunk Employee
Splunk Employee

added SHOULD_LINEMERGE = false and it split it up, but still not the right timestamp

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...