Getting Data In

Why is Splunk not parsing the correct datetime?

dpanych
Communicator

I have logs that contain the following datetime format:

29-06-2016_00-08-17

The props contain:

[odb]
TIME_PREFIX = ".+",".+","
TIME_FORMAT = %d-%m-%Y_%H-%M-%S
MAX_TIMESTAMP_LOOKAHEAD = 50
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
pulldown_type = true
SEDCMD-01_change_delims_in_oracle_logs = s/\|;\|/,/g
SEDCMD-02_remove_the_end = s/"\|\|\?--END---\?\|\|//g
REPORT-set_delimiters_oracle_logs = REPORT-delims_odb_logs

What could be the problem?

0 Karma
1 Solution

woodcock
Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

View solution in original post

woodcock
Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

woodcock
Esteemed Legend

Also, this needs to be deployed to your indexers, the splunk instances must be restarted, and only newly-indexed (post-restart) data will be effected (bad events will stay bad).

0 Karma

jplumsdaine22
Influencer

Does it make a difference if you use a capture group instead of a non capture group there? As in:

(?:"[^"]*",){3}"
0 Karma

gcusello
SplunkTrust
SplunkTrust

you have to use \ before " so the prefix is
\"\w+\",\"\w+\",\"\w+\",\"

Bye.
Giuseppe

0 Karma

woodcock
Esteemed Legend

In this case, they are the same.

0 Karma

dpanych
Communicator

I'm no regex guru, but how do I make the regex above work with the |;| delims? ---- "aaa"|;|"OIM"|;|"DELETE"|;|"29-06-2016_01-53-16"

0 Karma

jplumsdaine22
Influencer

Just modify the regex to pick the delimiters instead of a comma

 ("[^"]*"\|;\|){3}"

dpanych
Communicator

Thank you guys, this worked!

0 Karma

jplumsdaine22
Influencer

https://regex101.com/ is your friend

0 Karma

woodcock
Esteemed Legend

We cannot say without you posting a sample event.

0 Karma

dpanych
Communicator

Here's what the logs looks like after the sed:

"host1","MON","LOGOFF","30-06-2016_11-15-01","","0"
"host2","ODS","UPDATE","30-06-2016_12-51-05","UPDATE DS_ATTRSTORE SET ATTRVAL = :B1 WHERE ENTRY = :B2 AND ATTRNAME = 'modname'","0"
"host3","ODS","UPDATE","30-06-2016_08-28-43","UPDATE DS_ATTRSTORE SET ATTRVER = :B4 || CHR(94) || :B5 || CHR(94) || :B6 , ATTRVAL = :B3 , ATTRKIND = :B2 , ATTRSTYPE = :B1 WHERE ENTRY = :B8 AND ATTRNAME = :B7","0"
0 Karma

jplumsdaine22
Influencer

Can you post a sample event as well?

0 Karma

maciep
Champion

Not sure if I'm reading that correctly, but are you replacing the delimiters in the raw text from "|;|" to ","? If so, is that the same comma you're expecting to be there for the time prefix? If it is, I'm wondering if the timestamp recognition happens before sedcmd is applied?

Also, be sure 50 characters is enough to look ahead (assuming you did, but thought I'd mention it).

0 Karma

gcusello
SplunkTrust
SplunkTrust

I found a similar problem: time recognition is done before the character replacing.
Bye.
Giuseppe

0 Karma

dpanych
Communicator

So I updated the time prefix to include the delims before the sed, TIME_PREFIX = ".+"|;|".+"|;|".+"|;|"
and still no luck.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...