Getting Data In

Why is Splunk not parsing the correct datetime?

Communicator

I have logs that contain the following datetime format:

29-06-2016_00-08-17

The props contain:

[odb]
TIME_PREFIX = ".+",".+","
TIME_FORMAT = %d-%m-%Y_%H-%M-%S
MAX_TIMESTAMP_LOOKAHEAD = 50
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
pulldown_type = true
SEDCMD-01_change_delims_in_oracle_logs = s/\|;\|/,/g
SEDCMD-02_remove_the_end = s/"\|\|\?--END---\?\|\|//g
REPORT-set_delimiters_oracle_logs = REPORT-delims_odb_logs

What could be the problem?

0 Karma
1 Solution

Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

View solution in original post

Esteemed Legend

Your TIME_PREFIX is wrong. Try these:

TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19

View solution in original post

Esteemed Legend

Also, this needs to be deployed to your indexers, the splunk instances must be restarted, and only newly-indexed (post-restart) data will be effected (bad events will stay bad).

0 Karma

Influencer

Does it make a difference if you use a capture group instead of a non capture group there? As in:

(?:"[^"]*",){3}"
0 Karma

Legend

you have to use \ before " so the prefix is
\"\w+\",\"\w+\",\"\w+\",\"

Bye.
Giuseppe

0 Karma

Esteemed Legend

In this case, they are the same.

0 Karma

Communicator

I'm no regex guru, but how do I make the regex above work with the |;| delims? ---- "aaa"|;|"OIM"|;|"DELETE"|;|"29-06-2016_01-53-16"

0 Karma

Influencer

Just modify the regex to pick the delimiters instead of a comma

 ("[^"]*"\|;\|){3}"

Communicator

Thank you guys, this worked!

0 Karma

Influencer

https://regex101.com/ is your friend

0 Karma

Esteemed Legend

We cannot say without you posting a sample event.

0 Karma

Communicator

Here's what the logs looks like after the sed:

"host1","MON","LOGOFF","30-06-2016_11-15-01","","0"
"host2","ODS","UPDATE","30-06-2016_12-51-05","UPDATE DS_ATTRSTORE SET ATTRVAL = :B1 WHERE ENTRY = :B2 AND ATTRNAME = 'modname'","0"
"host3","ODS","UPDATE","30-06-2016_08-28-43","UPDATE DS_ATTRSTORE SET ATTRVER = :B4 || CHR(94) || :B5 || CHR(94) || :B6 , ATTRVAL = :B3 , ATTRKIND = :B2 , ATTRSTYPE = :B1 WHERE ENTRY = :B8 AND ATTRNAME = :B7","0"
0 Karma

Influencer

Can you post a sample event as well?

0 Karma

Champion

Not sure if I'm reading that correctly, but are you replacing the delimiters in the raw text from "|;|" to ","? If so, is that the same comma you're expecting to be there for the time prefix? If it is, I'm wondering if the timestamp recognition happens before sedcmd is applied?

Also, be sure 50 characters is enough to look ahead (assuming you did, but thought I'd mention it).

0 Karma

Legend

I found a similar problem: time recognition is done before the character replacing.
Bye.
Giuseppe

0 Karma

Communicator

So I updated the time prefix to include the delims before the sed, TIME_PREFIX = ".+"|;|".+"|;|".+"|;|"
and still no luck.

0 Karma