I have logs that contain the following datetime format:
29-06-2016_00-08-17
The props contain:
[odb]
TIME_PREFIX = ".+",".+","
TIME_FORMAT = %d-%m-%Y_%H-%M-%S
MAX_TIMESTAMP_LOOKAHEAD = 50
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
disabled = false
pulldown_type = true
SEDCMD-01_change_delims_in_oracle_logs = s/\|;\|/,/g
SEDCMD-02_remove_the_end = s/"\|\|\?--END---\?\|\|//g
REPORT-set_delimiters_oracle_logs = REPORT-delims_odb_logs
What could be the problem?
Your TIME_PREFIX
is wrong. Try these:
TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19
Your TIME_PREFIX
is wrong. Try these:
TIME_PREFIX = ("[^"]*",){3}"
MAX_TIMESTAMP_LOOKAHEAD = 19
Also, this needs to be deployed to your indexers, the splunk instances must be restarted, and only newly-indexed (post-restart) data will be effected (bad events will stay bad).
Does it make a difference if you use a capture group instead of a non capture group there? As in:
(?:"[^"]*",){3}"
you have to use \ before " so the prefix is
\"\w+\",\"\w+\",\"\w+\",\"
Bye.
Giuseppe
In this case, they are the same.
I'm no regex guru, but how do I make the regex above work with the |;| delims? ---- "aaa"|;|"OIM"|;|"DELETE"|;|"29-06-2016_01-53-16"
Just modify the regex to pick the delimiters instead of a comma
("[^"]*"\|;\|){3}"
Thank you guys, this worked!
https://regex101.com/ is your friend
We cannot say without you posting a sample event.
Here's what the logs looks like after the sed:
"host1","MON","LOGOFF","30-06-2016_11-15-01","","0"
"host2","ODS","UPDATE","30-06-2016_12-51-05","UPDATE DS_ATTRSTORE SET ATTRVAL = :B1 WHERE ENTRY = :B2 AND ATTRNAME = 'modname'","0"
"host3","ODS","UPDATE","30-06-2016_08-28-43","UPDATE DS_ATTRSTORE SET ATTRVER = :B4 || CHR(94) || :B5 || CHR(94) || :B6 , ATTRVAL = :B3 , ATTRKIND = :B2 , ATTRSTYPE = :B1 WHERE ENTRY = :B8 AND ATTRNAME = :B7","0"
Can you post a sample event as well?
Not sure if I'm reading that correctly, but are you replacing the delimiters in the raw text from "|;|" to ","? If so, is that the same comma you're expecting to be there for the time prefix? If it is, I'm wondering if the timestamp recognition happens before sedcmd is applied?
Also, be sure 50 characters is enough to look ahead (assuming you did, but thought I'd mention it).
I found a similar problem: time recognition is done before the character replacing.
Bye.
Giuseppe
So I updated the time prefix to include the delims before the sed, TIME_PREFIX = ".+"|;|".+"|;|".+"|;|"
and still no luck.