Hello
I found this attribute in mysql app in props.conf:
PREAMBLE_REGEX = #\sTime:\s\d+\s+\d{1,2}:\d{2}:\d{2}
test data: Time: 190807 13:26:28
The issue is that nothing happens and the line with this string is still comming in splunk and makes the whole parsing wrong as it creates a new event only for it instead of ignoring it:(
I tried the regex and is correct. Tried it also in splunk with regex command and works fine so I believe this is due to that command.
Some others to help or know more about it?
If I recall correct, PREAMBLE_REGEX
tells Splunk to ignore something unstructured in terms of extractions but that does not mean that it won't be indexed.
Skalli
PREAMBLE_REGEX =
* A regular expression that lets Splunk software ignore "preamble lines",
or lines that occur before lines that represent structured data.
* When set, Splunk software ignores these preamble lines,
based on the pattern you specify.
* Default: not set
* List item
Isn't suppose to ignore any lines which matches the specified pattern?
This attribute was in splunk_TA_mysql and not working. I was need to correct the regex as it was not matching but before or after , it doesn't work.
Maybe I understand wrong the purpose of that preamble liines and if so, can you give example in practice what kind of scenario this will work?
It's been an issue that is fixed in 7.2.0 and above. What is your splunk version?
It's 7.2.6
May be your preamble lines are in the middle of file?
Is the file content something like
===begin===
preamble line 1
structured data1
preamble line2
structured data2
preamble line3
structured data3
preamble line4
===end===
No , it's more like this:
===begin===
structured data1
preamble line 1
structured data2
preamble line2
structured data3
preamble line3
structured data4
structured data5
===end===
*The preamble line doesnt always exist.
Interesting use case. If this file growing? or it's one time file?
If it's growing, what is the time interval for file to grow?
It's growing.
time interval - no idea.
It's mysqld_slow sourcetype- there is an app for it but even that not parsing correct. I got this preamble_line from there
If important , I can try find this time interval but I think its different depending on mysql slow queries log.
try setting time_before_close=300 for this particular input and test.
Please share all the props.conf settings for that sourcetype and a sample preamble line.
[mysqld_slow]
pulldown_type = true
category = Database
description = Mysql Slow Query Logs
BREAK_ONLY_BEFORE = ^#\s*User@Host
MAX_TIMESTAMP_LOOKAHEAD = 128
TIME_FORMAT = %s
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = true
PREAMBLE_REGEX = #\sTime:\s\d+\s+\d{1,2}:\d{2}:\d{2}
TRUNCATE = 20000
I have already provided the test data in my first post but I will post for you again: