PREAMBLE_REGEX used to ignore line of log not working as stated in splunk docs

[net1993]

Hello
I found this attribute in mysql app in props.conf:
PREAMBLE_REGEX = #\sTime:\s\d+\s+\d{1,2}:\d{2}:\d{2}

test data: Time: 190807 13:26:28

The issue is that nothing happens and the line with this string is still comming in splunk and makes the whole parsing wrong as it creates a new event only for it instead of ignoring it:(

I tried the regex and is correct. Tried it also in splunk with regex command and works fine so I believe this is due to that command.

Some others to help or know more about it?

[skalliger]

If I recall correct, PREAMBLE_REGEX tells Splunk to ignore something unstructured in terms of extractions but that does not mean that it won't be indexed.

Skalli

[net1993]

PREAMBLE_REGEX =
* A regular expression that lets Splunk software ignore "preamble lines",
or lines that occur before lines that represent structured data.
* When set, Splunk software ignores these preamble lines,
based on the pattern you specify.
* Default: not set
* List item

Isn't suppose to ignore any lines which matches the specified pattern?
This attribute was in splunk_TA_mysql and not working. I was need to correct the regex as it was not matching but before or after , it doesn't work.
Maybe I understand wrong the purpose of that preamble liines and if so, can you give example in practice what kind of scenario this will work?

[hrawat_splunk]

It's been an issue that is fixed in 7.2.0 and above. What is your splunk version?

[net1993]

It's 7.2.6

[hrawat_splunk]

May be your preamble lines are in the middle of file?
Is the file content something like

===begin===
preamble line 1
structured data1
preamble line2
structured data2
preamble line3
structured data3
preamble line4
===end===

[net1993]

No , it's more like this:

===begin===
structured data1
preamble line 1
structured data2
preamble line2
structured data3
preamble line3
structured data4
structured data5
===end===

*The preamble line doesnt always exist.

[hrawat_splunk]

Interesting use case. If this file growing? or it's one time file?
If it's growing, what is the time interval for file to grow?

[net1993]

It's growing.
time interval - no idea.
It's mysqld_slow sourcetype- there is an app for it but even that not parsing correct. I got this preamble_line from there
If important , I can try find this time interval but I think its different depending on mysql slow queries log.

[hrawat_splunk]

try setting time_before_close=300 for this particular input and test.

[richgalloway]

Please share all the props.conf settings for that sourcetype and a sample preamble line.

[net1993]

[mysqld_slow]
pulldown_type = true
category = Database
description = Mysql Slow Query Logs
BREAK_ONLY_BEFORE = ^#\s*User@Host
MAX_TIMESTAMP_LOOKAHEAD = 128
TIME_FORMAT = %s
TIME_PREFIX = timestamp=
SHOULD_LINEMERGE = true
PREAMBLE_REGEX = #\sTime:\s\d+\s+\d{1,2}:\d{2}:\d{2}
TRUNCATE = 20000

[net1993]

I have already provided the test data in my first post but I will post for you again:

Time: 190807 13:26:28