I have defined a TCP input in inputs.conf with the following corresponding stanza in props.conf (Splunk Enterprise 6.4):
[source::tcp:6067]
KV_MODE = json
LINE_BREAKER = ((^[^{][^\r]*\r\n)*)\{\"[^}]+\}
SHOULD_LINEMERGE = false
If I send the following text to that port:
Preamble lines
That I do not want
To appear in the event
The following line is intentionally blank
{"myfield": "some_value"}
(with \r\n at the end of each line)
I get two events in Splunk:
- The event I want,
{"myfield": "some_value"}, with myfield correctly presented as a field (so, KV_MODE = json is working).
- An unwanted event, with a time stamp that is the same or earlier, consisting of the "preamble" lines that I thought I'd told
LINE_BREAKER to discard!
According to the props.conf documentation:
The contents of the first capturing group are discarded, and will not be present in any event.
Yes, the contents of the first capturing group are discarded from the event I want... but they are present in that unwanted (and unexpected) event.
Why do I get that unwanted event? How do I prevent it?
I'm deliberately using the descriptive term "preamble" here, because I have previously attempted to do the same thing (discard those "preamble" lines) using PREAMBLE_REGEX instead of LINE_BREAKER:
[source::tcp:6067]
KV_MODE = json
HEADER_FIELD_LINE_NUMBER = 1
PREAMBLE_REGEX = ^[^{].*
but I cannot get PREAMBLE_REGEX to work, no matter what combination of regex and preamble test cases I use; at least, not for a TCP input. I wonder whether PREAMBLE_REGEX only applies to, say, file inputs, not TCP (or other network) inputs. The props.conf documentation hints at this with the word "files":
Some files contain preamble lines.
but if it's true, I'd prefer that the documentation was more explicit (and this makes me wonder about the implicit limitations of other settings).
