topic event line break in props in Getting Data In

event line break in props

narenpg — Thu, 21 Nov 2024 21:31:11 GMT

---------------------------- This is an Example (He/She) ----------------------------- Version: 21.04.812-174001 Date/time: 2024-10-18/01:00:06 (2024-10-18/05:00:06 UTC) User/aplnid: /2370 ComputerName/-user: Ann/King Windows NT version 6.2, build no. 9200 /10872/6241785241 -> Loading program ---------------------------------------------------------------------------------------------------- ---------------------------- This is an Example (He/She) ----------------------------- Version: 21.04.812-174001 Date/time: 2024-10-18/01:00:06 (2024-10-18/05:00:06 UTC) User/aplnid: /2370 ComputerName/-user: James/Bond Windows NT version 6.2, build no. 9200 /10872/6241785241 -> Start APL (pid 8484) ---------------------------------------------------------------------------------------------------- ---------------------------- This is an Example (He/She) ----------------------------- Version: 21.04.812-174001 Date/time: 2024-10-18/01:00:06 (2024-10-18/05:00:06 UTC) User/aplnid: /2370 ComputerName/-user: Martin/King Windows NT version 6.2, build no. 9200 /10872/6241785241 -> Initialising external processes ----------------------------------------------------------------------------------------------------

I am trying to break events at "This is an Example"

[mysourcetype]
TIME_FORMAT = %Y-%m-%d/%H:%M:%S
TIME_PREFIX = Date\/time:\s+
TZ = US/Eastern
LINE_BREAKER = (.*)(This is An Example).*
SHOULD_LINEMERGE = false

This works when i test in "Add Data" but it is not working under props.conf. All the lines are merged into one event. What is the issue in this?

Re: event line break in props

PickleRick — Thu, 21 Nov 2024 21:50:29 GMT

1. Are you sure the LINE_BREAKER is right? I mean - the capture group in the LINE_BREAKER will be treated as the line breaker and will be removed from the stream. Are you sure you want to cut this much? Not more, not less? Also you usually include \r and/or \n explicitly in your line breaker definition. Otherwise the results might not be what you expect.

2. Are you sure you're putting your props.conf on the proper component in your environment?

3. Did you verify with btool that there is no other setting overwriting your line breaker?

Re: event line break in props

narenpg — Thu, 21 Nov 2024 22:33:42 GMT

1. It truncates hyphen - before the "This is an Example" now i added ([\r\n+])(.*)(This is an Example).* it captures everthing. But the events are broken into single lines. I have set SHOULD_LINE_MERGE = false.

2. Yes props.conf is on the proper component

3. i verified using this command

splunk btool inputs list --debug (there is no other setting that is overwriting LINE_BREAKER)

NOTE: can i use BREAK_ONLY_BEFORE instead of LINE_BREAKER

Re: event line break in props

PickleRick — Fri, 22 Nov 2024 09:48:45 GMT

1. OK. It's just that I'd probably just cut the whole "This is an example" line if it's just a constant delimiter between the events.

2. Where? And what does your ingestion process look like?

3. LINE_BREAKER is not defined at input level. It's defined in props but I'm assuming you meant "splunk btool props list", not inputs. If not, check props, not inputs.

BREAK_ONLY_BEFORE is a setting used only when SHOULD_LINEMERGE is set to true and that case is best avoided (there are very very rare cases where it makes sense; if possible, avoid enabling line-merging)

Re: event line break in props

narenpg — Fri, 22 Nov 2024 15:03:19 GMT

1. Yes This is the constant delimiter ---------------------------- This is an Example (He/She) -----------------------------

2. It picks up every 7th line and skips others. I think that is because i used \n+ right?
3. I should have used "splunk btool props list" instead of inputs.. I ran the command and i see only one LINE_BREAKER for that sourcetype.

Thanks for the info on BREAK_ONLY_BEFORE

What is the Regex i should use it on the LINE_BREAKER?