BREAK_ONLY_BEFORE and BREAK_ONLY_BEFORE_DATE=false in the same props.conf

[cweissnike]

We're trying to break up some log entries that look like:

2019-03-27 17:11:59.942 Request was not matched as were no stubs registered:
{
  "url" : "/",
  "absoluteUrl" : "http://localhost:8080/",
  "method" : "GET",
  "clientIp" : "127.0.0.1",
  "headers" : {
    "User-Agent" : "Wget",
    "Connection" : "close",
    "Host" : "localhost:8080"
  },
  "cookies" : { },
  "browserProxyRequest" : false,
  "loggedDate" : 1553706719942,
  "bodyAsBase64" : "",
  "body" : "",
  "loggedDateString" : "2019-03-27T17:11:59Z",
  "queryParams" : { }
}

Our props.conf looks like:

SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE = \d\d\d\d[-]\d\d[-]\d\d[ ]\d\d[:]\d\d[:]\d\d[.]\d\d\d

The trouble is, our Splunk result is breaking on every date. The above regex matches only the top date format in my regex tester.
So with the above data, Splunk is parsing it into:
Record 1:

2019-03-27 17:11:59.942 Request was not matched as were no stubs registered:
    {
      "url" : "/",
      "absoluteUrl" : "http://localhost:8080/",
      "method" : "GET",
      "clientIp" : "127.0.0.1",
      "headers" : {
        "User-Agent" : "Wget",
        "Connection" : "close",
        "Host" : "localhost:8080"
      },
      "cookies" : { },
      "browserProxyRequest" : false,

Record 2:

      "loggedDate" : 1553706719942,
      "bodyAsBase64" : "",
      "body" : "",

Record 3:

      "loggedDateString" : "2019-03-27T17:11:59Z",
      "queryParams" : { }
    }

That sure looks like it's using both BREAK_ONLY_BEFORE and BREAK_ONLY_BEFORE_DATE = true. We've tried adding BREAK_ONLY_BEFORE_DATE = false to the props.conf and it's still breaking up at all dates. I read in another Q that you cannot use both BREAK_ONLY_BEFORE and BREAK_ONLY_BEFORE_DATE in the same props.conf, how do we set it to not break at the other date instances?

[cweissnike · Accepted solution]

In our case, it turned out to be server-side filters that were overriding our local configs. Since our instance is managed by a central team, we did not have visibility into the server-side filters.

[simontam]

I have the same problem. No matter what I changed in the props.conf, Splunk still break the lines by date time. Did you fix your problem yet?

[cweissnike]

Nevermind - There's something far weirder going on - Splunk is not actually processing any of the directives in our props.conf file, it's just defaulting to the BREAK_ONLY_BEFORE_DATE=true.

[somesoni2]

I find this configuration works correct consistently

[YourSourcetype]
#making it false as we're changing line boundary in LINE_BREAKER
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\d{4}-\d{2}-\d{2}\s+\d{2})
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 23

[cweissnike]

Hmmm. That didn't seem to change the results. Just to verify, I threw some garbage into the file and it did correctly complain, so I know it's actually reading the file I'm editing.