props.conf Line Breaking

[Svill321]

Hello everyone,

I have several events with different time stamps that I'm trying to breakup. The props file I'm using is as follows:

sourcetype=applogs
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=false
BREAK_ONLY_BEFORE = ^(\d{2}\:\d{2}\:\d{2}\||\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}|\d\/\d{2}\/\d{4}\s+?\d{2}:\d{2}:\d{2}:)
disabled = false
pulldown_type = true

Alternatively, I also use:

sourcetype=applogs
SHOULD_LINEMERGE=false
LINE_BREAKER = ([\r\n]+)( \d{2}\:\d{2}\:\d{2}\||\d{4}\-\d{2}\-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}|\d\/\d{2}\/\d{4}\s+?\d{2}:\d{2}:\d{2}:)
disabled = false
pulldown_type = true

The logs look like this:

14:49:27| stuff
14:49:27|stuff
14:49:28|stuff

Another log looks like
7/17/2017 17:02:12:stuff
7/17/2017 17:02:12:stuff
7/17/2017 17:02:12:stuff

And yet another log looks like this:

7/17/2017 17:01:58.345, stuff
7/17/2017 17:01:58.355, stuff
7/17/2017 17:01:58.376, stuff

Could someone let me know what I'm doing wrong in my prop.conf files? For some reason they're not breaking up the file

[fredzhang]

Suggest use different stanzas with source::.... to do line break for each different type logs.
E.g. for each source:

[source::...source1...]
BREAK_ONLY_BEFORE_DATE = true
DATETIME_CONFIG = none
TIME_FORMAT = XXXX // set as log shows
...

[MuS]

Hi Svill321,

here is a quick check list:

1. What are you applying this on? source, sourcetype or host? Check for typos
2. Where is this props.conf applied? Must be on the parsing layer, either heavyweight forwarder or indexer
3. Check with $SPLUNK_HOME/bin/splunk btool props list if your config on the parsing instance is applied correctly
4. your first regex does not match, try this \d{2}\:\d{2}\:\d{2}\||\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}:|\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}\.\d{3},
5. BREAK_ONLY_BEFORE only works if Splunk encounters a new line that matches the regular expression.
6. same with the second regex, try this instead `([\r\n]+)\d{2}:\d{2}:\d{2}||\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}:|\d\/\d{2}\/\d{4}\s+\d{2}:\d{2}:\d{2}.\d{3},
7. did you restart the parsing Splunk instance after the changes?
8. This will only be applied to new arriving events, it is not applied to any existing already indexed events

Hope this helps to trace down the error ...

cheers, MuS