I would like to break my logs at every time + log level but it is not working as expected.
Here's my props.conf :
[log_name] TIME_FORMAT = %H:%M:%S,%3N MAX_TIMESTAMP_LOOKAHEAD = 12 NO_BINARY_CHECK = true disabled = false MAX_EVENTS = 10240 TRUNCATE = 0 [other_name] SHOULD_LINEMERGE = false NO_BINARY_CHECK = true MAX_TIMESTAMP_LOOKAHEAD = 32 disabled = false TRUNCATE = 0 MAX_EVENTS = 10240
My log file :
09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - text.texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | WARN | 640512999-933058 | someinformation | 204 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 72 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,359 | INFO | 640512999-933058 | someinformation | 243 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,360 | INFO | 640512999-933058 | someinformation | 243 - texttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttexttext 09:31:51,361 | INFO | 640512999-933058 | someinformation | 240 -
I tried to use BREAKONLYBEFORE option in props.conf with the following regexp :
Same behavior with LINE_BREAKER option.
Can someone give me a hint on how to configure my props.conf ? What am I missing ?
if each of your events is in one row (in other words, if it has LF CR) you don't need of LINE BREAK, it's useful when you have multilines events.
Splunk divide events using the date, in this way you say to Splunk that the date is at the start of the row.
Add this oprtion to your props anche check the results, it should be sufficient.
Don't add also the other options.
Hi @gcusello !
Thank you for your help. I tried your solution but it doesn' t work.
Maybe I applied it wrong. What do you mean by "Don't add also the other options." ? I should use only TIMEPREFIX = ^ and not TIMEFORMAT etc ?
Thanks in advance
Sorry, I wasn't clear, try this:
[log_name] TIME_FORMAT = %H:%M:%S,%3N MAX_TIMESTAMP_LOOKAHEAD = 12 NO_BINARY_CHECK = true disabled = false MAX_EVENTS = 10240 TRUNCATE = 0 TIME_PREFIX = ^
I tried your solution, I see improvement but I still have some weird behavior.
For example, it still breaks before the "Header" line, and I can't explain or understand why.
The following picture is where I would like logs to be cut FYI.
Do you have any idea ?
Thanks a lot for your help !
I also tried this props.conf as suggested by @whrg :
[log] pulldown_type = true DATETIME_CONFIG = NO_BINARY_CHECK = true TIME_PREFIX = ^ TIME_FORMAT = %H:%M:%S,%3N MAX_TIMESTAMP_LOOKAHEAD = 12 SHOULD_LINEMERGE = false (& true) BREAK_ONLY_BEFORE_DATE = true
but I had the same result as before.
How long after restarting Splunk service should I check my logs ?
This article Configure event line breaking goes into detail on how line breaking works.
The dates in your logs (a space character at the beginning of each line? and no year/month/day) look peculiar. You will need to set the timestamp extraction manually. The rest should work fine. By default, SHOULDLINEMERGE and BREAKONLYBEFOREDATE are set to true.
This is the props.conf:
[log_name] category = Custom pulldown_type = true DATETIME_CONFIG = NO_BINARY_CHECK = true TIME_PREFIX = \s* TIME_FORMAT = %H:%M:%S,%3N SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE_DATE = true