topic Re: Crashplan Service Log Date timestamp incorrect in Getting Data In

Crashplan Service Log Date timestamp incorrect

bmorgenthaler — Tue, 11 Feb 2020 19:13:08 GMT

I've looked through a lot of the posts about date timestamp extraction and I think I'm decent enough at it but for the life of me I can't figure out what is going on with my logs for Crashplan. I found a post with a working example of crashplan service log props and mine matched almost exactly but still no go.

props.conf

[crashplan_service]
TIME_PREFIX = ^\[
MAX_TIMESTAMP_LOOKAHEAD = 21
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N
SHOULD_LINEMERGE = false
NO_BINARY_CHECK = true

inputs.conf

[monitor:///opt/crashplan/log/service.log.0]
source = crashplan
sourcetype = crashplan_service
index = crashplan
disabled = false

Here is one event where the day/month look to be swapped:

With this event I have no idea how it's getting the day/month:

Re: Crashplan Service Log Date timestamp incorrect

somesoni2 — Tue, 11 Feb 2020 19:38:45 GMT

I would check if props.conf is located in correct place. It should be placed on the first Splunk Enterprise instance in your data flow. If it's placed on your heavy forwarder/intermediate heavy forwarder/indexer (whichever comes first), then give this configuration a try:

[crashplan_service]
 SHOULD_LINEMERGE = false
 LINE_BREAKER = ([\r\n]+)(?=\[\d+\.\d+\d\.\d+)
 TIME_PREFIX = ^\[
 MAX_TIMESTAMP_LOOKAHEAD = 21
 TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N

Re: Crashplan Service Log Date timestamp incorrect

bmorgenthaler — Tue, 11 Feb 2020 23:46:24 GMT

@somesoni2 I tried that out and it someone worked but then I found a number of multiline events that didn't play nicely with that. I ended up using EVENT_BREAKER

Re: Crashplan Service Log Date timestamp incorrect

bmorgenthaler — Tue, 11 Feb 2020 23:49:44 GMT

Well I've solved my ingestion issues for CrashPlan's service log, now for the other 6. Here is the props stanza I ended up using.

[crashplan_service]
TIME_PREFIX = ^\[
BREAK_ONLY_BEFORE = (\r\n)?\[\d{2}\.\d{2}\.\d{2}\s\d{2}\:\d{2}\:\d{2}\.\d{3}
MAX_TIMESTAMP_LOOKAHEAD = 21
MAX_EVENTS = 1000
NO_BINARY_CHECK = true
TIME_FORMAT = %m.%d.%y %H:%M:%S.%3N

This allowed Splunk to ingest Crashplan configuration output (yay 600+ lines of xml), into a single event.