timestamp extra

[crazyeva]

I put some oracle-exported data into splunk, with props.conf:

NO_BINARY_CHECK = true
CHARSET = GB2312
SHOULD_LINEMERGE = false
TIME_PREFIX = (?(?:\d{2}(?:\/\d{2}){2}\s\d{2}(?::\d{2}){2})|(?:\d{4}(?:-\d{2}){2}\s\d{2}(?::\d{2}){2}))(?=(?:^[^^]*){27}$)

same result,easy to read: TIMEPREFIX = (?[^\^]*)(?=(?:\^[^\^]*){27}$)

Most timestamps are extracted correctly, but two of them are unexpected:

event 1, 7/15/12 9:35:17.000 PM should be "05/03/12 15:56:32" and event 2, 7/15/12 9:27:02.000 PM should be "04/11/12 19:15:18" stong characters in raw

1 » 7/15/12 9:35:17.000 PM

2012-05-03 15:57:45^INSERT^ "ipb-a-cjx-cx600-101SHELL/5/CMDRECORD(l): Record command information. (Task vt0 Ip 58.246.74.188 User gongchuang Command efu np-2 slot 2 ingress display status )"^129920652^"ipb-a-cjx-cx600-1"^"124.75.5.14"^"Syslog Probe on nmman5-pd"^"adsl-CX600"^""^"%%01SHELL"^2^"01SHELL/5/CMDRECORD(l): Record command information. (Task vt0 Ip 58.246.74.188 User gongchuang Command efu np-2 slot 2 ingress display status )"^05/03/12 15:57:22^05/03/12 15:56:32^05/03/12 15:56:32^05/03/12 15:57:22^0^1^1^200^0^""^65534^0^0^0^"TROU5"^60637884^""^""^0^0^""^"124.75.5.14"^""^""^""^""^""^""^""
FirstOccurrence=05/03/12 15:56:32 Options| InternalLast=05/03/12 15:57:22 Options| StateChange=05/03/12 15:57:22 Options

2 » 7/15/12 9:27:02.000 PM

2012-04-11 19:16:49^UPDATE^ "ipb-a-yh-9312-101SHELL/6/DISPLAY_CMDRECORD(l): Record command information. (Task vt0 Ip 124.74.213.3 User root Command display igmp-snooping port-info vlan 51 )DISPLAY_CMDRECORD(l):%%01SHELL/6/DISPLAY_CMDRECORD(l):Recordcommand"^126438075^"ipb-a-yh-9312-1"^"124.75.192.222"^"Syslog Probe on nmman5-pd"^"9312"^"DISPLAY_CMDRECORD(l):"^"%%01SHELL/6/DISPLAY_CMDRECORD(l):Recordcommand"^1^"01SHELL/6/DISPLAY_CMDRECORD(l): Record command information. (Task vt0 Ip 124.74.213.3 User root Command display igmp-snooping port-info vlan 51 )"^04/11/12 19:15:58^04/10/12 19:15:16^04/11/12 19:15:18^04/11/12 19:15:18^0^1^97^200^0^""^65534^0^0^0^"TROU5"^58515545^""^""^0^0^""^"124.75.192.222"^""^""^""^""^""^""^""
FirstOccurrence=04/10/12 19:15:16 Options| InternalLast=04/11/12 19:15:18 Options| StateChange=04/11/12 19:15:58

[mloven_splunk · Accepted solution]

crazyeva,

That first TIME_PREFIX makes my head hurt. Let's go with something closer to your second example.

I would do something along these lines:

TIME_PREFIX = ([\r\n]+)(?([^^]+^){14})

[mloven_splunk]

TIME_PREFIX is used to tell Splunk what comes before the timestamp.

Also, in that TIME_PREFIX that I provided, the regex in the first set of parenthesis will match any number of returns or newlines.

[crazyeva]

Sorry to reply so late.
The number of "^" is not sure from line starts, so i tried to find timestamp from their tails.
By the way, I am confused that TIME_PREFIX attempts to match what is before timestamp or just to match timestamp?