I recently had cause to ingest Oracle Unified Directory logs in ODL format. I'm performing pretty simple file-based ingestion and didn't want to go down the path of DB connect, etc. so it made sense to write my own parsing logic. I didn't have a lot of luck finding things precisely relevant to my logs and I could have benefited from the concrete example I'll lay out below, so here it is, hopefully it will help others. This specific example is OUD logs, but it's relevant to any arbitrary _KEY_n, _VAL_n extraction in Splunk. Log format The ODL logs are mostly well structured, but the format is a little odd to work with. Most fields are bracketed [FIELD_1] The log starts with "header" fields which are fixed and contain single values The next section of the log has an arbitrary number of bracketed key value pairs: [KEY: VALUE] The final piece is a non-structured string that may also contain brackets and colons Below is a small sample of a few different types of logs: [2016-12-30T11:08:46.216-05:00] [ESSBASE0] [NOTIFICATION:16] [TCP-59] [TCP] [ecid: 1482887126970,0] [tid: 140198389143872] Connected from [::ffff:999.999.99.999] [2016-12-30T11:08:27.60-05:00] [ESSBASE0] [NOTIFICATION:16] [AGENT-1001] [AGENT] [ecid: 1482887126970,0] [tid: 140198073563456] Received client request: Clear Application/Database (from user [sampleuser@Native Directory]) [2016-12-30T11:08:24.302-05:00] [PLN3] [NOTIFICATION:16] [REQ-91] [REQ] [ecid: 148308120489,0] [tid: 140641102035264] [DBNAME: SAMPLE] Received Command [SetAlias] from user [sampleuser@Native Directory] [2016-12-30T11:08:26.932-05:00] [PLN3] [NOTIFICATION:16] [SSE-82] [SSE] [ecid: 148308120489,0] [tid: 140641102035264] [DBNAME: SAMPLE] Spreadsheet Extractor Big Block Allocs -- Dyn.Calc.Cache : [202] non-Dyn.Calc.Cache : [0] [2025-02-07T15:18:35.014-05:00] [OUD] [TRACE] [OUD-24641549] [PROTOCOL] [host: testds01.redacted.fake] [nwaddr: redacted] [tid: 200] [userId: ldap] [ecid: 0000PJY46q64Usw5sFt1iX1bdZJL0003QL,0:1] [category: RES] [conn: 1285] [op: 0] [msgID: 1] [result: 0] [authDN: uid=redacted,ou=redacted,o=redacted,c=redacted] [etime: 0] BIND [2025-02-07T15:18:35.014-05:00] [OUD] [TRACE] [OUD-24641548] [PROTOCOL] [host: testds01.redacted.fake] [nwaddr: redacted] [tid: 200] [userId: ldap] [ecid: 0000PJY46q64Usw5sFt1iX1bdZJL0003QK,0:1] [category: REQ] [conn: 1285] [op: 0] [msgID: 1] [bindType: SIMPLE] [dn: uid=redacted,ou=redacted,o=redacted,c=redacted] BIND Configuration files We opted to use the sourcetype "odl:oud". This allows future extension into other ODL formatted logs. props.conf Note the 3-part REPORT processing. This utilizes regexes in transforms.conf to process the headers, the key-value pairs, and the trailing message. This could then be extended to utilize portions of that extraction for other log types that fall under ODL format. [odl:oud] REPORT-oudparse = extractOUDheader, extractODLkv, extractOUDmessage  transforms.conf This was the piece I could have used some concrete examples of. These three report extractions allow a pretty flexible ingestion of all parts of the logs. The key here was separating the _KEY_1 _VAL_1 extraction into its own process with the REPEAT_MATCH flag set # extract fixed leading values from Oracle Unified Directory log message [extractOUDheader] REGEX = ^\[(?<timestamp>\d{4}[^\]]+)\] \[(?<organization_id>[^\]]+)\] \[(?<message_type>[^\]]+)\] \[(?<message_id>[^\]]+)\] \[(?<component>[^\]]+?)\] # extract N number of key-value pairs from Oracle Diagnostic Logging log body [extractODLkv] REGEX = \[(?<_KEY_1>[^:]+?): (?<_VAL_1>[^\]]+?)\] REPEAT_MATCH = true # extract trailing, arbitrary message text from Oracle Unified Directory log [extractOUDmessage] REGEX = \[[^:]+: [^\]]+\] (?<message>[^\[].*)$.    The final regex there looks for a preceding key-value pair, NOT followed by a new square bracket, with any arbitrary characters thereafter to end the line. I initially tried to make one regex to perform all of this, which doesn't really work without being very prescriptive in the structure. With an unknown number of key-value pairs, this is not ideal and this solution seemed much more Splunk-esque. I hope this helps someone else! ... View more

For anyone using Hurricane Labs "Broken hosts" app (https://splunkbase.splunk.com/app/3247) note that the latest version, 4.2.2, appears to have a very minor but breaking bug. The file /default/savedsearches.conf has a stanza for the "Broken Hosts Alert - by contact" alert. Depending how you use the app, that potentially drives your entire alerting mechanism. Two lines in that file (121 & 130) wrap a built-in search macro in double quotes where they should not exist:   | fillnull value="`default_expected_time`" lateSecs   should be:   | fillnull value=`default_expected_time` lateSecs   The result of this is to assign the string value "`default_expected_time`" to the lateSecs variable, rather than expanding to whatever default integer you configured in the macro. Removing those double quotes from both lines seems to fix the issue. I've also raised an issue on the Hurricane Labs github page below...though activity there is pretty stale and I'm not sure if anyone is looking there... https://github.com/HurricaneLabs/brokenhosts/issues/3 ... View more