Getting Data In

New sourcetype, problems creating transforms with field names, weird delimiter


I'm trying to ingest historical Windows security event logs from Nitro into Splunk. The event fields are delimited by a double-pipe. I'm green on creating a transforms that will deal with this data, and would love some input. An example event is shown below:

2017 Feb 28 23:57:31,||Security||4094031727||Microsoft-Windows-Security-Auditing||4656||61||1488344058||4||||||File System||16||S-1-5-18||DCNDCDNSFF01$||domain||0x3e7||Security||File||C:\Windows\Boot\PCAT||0x154||{00000000-0000-0000-0000-000000000000}||%25%251538%0D %09%09%09%09%25%251539%0D %09%09%09%09%25%251540%0D %09%09%09%09%25%251542%0D %09%09%09%09||%25%251538:%09%25%251804%0D %09%09%09%09%25%251539:%09%25%251804%0D %09%09%09%09%25%251540:%09%25%251801%09SeTakeOwnershipPrivilege%0D %09%09%09%09%25%251542:%09%25%251801%09SeSecurityPrivilege%0D %09%09%09%09||0x10e0000||SeSecurityPrivilege%0D %09%09%09SeTakeOwnershipPrivilege||0||0x208||C:\Windows\System32\services.exe||A handle to an object was requested.%0D %0D Subject:%0D %09Security ID:%09%09S-1-5-18%0D %09Account Name:%09%09DCNDCDNSFF01$%0D %09Account Domain:%09%09domain%0D %09Logon ID:%09%090x3e7%0D %0D Object:%0D %09Object Server:%09%09Security%0D %09Object Type:%09%09File%0D %09Object Name:%09%09C:\Windows\Boot\PCAT%0D %09Handle ID:%09%090x154%0D %0D Process Information:%0D %09Process ID:%09%090x208%0D %09Process Name:%09%09C:\Windows\System32\services.exe%0D %0D Access Request Information:%0D %09Transaction ID:%09%09{00000000-0000-0000-0000-000000000000}%0D %09Accesses:%09%09READ_CONTROL%0D %09%09%09%09WRITE_DAC%0D %09%09%09%09WRITE_OWNER%0D %09%09%09%09ACCESS_SYS_SEC%0D %09%09%09%09%0D %09Access Reasons:%09%09READ_CONTROL:%09Granted by Ownership%0D %09%09%09%09WRITE_DAC:%09Granted by Ownership%0D %09%09%09%09WRITE_OWNER:%09Granted by%09SeTakeOwnershipPrivilege%0D %09%09%09%09ACCESS_SYS_SEC:%09Granted by%09SeSecurityPrivilege%0D %09%09%09%09%0D %09Access Mask:%09%090x10e0000%0D %09Privileges Used for Access Check:%09SeSecurityPrivilege%0D %09%09%09SeTakeOwnershipPrivilege%0D %09Restricted SID Count:%090

I've created a sourcetype, winevent:sec:archive, and on ingesting the events into my local splunk instance, a single pipe, |, seems to break the fields up, while a double pipe just shows the time field, and no other fields, telling me that splunk doesn't like a double-pipe delimiter.

FWIW, here's the props I've got, but I need help setting up the transforms with field names, of which I have most of them.

KV_MODE = none
category = Structured
description = Pipe-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

I hope this question makes sense. I'd appreciate any help you can provide. Thanks.

0 Karma


Can you try escaping the pipes in your delimiter setting and let us know how it goes? ||

0 Karma


No change with the field extractor. I added the following line to my props.conf but the field wasn't extracted:

EXTRACT-LogName = ^\d+\s\w+\s+\d+\s\d+\:\d+\:\d+,\d+.\d+.\d+.\d+\|\|\w+(?<LogName>)

Edit: I should say I also edited the delimiter line and changed it to ||, again no change.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!