New sourcetype, problems creating transforms with field names, weird delimiter

manderson7 — Wed, 17 Jan 2018 16:42:44 GMT

I'm trying to ingest historical Windows security event logs from Nitro into Splunk. The event fields are delimited by a double-pipe. I'm green on creating a transforms that will deal with this data, and would love some input. An example event is shown below:

2017 Feb 28 23:57:31,172.30.66.143||Security||4094031727||Microsoft-Windows-Security-Auditing||4656||61||1488344058||4||DCNDCDNSFF01.domain.dev||||File System||16||S-1-5-18||DCNDCDNSFF01$||domain||0x3e7||Security||File||C:\Windows\Boot\PCAT||0x154||{00000000-0000-0000-0000-000000000000}||%25%251538%0D %09%09%09%09%25%251539%0D %09%09%09%09%25%251540%0D %09%09%09%09%25%251542%0D %09%09%09%09||%25%251538:%09%25%251804%0D %09%09%09%09%25%251539:%09%25%251804%0D %09%09%09%09%25%251540:%09%25%251801%09SeTakeOwnershipPrivilege%0D %09%09%09%09%25%251542:%09%25%251801%09SeSecurityPrivilege%0D %09%09%09%09||0x10e0000||SeSecurityPrivilege%0D %09%09%09SeTakeOwnershipPrivilege||0||0x208||C:\Windows\System32\services.exe||A handle to an object was requested.%0D %0D Subject:%0D %09Security ID:%09%09S-1-5-18%0D %09Account Name:%09%09DCNDCDNSFF01$%0D %09Account Domain:%09%09domain%0D %09Logon ID:%09%090x3e7%0D %0D Object:%0D %09Object Server:%09%09Security%0D %09Object Type:%09%09File%0D %09Object Name:%09%09C:\Windows\Boot\PCAT%0D %09Handle ID:%09%090x154%0D %0D Process Information:%0D %09Process ID:%09%090x208%0D %09Process Name:%09%09C:\Windows\System32\services.exe%0D %0D Access Request Information:%0D %09Transaction ID:%09%09{00000000-0000-0000-0000-000000000000}%0D %09Accesses:%09%09READ_CONTROL%0D %09%09%09%09WRITE_DAC%0D %09%09%09%09WRITE_OWNER%0D %09%09%09%09ACCESS_SYS_SEC%0D %09%09%09%09%0D %09Access Reasons:%09%09READ_CONTROL:%09Granted by Ownership%0D %09%09%09%09WRITE_DAC:%09Granted by Ownership%0D %09%09%09%09WRITE_OWNER:%09Granted by%09SeTakeOwnershipPrivilege%0D %09%09%09%09ACCESS_SYS_SEC:%09Granted by%09SeSecurityPrivilege%0D %09%09%09%09%0D %09Access Mask:%09%090x10e0000%0D %09Privileges Used for Access Check:%09SeSecurityPrivilege%0D %09%09%09SeTakeOwnershipPrivilege%0D %09Restricted SID Count:%090

I've created a sourcetype, winevent:sec:archive, and on ingesting the events into my local splunk instance, a single pipe, |, seems to break the fields up, while a double pipe just shows the time field, and no other fields, telling me that splunk doesn't like a double-pipe delimiter.

FWIW, here's the props I've got, but I need help setting up the transforms with field names, of which I have most of them.

[wineventlog:sec:archive]
DATETIME_CONFIG = 
FIELD_DELIMITER = |
HEADER_FIELD_DELIMITER = |
INDEXED_EXTRACTIONS = psv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Pipe-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true

I hope this question makes sense. I'd appreciate any help you can provide. Thanks.

Re: New sourcetype, problems creating transforms with field names, weird delimiter

davpx — Thu, 18 Jan 2018 23:53:25 GMT

Can you try escaping the pipes in your delimiter setting and let us know how it goes? ||

Re: New sourcetype, problems creating transforms with field names, weird delimiter

manderson7 — Fri, 19 Jan 2018 21:37:17 GMT

No change with the field extractor. I added the following line to my props.conf but the field wasn't extracted:

EXTRACT-LogName = ^\d+\s\w+\s+\d+\s\d+\:\d+\:\d+,\d+.\d+.\d+.\d+\|\|\w+(?<LogName>)

Edit: I should say I also edited the delimiter line and changed it to ||, again no change.

topic Re: New sourcetype, problems creating transforms with field names, weird delimiter in Getting Data In

New sourcetype, problems creating transforms with field names, weird delimiter

Re: New sourcetype, problems creating transforms with field names, weird delimiter

Re: New sourcetype, problems creating transforms with field names, weird delimiter