How to extract Message field from a new WinEventLog source?

[att35]

Hi,

Splunk is unable to parse the Message field for a new WinEventLog source. These are AD changes(Recorded by ChangeAuditor) into the local Windows logs on the Domain Controller which are then picked up by Universal Forwarder just like other default WinEvent Logs. But Splunk doesn't split the contents of Message field from this source as it does for say WinEventLog:Security. Here are two examples:

07/12/2018 08:12:48 AM
LogName=InTrust for AD
SourceName=ITAD Directory Changes
EventCode=3
EventType=4
Type=Information
ComputerName=DomainController.x.y.z
User=DomainController$
Sid=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
SidType=1
TaskCategory=%1
OpCode=None
RecordNumber=23398213
Keywords=Classic
Message=AD object was successfully modified.
    Client Computer : X.X.X.X
    Object DN : CN=,OU=,DC=x,DC=y,DC=local
    Object Class : computer
    Object GUID : CN=DOMAINCONTROLLER,OU=Domain Controllers,DC=x,DC=y,DC=z  Attribute Name : servicePrincipalName
    Action : Append
    Old Value : <not set>
    New Value : 
    Request ID : {8C7D7}


07/12/2018 08:12:18 AM
LogName=InTrust for AD
SourceName=ITAD Directory Changes
EventCode=87
EventType=4
Type=Information
ComputerName=DomainController.x.y.z
User=SYSTEM
Sid=xxxxxxxxxxxxxxxxxxxxxxxx
SidType=1
TaskCategory=%1
OpCode=None
RecordNumber=23398197
Keywords=Classic
Message=Account locked out
    Client Computer : xxxxxxx
    Account DN : CN=,OU=,OU=,OU=,OU=,DC=x,DC=y,DC=z
    Object Class : user
    Object GUID : CN=,OU=,OU=,OU=,OU=,DC=,DC=,DC=   Request ID : {0637DFC03183}

One difference between these and the logs from Security is that there is no newline once the Message description ends. Came across another answer which describes a fix for this exact scenario but that doesn't seem to work. https://answers.splunk.com/answers/49310/field-extraction-in-message-field-of-windows-event-log.html

Here are the new props/transforms adapted for this source as per the link.

props.conf

[source::WinEventLog:InTrust for AD]
REPORT-MESSAGE = welitad-message, welitad-eq-kv, welitad-col-kv
KV_MODE=none
# Note the below  settings are effectively legacy, in place here to handle
# data coming from much much older forwarders (3.x & 4.x)
SHOULD_LINEMERGE = false
MAX_TIMESTAMP_LOOKAHEAD=30
LINE_BREAKER = ([\r\n](?=\d{2}/\d{2}/\d{2,4} \d{2}:\d{2}:\d{2} [aApPmM]{2}))
TRANSFORMS-FIELDS = strip-winevt-linebreaker

transforms.conf

[welitad-message]
REGEX = (?sm)^(?<_pre_msg>.+)\nMessage=(?<Message>.+)$
CLEAN_KEYS = false

[welitad-eq-kv]
SOURCE_KEY = _pre_msg
DELIMS     = "\n","="
MV_ADD     = true

[welitad-col-kv]
SOURCE_KEY = Message
REGEX      = \n?([^:\n\r]+):[ \t]++([^\n]*)
FORMAT     = $1::$2
MV_ADD     = true

Only change was to add the "?" for the REGEX to make newline optional. Is there anything wrong in the way this is getting implemented? Also, to make sure these AD logs pass through these new stanza's I also tried [source::WinEventLog:InTrust...] instead of [source::WinEventLog:InTrust for AD] but no change in results.

Note: Above changes were done in props/transforms for Search Head. /opt/splunk/etc/system/local

Any ideas on what could be going wrong?

Thanks,

~Abhi

[williamchenyp]

Hi @abhijittikekar,

I too am struggling with the field extraction regex for Change Auditor events. I can't tell if you made any more progress on this since March 2019, so i just want to add what I have to the knowledge base.

I was able to extract some fields, but only if the field value is a single word string with no spaces. I also have a different regex for the Message line because I did not have any luck getting my regex to work using \n. So below are the 4 fields that I managed to get working. The only catch is these regex still fails if the values for Attribute Name, Object Class or Client Computer contains a space....

In addition, the regex I used for Message is only applicable if Client Computer comes directly after it in the next line. This is true for almost all Change Auditor logs except for those rare ones where it's just a Message. Those I don't care for as they're more system level errors that's not applicable to what I'm using Change Auditor logs for.

FYI, I thought using \s\s help finding the \n and \t that comes between each fields/lines; figuring that carriage return and tab are technically a "space" character. However, this seem to just work to pull values that does not contain a space. 😞

[source::WinEventLog:InTrust for AD]
EXTRACT-signature = Message\=(?<signature>.+)\s+Client Computer :
EXTRACT-attribute_name = Attribute Name : (?<attribute_name>[^(\s\s)]+)
EXTRACT-object_class = Object Class : (?<object_class>[^(\s\s)]+)
EXTRACT-src = Client Computer : (?<src>[^(\s\s)]+)

Hope this is helpful to anyone else trying to do custom field extracts for Change Auditor logs.

Best,

Will

[jnahuelperez35]

Do you already try to create new field using REGEX? i used to catch the Logon Type. Which specified field you want to extract?

[att35]

Hi,

I want to extract the sub-fields that are part of the Message field content. e.g.
Message=Account locked out
Client Computer : xxxxxxx
Account DN : CN=,OU=,OU=,OU=,OU=,DC=x,DC=y,DC=z
Object Class : user
Object GUID : CN=,OU=,OU=,OU=,OU=,DC=,DC=,DC= Request ID : {0637DFC03183}

Here everything is the value of "Message" field. We would like them to be their own field:value pair, e.g.
Message=Account locked out
Client Computer = xxxxxxx
Account DN = CN=,OU=,OU=,OU=,OU=,DC=x,DC=y,DC=z

Splunk does this by default for other known WinEventLog sources, e.g. Security or Application. but it doesn't parse the Message value in this particular case so we end up with the entire Message content dumped into one single value.

~ Abhi

[att35]

We worked with Splunk support and according to them, the extractions mentioned here work as expected for them, but they were not able to isolate the issue why it would not work in our environment.

one of the suggestions was this: "sourcetype "In trust " does not exist on your search head."
To be honest, I am not sure what that means. We can clearly see the sourcetype for searches done on SH and the data source is no different than say WinEventLog:Security. This is how we have configured the forwarders to collect this data stream:

[WinEventLog://InTrust for AD]
disabled = 0
evt_resolve_ad_obj = 1

They also think it could be some other extractions that are conflicting with these when it comes to [source::WinEventLog:InTrust...]

Is there any troubleshooting method that I can use to see the exact sequence of config stanza's Splunk follows for a particular event or a sourcetype?

Any troubleshooting ideas on those two suggestions would be highly appreciated.

Thanks,

~ Abhi

[att35]

seems like i've exhausted all options here.. 😞

came across several suggestions for extracting these sub-fields from original Message but none work.. Any ideas?

Is there a way to find out what extractions are being used for a specific sourcetype? maybe I am not making changes at the correct location..

[att35]

following search time extraction works to some extent.. but only for extracting the Message part until .

sourcetype="WinEventLog:InTrust for AD" | rex field=Message "(?<msg>[^\.\n]+)"

Although for the values where "." is missing this fails and ends up adding other data until it reaches a "."

Based on the log sample above, is there a easy way to extract all the sub-fields during search time? In another question someone had suggested to use the following but that didn't extract any fields. Is this not application in current scenario?

| extract kvdelim=" : " pairdelim="\n"