Splunk Search

Extract fields from Windows Event Log Message

dw385
Explorer

I have events that do not extract the fields from the message field by default. I'm trying to setup props/transforms to auto extract the fields on search. I believe I have the props and transforms setup properly however it's not working. I think the issue is the regex I'm using and possibly the first 2 lines of the message data.

Props:

[source::WinEventLog:Directory Service]
REPORT-dsfields = ds_fields

Transforms:

[ds_fields]
SOURCE_KEY = Message
REGEX = ([^:]+):\n(.*[\S])
FORMAT = $1::$2
MV_ADD = true
KEEP_EMPTY_VALS = true

The regex appears to extract the fields properly with the exception of the first line which I don't need. I also tried this regex

(?:.*\n){2}([^:]+):\n(.*[\S])

Which also appears to work to exclude the first two lines. However the combination doesn't extract fields when searching. If I extract a single value in the same props and transforms it works so I believe the files are configured properly.

I don't care if the first line (Internal Event:) is extracted or not. The goal is to extract the field value pairs in the rest of the message.

The message field data looks like:

Internal event: A client issued a search operation with the following options. 

Client:
10.215.215.215:61759 
Starting node:
DC=domain,DC=Domain,DC=Domain,DC=local 
Filter:
 (accountNameHistory=*[DG]*)  
Search scope:
subtree 
Attribute selection:
accountNameHistory,employeeType,title,info,groupType,distinguishedName,objectGUID,sAMAccountType,departmentNumber,memberOf,objectSid,objectClass
Server controls:

Visited entries:
1556 
Returned entries:
8 
Used indexes:
Ancestors_index:3356:N; 
Pages referenced:
16745 
Pages read from disk:
31 
Pages preread from disk:
9 
Clean pages modified:
0 
Dirty pages modified:
0 
Search time (ms):
313 
Attributes Preventing Optimization:
accountNameHistory  
User:
Domain\some-user-account
Tags (1)
0 Karma

adonio
Ultra Champion

hello there,
the configurations you need are pre-built and packaged nicely on the Splunk_TA_microsoft_ad
download here: https://splunkbase.splunk.com/app/3207/
install it on the forwarder of your domain controller and you can remove the props and transforms
read more here:
http://docs.splunk.com/Documentation/MSApp/1.4.1/MSInfra/WhataSplunkAppforWindowsInfrastructuredeplo...
and here:
http://docs.splunk.com/Documentation/MSApp/1.4.1/MSInfra/ConfigureActiveDirectoryauditpolicy
hope it helps

0 Karma

dw385
Explorer

I have that deployed to my DCs and on the SHs. The data is collected but some of the logs in Directory Services is in a header over value format. It seems none of values are extracted properly (not just for this particular event I need). I don't think I see anything in the AD TA that seems to suggest there is anything for extracting the fields from the message.

0 Karma

adonio
Ultra Champion

do you have the TA on the indexer as well?

0 Karma

dw385
Explorer

Yes, it's on the indexers as well. Latest version (1.0) with no customization to it. This is the same for the Windows TA as well, although not the latest version, I am a release or two behind on that one.

0 Karma

adonio
Ultra Champion

can you verify the source of the data you are trying to transform?
i see in the exchange app i have access to that the source is "Directory_Service" and not
"Directory Service"
in any case, its odd, i see all extractions working fine on the environment i have access to
hope it helps

0 Karma

Richfez
SplunkTrust
SplunkTrust

I changed the text of the event code to "code" so the asterisks won't be converted to italics. It did not change the carriage returns, but I don't know if this is the way it is supposed to be. Can you confirm it still looks like the event as you see it in the original place?

Thanks!

0 Karma

dw385
Explorer

Thank you, the example looks correct.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...