How to extract Windows fields at search time using regex?

[kiran331]

How to extract the Account Name and other fields in the description field from the below windows event from azure? It has both JOSN and XMl data in JSON event. Attahced are RAw event and JSON event. Please advise.

[cpetterborg]

Try the following:

<yoursearch> | rex field=description "Security ID:\s+(?P<securityid>.*)[\s\S]+?Account Name:\s+(?P<accountname>.*)[\s\S]+?Account Domain:\s+(?P<accountdomain>.*)[\s\S]+?Login ID:\s+(?P<loginid>.*)[\s\S]+?Provider Name:\s+(?P<providername>.*)[\s\S]+?Algorithm Name:\s+(?P<algname>.*)[\s\S]+?Key Type:\s+(?P<keytype>.*)[\s\S]+?Operation:[\s\S]+?Operation:\s+(?P<operation>.*)[\s\S]+?Return Code:\s+(?P<returncode>.*)"

Since you provided a screenshot, I'm not sure if I got all the spacing, etc. right, but it should work for you.

[knalla]

Thanks cpetterborg! its working, is there a way I can use splunk add-on for windows for these logs OR how can I parse them at indexing time?

[cpetterborg]

I'm not sure if you can use them with the Splunk Add-On for Windows or not.

If you really want to do the field extractions at index time (best practice is to do it at search time), then you could put something like this in your transforms.conf file:

[my_extraction]
REGEX = Security ID:\s+(?P<securityid>.*)[\s\S]+?Account Name:\s+(?P<accountname>.*)[\s\S]+?Account Domain:\s+(?P<accountdomain>.*)[\s\S]+?Login ID:\s+(?P<loginid>.*)[\s\S]+?Provider Name:\s+(?P<providername>.*)[\s\S]+?Algorithm Name:\s+(?P<algname>.*)[\s\S]+?Key Type:\s+(?P<keytype>.*)[\s\S]+?Operation:[\s\S]+?Operation:\s+(?P<operation>.*)[\s\S]+?Return Code:\s+(?P<returncode>.*)
FORMAT = securityid::"$1" accountname::"$2" accountdomain::"$3" loginid::"$4" providername::"$5" algname="$6" keytype="$7" operation::"$8" returncode::"$9"
WRITE_META = true

and this in the props.conf:

[my_source_type]
TRANSFORM-extract-fields = my_extraction

and this in fields.conf:

[securityid]
INDEXED=true
[accountname]
INDEXED=true
[accountdomain]
INDEXED=true
[loginid]
INDEXED=true
[providername]
INDEXED=true
[algname]
INDEXED=true
[keytype]
INDEXED=true
[operation]
INDEXED=true
[returncode]
INDEXED=true

[cpetterborg]

What is the outcome of your efforts?

[aholzer]

If you are looking to do this at search time I suggest looking into the spath command. Link to documentation here.

If you are looking to do this automatically, then you should be able to follow the instructions on this answer.

Hope these help

[cpetterborg]

Are you looking to do this at index time, or search time?

[kiran331]

At search time I need to search for previous events and also I have to add them at indexing data for new data