SEDCMD - Overwrite entire field

[benjeffers2501]

Hi,

I currently have Windows Event Logs ingesting, they are all being rendered as XML. Logs are being parsed at the indexer, no HF involvement.

I have Windows TA 8.4.0 installed and pushed to all indexers, and this I know comes with default SEDCMD commands in the default props.conf file.

What I am trying to acheive is to entirely overwrite the 'Message' field of XmlWinEventLog:Security logs with a blank field. This is to reduce license consumption, as the majority of the content within the message field is already denoted previously in the same log and is essentially just duplicating content.

Anyway, have transferred the relevant SEDCMD lins to a local props.conf file however the filters did not work, even after pushing. I believe this is because the logs are in an XML format and not the native format, however I am happy to be corrected there if I am wrong.

The current config file I am running in local/props.conf is as follows:

[source::WinEventLog:Security]
SEDCMD-windows_security_event_formater = s/(?m)(^\s+[^:]+\:)\s+-?$/\1/g
SEDCMD-windows_security_event_formater_null_sid_id = s/(?m)(:)(\s+NULL SID)$/\1/g s/(?m)(ID:)(\s+0x0)$/\1/g
SEDCMD-cleansrcip = s/(Source Network Address:    (\:\:1|127\.0\.0\.1))/Source Network Address:/
SEDCMD-cleansrcport = s/(Source Port:\s*0)/Source Port:/
SEDCMD-remove_ffff = s/::ffff://g
SEDCMD-clean_info_text_from_winsecurity_events_certificate_information = s/Certificate information is only[\S\s\r\n]+$//g
SEDCMD-clean_info_text_from_winsecurity_events_token_elevation_type = s/Token Elevation Type indicates[\S\s\r\n]+$//g
SEDCMD-clean_info_text_from_winsecurity_events_this_event = s/This event is generated[\S\s\r\n]+$//g

#For XmlWinEventLog:Security
SEDCMD-cleanxmlsrcport = s/<Data Name='IpPort'>0<\/Data>/<Data Name='IpPort'><\/Data>/
SEDCMD-cleanxmlsrcip = s/<Data Name='IpAddress'>(\:\:1|127\.0\.0\.1)<\/Data>/<Data Name='IpAddress'><\/Data>/
SEDCMD-cleanxmlseclogs = s/<Message>[\S\s\r\n]+<\/Message>/<Message></Message>

I have left some of the default lines in for WinEventLog:Security for no other reason that just to test.

I have added the cleanxmlseclogs line at the end. It is here I am trying to detect the whole Message field and then overwrite with just the headers, so that the content of the field gets dropped.

Can anyone assist with where I am going wrong here?

[benjeffers2501]

One thing I am unsure of, within my Splunk environment the sourcetype is 'XmlWinEventLog' and the source is XmlWinEventLog:Security.

Will this change the config I need in props.conf? I read another article saying it should remain as [source::WinEventLog:Security] in the props file but I'm wondering if it requires the Xml prefix?

[richgalloway]

Use the XmlWinEventLog sourcetype.

[benjeffers2501]

Hi Rich,

Thank you for responding so quickly.

Just to confirm should I use

[XmlWinEventLog]

or

[source::XmlWinEventLog:Security]

in the props.conf file?

[richgalloway]

Yes, use the source specifier.  Sorry for the confusion.

[benjeffers2501]

You're spot on it was missing the '/' however it didn't correct the issue.

I have tried to replicate the command above it and using the following now:

SEDCMD-cleanxmlseclogs = s/<Message>[\S\s\r\n]+<\/Message>/<Message>'null'</Message>

This is in an attempt to overwrite the field with the string 'null' as the empty field doesn't seem to be playing ball.

[richgalloway]

It may just be a typo when writing the question, but the last SEDCMD needs a closing slash.

SEDCMD-cleanxmlseclogs = s/<Message>[\S\s\r\n]+<\/Message>/<Message></Message>/