topic Re: Remove ::ffff: from logs in Getting Data In

Remove ::ffff: from logs

diegosainz — Tue, 04 Jun 2013 17:10:52 GMT

I am looking to remove the ::ffff: from Windows event logs:

Network Information:
Client Address: ::ffff:XX.XX.XX.XX
Client Port: 51806

Any assistance would be appreciated.

Re: Remove ::ffff: from logs

Gilberto_Castil — Tue, 04 Jun 2013 20:33:05 GMT

There is a command called REX that can be used at search time, or SEDCMD that can be used at index time. This can be used to replace the string in question. You should do this only if you are sure that you do not need the data.

You can do this at search time:

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

Or, you can do this at index time by setting an entry in props.conf.

props.conf

[answers-1370377923]
SEDCMD-remove_ffff = s/::ffff://g

I hope this helps,

-gc

Re: Remove ::ffff: from logs

ryanoconnor — Tue, 29 Sep 2020 08:43:04 GMT

I would also recommend here that this be done with a simple modification to Splunk_TA_Windows.

In order to remove the ::ffff: from this field, you can create two new transforms and modify two extractions in the Splunk_TA_Windows. You need two because the Client_Address field is used for both src and src_ip in the Windows logs.

Instructions are below:

Transformation for src_ip:

Start by making a new transformation
Set the Name to: Client_Address_as_src_ip_modified
Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
Set the Source_Key to: Client_Address
Set the Format to: src_ip::"$2"
Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
Go to Settings > Fields > Field extractions
Find and modify the extraction named “source::*:Security : REPORT-src_ip_for_windows_security”
Set the Extraction/Transform to “Source_Network_Address_as_src_ip,Client_Address_as_src_ip_modified”

Transformation for src:

Making another new transformation
Set the Name to: Client_Address_as_src_modified
Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
Set the Source_Key to: Client_Address
Set the Format to: src::”$2"
Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
Go to Settings > Fields > Field extractions
Find and modify the extraction named “source::*:Security : REPORT-src_for_windows_security”
Set the Extraction/Transform to “Source_Workstation_as_src,Workstation_Name_as_src,Caller_Machine_Name_as_src,Client_Machine_Name_as_src,Source_Network_Address_as_src,Client_Address_as_src_modified,ComputerName_as_src”

Re: Remove ::ffff: from logs

agadayev — Tue, 29 Sep 2020 09:43:26 GMT

Hi, thank you this fix worked for me. Just to clarify as I am new to Splunk the Index time props.conf you are referring to is located at: $SPLUNK_HOME/etc/system/local. And the sourcetype asked about in this case is Windows Event Logs so my stanza looks like this:
[WinEventLog]
SEDCMD-remove_ffff = s/::ffff://g
I realized that this works only after Splunk has been restarted.
Regards,

Re: Remove ::ffff: from logs

lmedina — Tue, 11 Oct 2016 22:26:24 GMT

Hello,

I am trying to get the same fixed and am relatively new to Splunk as well... I was wondering if this change should be done at the indexer, forwarder or search head level?

Please advise at your convenience.

Thank you!

Re: Remove ::ffff: from logs

Gilberto_Castil — Wed, 12 Oct 2016 01:12:48 GMT

Hi there - If you are doing this permanently, then it is done at index time on your indexer layer. In that case, you will configure this via the props.conf entry.

 #props.conf
 [answers-1370377923]
 SEDCMD-remove_ffff = s/::ffff://g

See the docs. (Look for SEDCMD)

If this is a general context obfuscation, where the end result is presented as a non-drillable component, then it can be done at search time - it would just be part of your search syntax.

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

See the docs.

Re: Remove ::ffff: from logs

gf13579 — Fri, 09 Mar 2018 05:47:32 GMT

Use a source key of IpAddress if you're ingesting those logs as sourcetype XmlWinEventLog:Security

Re: Remove ::ffff: from logs

FrankVl — Fri, 09 Mar 2018 11:20:06 GMT

Be very careful with using this sample rex or SEDCMD, as it will also blow away this string inside a perfectly valid ipv6 address (e.g 2001:1337::ffff:1234:1). You probably want to adjust the regex such that it only strips the ::ffff: part when it occurs as a prefix to an ipv4 address.

Re: Remove ::ffff: from logs

ozatsepin — Tue, 29 Sep 2020 22:13:32 GMT

On Splunk Enterprise 7.2.1 with Splunk Add-on for Microsoft Windows 5.0.1 I solved problem in the following way:

Create file $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local/transforms.conf.

[Client_Address_as_src]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src::"$2"

[Client_Address_as_src_ip]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src_ip::"$2"

Re: Remove ::ffff: from logs

jwalzerpitt — Wed, 14 Aug 2019 13:47:14 GMT

This worked perfectly for me - thx for posting

Re: Remove ::ffff: from logs

maraman_splunk — Wed, 30 Sep 2020 01:43:46 GMT

the SEDCMD-remove_ffff is already present and commented in Splunk_TA_windows version 6.0 (but dont change in default file)

so you could just :
create/update Splunk_TA_Windows/local/props.conf

[source::WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g

[source::WinEventLog:ForwardedEvents]
SEDCMD-remove_ffff = s/::ffff://g

[WMI:WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g