Getting Data In

Remove ::ffff: from logs

diegosainz
Path Finder

I am looking to remove the ::ffff: from Windows event logs:

Network Information:
Client Address: ::ffff:XX.XX.XX.XX
Client Port: 51806

Any assistance would be appreciated.

1 Solution

Gilberto_Castil
Splunk Employee
Splunk Employee

There is a command called REX that can be used at search time, or SEDCMD that can be used at index time. This can be used to replace the string in question. You should do this only if you are sure that you do not need the data.

You can do this at search time:

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

Or, you can do this at index time by setting an entry in props.conf.

props.conf

[answers-1370377923]
SEDCMD-remove_ffff = s/::ffff://g

I hope this helps,

-gc

View solution in original post

maraman_splunk
Splunk Employee
Splunk Employee

the SEDCMD-remove_ffff is already present and commented in Splunk_TA_windows version 6.0 (but dont change in default file)

so you could just :
create/update Splunk_TA_Windows/local/props.conf

[source::WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g

[source::WinEventLog:ForwardedEvents]
SEDCMD-remove_ffff = s/::ffff://g

[WMI:WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g

ozatsepin
Engager

On Splunk Enterprise 7.2.1 with Splunk Add-on for Microsoft Windows 5.0.1 I solved problem in the following way:

Create file $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local/transforms.conf.

[Client_Address_as_src]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src::"$2"

[Client_Address_as_src_ip]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src_ip::"$2"

jwalzerpitt
Motivator

This worked perfectly for me - thx for posting

0 Karma

ryanoconnor
Builder

I would also recommend here that this be done with a simple modification to Splunk_TA_Windows.

In order to remove the ::ffff: from this field, you can create two new transforms and modify two extractions in the Splunk_TA_Windows. You need two because the Client_Address field is used for both src and src_ip in the Windows logs.

Instructions are below:

Transformation for src_ip:

  1. Start by making a new transformation
  2. Set the Name to: Client_Address_as_src_ip_modified
  3. Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
  4. Set the Source_Key to: Client_Address
  5. Set the Format to: src_ip::"$2"
  6. Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
  7. Go to Settings > Fields > Field extractions
  8. Find and modify the extraction named “source::*:Security : REPORT-src_ip_for_windows_security”
  9. Set the Extraction/Transform to “Source_Network_Address_as_src_ip,Client_Address_as_src_ip_modified”

Transformation for src:

  1. Making another new transformation
  2. Set the Name to: Client_Address_as_src_modified
  3. Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
  4. Set the Source_Key to: Client_Address
  5. Set the Format to: src::”$2"
  6. Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
  7. Go to Settings > Fields > Field extractions
  8. Find and modify the extraction named “source::*:Security : REPORT-src_for_windows_security”
  9. Set the Extraction/Transform to “Source_Workstation_as_src,Workstation_Name_as_src,Caller_Machine_Name_as_src,Client_Machine_Name_as_src,Source_Network_Address_as_src,Client_Address_as_src_modified,ComputerName_as_src”

gf13579
Communicator

Use a source key of IpAddress if you're ingesting those logs as sourcetype XmlWinEventLog:Security

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

There is a command called REX that can be used at search time, or SEDCMD that can be used at index time. This can be used to replace the string in question. You should do this only if you are sure that you do not need the data.

You can do this at search time:

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

Or, you can do this at index time by setting an entry in props.conf.

props.conf

[answers-1370377923]
SEDCMD-remove_ffff = s/::ffff://g

I hope this helps,

-gc

FrankVl
Ultra Champion

Be very careful with using this sample rex or SEDCMD, as it will also blow away this string inside a perfectly valid ipv6 address (e.g 2001:1337::ffff:1234:1). You probably want to adjust the regex such that it only strips the ::ffff: part when it occurs as a prefix to an ipv4 address.

0 Karma

agadayev
Path Finder

Hi, thank you this fix worked for me. Just to clarify as I am new to Splunk the Index time props.conf you are referring to is located at: $SPLUNK_HOME/etc/system/local. And the sourcetype asked about in this case is Windows Event Logs so my stanza looks like this:
[WinEventLog]
SEDCMD-remove_ffff = s/::ffff://g
I realized that this works only after Splunk has been restarted.
Regards,

0 Karma

lmedina
New Member

Hello,

I am trying to get the same fixed and am relatively new to Splunk as well... I was wondering if this change should be done at the indexer, forwarder or search head level?

Please advise at your convenience.

Thank you!

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

Hi there - If you are doing this permanently, then it is done at index time on your indexer layer. In that case, you will configure this via the props.conf entry.

 #props.conf
 [answers-1370377923]
 SEDCMD-remove_ffff = s/::ffff://g

See the docs. (Look for SEDCMD)

--

If this is a general context obfuscation, where the end result is presented as a non-drillable component, then it can be done at search time - it would just be part of your search syntax.

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

See the docs.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...