Getting Data In

Remove ::ffff: from logs

diegosainz
Path Finder

I am looking to remove the ::ffff: from Windows event logs:

Network Information:
Client Address: ::ffff:XX.XX.XX.XX
Client Port: 51806

Any assistance would be appreciated.

1 Solution

Gilberto_Castil
Splunk Employee
Splunk Employee

There is a command called REX that can be used at search time, or SEDCMD that can be used at index time. This can be used to replace the string in question. You should do this only if you are sure that you do not need the data.

You can do this at search time:

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

Or, you can do this at index time by setting an entry in props.conf.

props.conf

[answers-1370377923]
SEDCMD-remove_ffff = s/::ffff://g

I hope this helps,

-gc

View solution in original post

maraman_splunk
Splunk Employee
Splunk Employee

the SEDCMD-remove_ffff is already present and commented in Splunk_TA_windows version 6.0 (but dont change in default file)

so you could just :
create/update Splunk_TA_Windows/local/props.conf

[source::WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g

[source::WinEventLog:ForwardedEvents]
SEDCMD-remove_ffff = s/::ffff://g

[WMI:WinEventLog:Security]
SEDCMD-remove_ffff = s/::ffff://g

ozatsepin
Engager

On Splunk Enterprise 7.2.1 with Splunk Add-on for Microsoft Windows 5.0.1 I solved problem in the following way:

Create file $SPLUNK_HOME/etc/apps/Splunk_TA_windows/local/transforms.conf.

[Client_Address_as_src]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src::"$2"

[Client_Address_as_src_ip]
SOURCE_KEY = Client_Address
REGEX = ([\\]+)?([^f:\n][^-].*)
FORMAT = src_ip::"$2"

jwalzerpitt
Motivator

This worked perfectly for me - thx for posting

0 Karma

ryanoconnor
Builder

I would also recommend here that this be done with a simple modification to Splunk_TA_Windows.

In order to remove the ::ffff: from this field, you can create two new transforms and modify two extractions in the Splunk_TA_Windows. You need two because the Client_Address field is used for both src and src_ip in the Windows logs.

Instructions are below:

Transformation for src_ip:

  1. Start by making a new transformation
  2. Set the Name to: Client_Address_as_src_ip_modified
  3. Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
  4. Set the Source_Key to: Client_Address
  5. Set the Format to: src_ip::"$2"
  6. Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
  7. Go to Settings > Fields > Field extractions
  8. Find and modify the extraction named “source::*:Security : REPORT-src_ip_for_windows_security”
  9. Set the Extraction/Transform to “Source_Network_Address_as_src_ip,Client_Address_as_src_ip_modified”

Transformation for src:

  1. Making another new transformation
  2. Set the Name to: Client_Address_as_src_modified
  3. Set the Regular expression to: ([\]+)?([^f:\n][^-].*)
  4. Set the Source_Key to: Client_Address
  5. Set the Format to: src::”$2"
  6. Save the extraction. Note: Make sure the permissions for this are Global and also that the transformation goes into the Splunk_TA_Windows App.
  7. Go to Settings > Fields > Field extractions
  8. Find and modify the extraction named “source::*:Security : REPORT-src_for_windows_security”
  9. Set the Extraction/Transform to “Source_Workstation_as_src,Workstation_Name_as_src,Caller_Machine_Name_as_src,Client_Machine_Name_as_src,Source_Network_Address_as_src,Client_Address_as_src_modified,ComputerName_as_src”

gf13579
Communicator

Use a source key of IpAddress if you're ingesting those logs as sourcetype XmlWinEventLog:Security

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

There is a command called REX that can be used at search time, or SEDCMD that can be used at index time. This can be used to replace the string in question. You should do this only if you are sure that you do not need the data.

You can do this at search time:

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

Or, you can do this at index time by setting an entry in props.conf.

props.conf

[answers-1370377923]
SEDCMD-remove_ffff = s/::ffff://g

I hope this helps,

-gc

FrankVl
Ultra Champion

Be very careful with using this sample rex or SEDCMD, as it will also blow away this string inside a perfectly valid ipv6 address (e.g 2001:1337::ffff:1234:1). You probably want to adjust the regex such that it only strips the ::ffff: part when it occurs as a prefix to an ipv4 address.

0 Karma

agadayev
Path Finder

Hi, thank you this fix worked for me. Just to clarify as I am new to Splunk the Index time props.conf you are referring to is located at: $SPLUNK_HOME/etc/system/local. And the sourcetype asked about in this case is Windows Event Logs so my stanza looks like this:
[WinEventLog]
SEDCMD-remove_ffff = s/::ffff://g
I realized that this works only after Splunk has been restarted.
Regards,

0 Karma

lmedina
New Member

Hello,

I am trying to get the same fixed and am relatively new to Splunk as well... I was wondering if this change should be done at the indexer, forwarder or search head level?

Please advise at your convenience.

Thank you!

0 Karma

Gilberto_Castil
Splunk Employee
Splunk Employee

Hi there - If you are doing this permanently, then it is done at index time on your indexer layer. In that case, you will configure this via the props.conf entry.

 #props.conf
 [answers-1370377923]
 SEDCMD-remove_ffff = s/::ffff://g

See the docs. (Look for SEDCMD)

--

If this is a general context obfuscation, where the end result is presented as a non-drillable component, then it can be done at search time - it would just be part of your search syntax.

sourcetype="answers-1370377923" | rex mode=sed "s/::ffff://g"

See the docs.

0 Karma
Get Updates on the Splunk Community!

How I instrumented a Rust application without knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...