How can you rewrite log data to overwrite sensitive information?

[jonzatlmi]

If there were a field that one wanted to overwrite, say it was an API token for example, and it had already been logged in Splunk.

Deleting may not be an option, how about overwriting that field with something like 'xxx'?

Thanks in advance!

[PavelP · Accepted solution]

as @darrenfuller mentioned, there is no way to modify already indexed data.

You options:

• using an appropriate SPL filter "search index=xxx sourcetype=yyyy token=regex" export events to file including the timestamp.
• using a data input wizard find the correct configuration so Splunk can read the timestamp correctly and test the SEDCMD to delete or anonymize tokens. You need to apply the same sourcetype and host. Rewrite the source using conf files. Don't import anything yet!
  • if you 100% sure you tested it right, delete events using the previous filter with delete command ""search index=xxx sourcetype=yyyy token=regex | delete"
  • now import the previously exported events with the prepared configuration (original timestamp + SEDCMD). The only difference will be the different index_time and splunk_server fields.

good luck!

[jonzatlmi]

/answers/204549 - so this is something completely different then, rewriting host field in this example question?

[PavelP]

rewriting with DEST_KEY is possible only during the index phase:

DEST_KEY = <KEY>
* NOTE: This setting is only valid for index-time field extractions.
* Specifies where Splunk software stores the expanded FORMAT results in
  accordance with the REGEX match.
* Required for index-time field extractions where WRITE_META = false or is
  not set.
* For index-time extractions, DEST_KEY can be set to a number of values
  mentioned in the KEYS section at the bottom of this file.
  * If DEST_KEY = _meta (not recommended) you should also add $0 to the
    start of your FORMAT setting.  $0 represents the DEST_KEY value before
    Splunk software performs the REGEX (in other words, _meta).
    * The $0 value is in no way derived *from* the REGEX match. (It
      does not represent a captured group.)
* KEY names are case-sensitive, and should be used exactly as they appear in
  the KEYs list at the bottom of this file. (For example, you would say
  DEST_KEY = MetaData:Host, *not* DEST_KEY = metadata:host .)

[jonzatlmi]

thank you!

[darrenfuller]

Alas, no. Once the data is indexed, you are scrogged. The raw data can't be changed to mask sensitive data.

[jonzatlmi]

so then the next best is a a tight search that pipes into delete command it would seem. Do you think there are alternatives?

I was curious if props or transforms would show up as a suggestion.