Ingest actions and base64 decode

[_olivier_]

Hi splunkers,

I need to decode base64 fields before indexing them.

I found a very old post with no good proposal for this need : Solved: Re: Anybody have an idea for base64 decoding of fi... - Splunk Community

Does anybody has set base64 decode , maybe with ingest actions ?

Thank you.

[tscroggins]

Hi @_olivier_,

Search-time base64 decoding is common, but depending on the context and underlying type of the encoded data, it may be possible to decode it at index-time using transforms, rulesets (ingest actions), and creative sequencing or routing.

How is the field data represented in _raw? Indexed (cooked) extractions? JSON? Key-value pairs? Delimiter and position?

What does the base64 octet stream represent? UTF-8 or another Unicode variant? CP437? ASCII? 8859-1? An EBCDIC variant? Binary data? An arbitrary MIME type? More than one of the preceding options?

@livehybrid's suggestion is a good one, and there are several examples of creative use of unarchive_cmd here on the community, but this would be a batch-like option. Scripted and modular inputs can also do the work.

[livehybrid]

Hi @_olivier_ 

Im not aware of a way to do this using Ingest Action (or INGEST_EVAL) specifically, but you could explore the unarchive_cmd in props.conf (https://help.splunk.com/en/splunk-enterprise/administer/admin-manual/10.0/configuration-file-referen...) which is intended to pass the data into a script/file as its ingested and can be used to unarchive tarballs (for example) but you could use a simple python script to take the input and base64 decode it, returning the decoded data to stdout for ingestion.

Is it the entire event that is base64?

You might also want to check out https://community.splunk.com/t5/Getting-Data-In/What-s-the-trick-to-get-unarchive-cmd-to-work-for-a-...

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing