Getting Data In

How to store the urldecode data indexing using heavy forwarder to indexer?

leeyounsoo
Path Finder

I want to store the urlencoded data on the server, install the heavy forwarder on the server, urldecoding the data (encoded original data) from the installed heavy forwarder, and then transfer it to the indexer.

Is there a way? Or should I preprocess it?

data(encoding) ----------> server | [encoding data] -----(indexing)-----> [heavy forwarder] | --------(URL decoding data)---------> indexer

0 Karma
1 Solution

xpac
SplunkTrust
SplunkTrust

There is no built-in mechanism in Splunk that allows you to urldecode() before writing to an index, so you can't easily manipulate it like this.

You can either stick with the "Decode during search time" approach, but that might making fast searches impossible because the data is simply written to index encoded.
Preprocessing would mean running a scripted input, or something like this. The script would have to ingest the data, urldecode it, and then output it, so Splunk gets the proper data. If the encoded data is important to your use case, that's the way I would go. 🙂

View solution in original post

0 Karma

xpac
SplunkTrust
SplunkTrust

There is no built-in mechanism in Splunk that allows you to urldecode() before writing to an index, so you can't easily manipulate it like this.

You can either stick with the "Decode during search time" approach, but that might making fast searches impossible because the data is simply written to index encoded.
Preprocessing would mean running a scripted input, or something like this. The script would have to ingest the data, urldecode it, and then output it, so Splunk gets the proper data. If the encoded data is important to your use case, that's the way I would go. 🙂

0 Karma

leeyounsoo
Path Finder

I found that there was no way to do it in splunk without the preprocessing, but I was worried about different ways because the customer gave me the answer "I can not decode the data".

If preprocessing is performed, real-time processing may not be possible (due to environmental or structural reasons).

your answer made it easy to explain the problem to the customer.

We are trying to solve it through consultation.

thank you for answer

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...