Getting Data In

How to store the urldecode data indexing using heavy forwarder to indexer?

leeyounsoo
Path Finder

I want to store the urlencoded data on the server, install the heavy forwarder on the server, urldecoding the data (encoded original data) from the installed heavy forwarder, and then transfer it to the indexer.

Is there a way? Or should I preprocess it?

data(encoding) ----------> server | [encoding data] -----(indexing)-----> [heavy forwarder] | --------(URL decoding data)---------> indexer

0 Karma
1 Solution

xpac
SplunkTrust
SplunkTrust

There is no built-in mechanism in Splunk that allows you to urldecode() before writing to an index, so you can't easily manipulate it like this.

You can either stick with the "Decode during search time" approach, but that might making fast searches impossible because the data is simply written to index encoded.
Preprocessing would mean running a scripted input, or something like this. The script would have to ingest the data, urldecode it, and then output it, so Splunk gets the proper data. If the encoded data is important to your use case, that's the way I would go. 🙂

View solution in original post

0 Karma

xpac
SplunkTrust
SplunkTrust

There is no built-in mechanism in Splunk that allows you to urldecode() before writing to an index, so you can't easily manipulate it like this.

You can either stick with the "Decode during search time" approach, but that might making fast searches impossible because the data is simply written to index encoded.
Preprocessing would mean running a scripted input, or something like this. The script would have to ingest the data, urldecode it, and then output it, so Splunk gets the proper data. If the encoded data is important to your use case, that's the way I would go. 🙂

0 Karma

leeyounsoo
Path Finder

I found that there was no way to do it in splunk without the preprocessing, but I was worried about different ways because the customer gave me the answer "I can not decode the data".

If preprocessing is performed, real-time processing may not be possible (due to environmental or structural reasons).

your answer made it easy to explain the problem to the customer.

We are trying to solve it through consultation.

thank you for answer

Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...