Getting Data In

Filter/modify data prior to ingestion?

BogeyMan
Loves-to-Learn Lots

Not sure this is even possible, but I'll ask anyway...

I have application(s) that are sending JSON data into Splunk, for example:

{
 "key1": "value1",
 "key2": "value2",
 "key3": "value3",
 "key4": "value1"
}

As you can see, the value for "key4" is the same as "key1".

So, in my example, I don't want to ingest the complete JSON payload, but only:

{
 "key1": "value1",
 "key2": "value2",
 "key3": "value3"
}

 

Can this be done?

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Manipulating structured data on ingest is best done by external means (particular method would depend on the use case). In some limited cases you probably could do some regex magic to remove some parts of incoming events but this solution would be highly sensitive to data format.

Oh, and I'm assuming you want to just remove key4, not dynamically decide which one to drop.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Watch On-Demand   Replay the Accelerate ...

Splunk Agent Observability: Measuring AI Value with Splunk Tokenomics on AWS

The AI landscape is moving faster than ever, and our latest 2026 State of Observability report provides the ...

Enterprise Security Watch Party: What’s New in 8.8 - Recap

Splunk Community HQ hosted a Splunk Enterprise Security Watch Party: What’s New In ES 8.8 last Thursday. I had ...