topic Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding? in Getting Data In

Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vincentwhn — Sun, 14 Sep 2025 10:26:15 GMT

Due to privacy concerns, I would like to modify the _raw content during the data onboarding phase in order to implement a whitelist for JSON fields. Through testing, I found that I can achieve this using SEDCMD. However, SEDCMD applies to the entire sourcetype.

Is there any way to use SEDCMD while applying it only to specific events within same sourcetype?

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vjdev — Sun, 14 Sep 2025 11:00:59 GMT

Hello,

Hope below example help for you!

1st Option Transform.

[session-anonymizer]
REGEX = (?m)^(.*)SessionId=\w+(\w{4}[&"].*)$
FORMAT = $1SessionId=########$2
DEST_KEY = _raw

2nd option is Ingest actions.

If both not helpful, explain more about the requirement .

Thank you!

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vincentwhn — Sun, 14 Sep 2025 12:58:19 GMT

Thanks for your assist.

"Could you share more about Ingest actions?

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vjdev — Sun, 14 Sep 2025 14:02:52 GMT

Hello,

Mask using regex , Filter using regex and EVAL can be achievable.

This can help you lot.
https://help.splunk.com/en/splunk-cloud-platform/get-started/splunk-validated-architectures/getting-data-in-forwarding-and-preprocessing/ingest-actions-for-splunk-platform

Mask, redact, remove, or otherwise change raw data before indexing
Tag, add, lookup, or otherwise augment raw data before indexing

Thank You!

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

PickleRick — Sun, 14 Sep 2025 20:00:02 GMT

Don't use regexes to manipulate structured data. It will cause pain sooner or later.

Use a tool which can understand the json structure. If you're on Splunk 10 you can use Edge Processor.

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vincentwhn — Mon, 15 Sep 2025 01:56:26 GMT

I am still using Splunk 9.3.......

Can you share more about Edge Processor? Thanks.

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

PickleRick — Mon, 15 Sep 2025 07:32:36 GMT

https://www.splunk.com/en_us/blog/platform/introducing-edge-processor-for-splunk-enterprise-data-management-on-your-premises.html

Re: Are there any ways to whitelist specific fields from json format logs during data-onboarding?

vincentwhn — Mon, 15 Sep 2025 07:47:49 GMT

Thanks a lot. Will take a look at it.