Getting Data In

How do I remove field data before indexing?

SathyaNarayanan
Path Finder

Hi Team,

I need to remove a field data before the data gets indexed to Splunk.

The field that needs to be removed is in between the log and it has several lines.

sample data:

dv_u_business_duration_in_seconds="",closed_by="",sys_created_by="",sys_domain_path="/",dv_business_duration="",dv_work_notes="2018-08-05 09:46:09 - *******



active="true",dv_made_sla="true",additional_assignee_list="",delivery_task=""

field name in work_Notes

Note: I tried the null

In Props.conf

[sourcetype:test_task]
TRANSFORMS-null= tasknull

transforms.conf

[tasknull]
FIELDS = work_Notes
DEST_KEY = queue
FORMAT = nullQueue

Thanks in advance.

0 Karma

mstjohn_splunk
Splunk Employee
Splunk Employee

hi @SathyaNarayanan

Did the answer below solve your problem? If so, please resolve this post by approving it!
If your problem is still not solved, keep us updated so that someone else can help ya. Thanks for posting!

0 Karma

ddrillic
Ultra Champion

These name value pairs are extracted at search time, not index time.

Btw, props.conf might have been -

[<sourcetype name>]
TRANSFORMS = tasknull

Suppressing the name value pairs extraction at search time can be done via the explanations at Disable Automatic Field Extraction or How to avoid extracting fields from quoted values?

Unfortunately, it requires disabling the extractions by KV_MODE=none in props.conf in the search heads and creating your own extractions.

0 Karma

493669
Super Champion

You can replace the other portion of the events with blank during index time, so only the necessary information gets logged.
http://docs.splunk.com/Documentation/Splunk/6.2.4/Data/Anonymizedatausingconfigurationfiles#Anonymiz...

0 Karma
Get Updates on the Splunk Community!

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...

Brains, Bytes, and Boston: Learn from the Best at .conf25

When you think of Boston, you might picture colonial charm, world-class universities, or even the crack of a ...