Splunk ServiceNow app: how to Selectively remove Display value from indexing?

[koshyk]

hi
After recently upgrading to new version of ServiceNow app, we have found that the display value (dv_*) field have automatically been enabled. This makes each event/payload quite huge and makes them cumbersome to investigate. Some of the display_value fields are quite useful.

We could see the workaround provided per the docs to disable dv_* fields entirely: https://docs.splunk.com/Documentation/AddOns/released/ServiceNow/Upgrade

So the question is
Can we selectively NOT index certain fields? (For example, we don't need fields like dv_approval_history from change_request table)

How to select the fields which are not required for indexing?

[koshyk · Accepted solution]

Found the answer. This can be done at inputs.conf

[snow://change_request]
exclude = description,dv_close_notes,dv_comments_and_work_notes, ......

and so on. unfortunately it have to be done manually as per my testing

[cybersecnutant]

Used this to drop the comments related fields. Longstanding tickets had more than 10,000 characters and would cause false negatives and/or crash browsers.

[efavreau]

Yes you can be selective about it. You are looking to filter events out before they are indexed. You'll need to modify props.conf and transforms.conf files with some parsing to remove/replace data. Here's some links to get you started.
https://answers.splunk.com/answers/578334/how-can-i-filter-events-before-they-are-indexed-so.html
https://answers.splunk.com/topics/event-filtering.html
https://docs.splunk.com/Documentation/Splunk/7.2.5/Forwarding/Routeandfilterdatad
A gotcha to be aware of is SEDCMD in props.conf comes before Transforms. (hattip: @davidpaper )