How to filter out the EVENT_TYPE in sfdc:logfile using Splunk Add on for Salesforce?

[martaBenedetti]

Hi community,

I have installed Splunk Add on for Salesforce on Heavy Forwarder and have been collecting data from Salesforce Object and Event Log.

I've noticed that sfdc:logfile is huge and I don't need all the records but from UI there is no why to filter out the collection.

Is there a way where we can filter out the EVENT_TYPE? I need only events with EVENT_TYPE="LightningPageView"

Any help is appreciated.

Thank you

Marta

[FelixLeh]

 I haven't worked with this particular addon but normally if you would like to exclude events from indexing you'd create a section in the transforms.conf

[choose_a_unique_name]
REGEX = regex_that_matches_events_that_you_want_to_exclude
DEST_KEY = queue
FORMAT = nullQueue

and then add it to you the sourcetype/source/host in the props.conf.

[sourcetype_name]
TRANSFORMS-unique_name = name_from_the_transforms.conf

Since this is an addon not developed by you these changes should be done in local.

Eventtypes are used on data that is already indexed in Splunk. This is why you cant use them to stop certain data from getting indexed into Splunk since they are calculated at search time.

[martaBenedetti]

I've tried props.conf and transforms.conf both on Heavy Forwarder where the add-on is installed and on indexers with no succeed, it seems that are ignored anche all the event_type are being collected.

This is the splunk doc I've followed: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Keep_specific_eve...

transforms.conf

[setnull_EVENT_TYPE]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing_EVENT_TYPE]
REGEX = EVENT_TYPE\=\"LightningPageView\"
DEST_KEY = queue
FORMAT = indexQueue

props.conf

[sfdc:logfile]
TRANSFORMS-setEventType = setnull_EVENT_TYPE, setparsing_EVENT_TYPE

This is a sample log:

2023-11-06T22:58:22.108+0000 SFDCLogType="LightningPageView" SFDCLogId="aaaa" SFDCLogDate="2023-11-06T00:00:00.000+0000" EVENT_TYPE="LightningPageView" TIMESTAMP="20231106225822.108" REQUEST_ID="TID:aaaa" ORGANIZATION_ID="aaaa" USER_ID="aaaa" CLIENT_ID="" SESSION_KEY="aaaa" LOGIN_KEY="aaaa/aaaa" USER_TYPE="Standard" APP_NAME="aaaa:aaaa" DEVICE_PLATFORM="SFX:BROWSER:DESKTOP" SDK_APP_VERSION="" OS_NAME="WINDOWS" OS_VERSION="10" USER_AGENT=""Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0"" BROWSER_NAME="EDGE" BROWSER_VERSION="119" SDK_VERSION="" DEVICE_MODEL="" DEVICE_ID="" SDK_APP_TYPE="" CLIENT_GEO="Italy/Rome" CONNECTION_TYPE="" UI_EVENT_ID="ltng:pageView" UI_EVENT_SOURCE="" UI_EVENT_TIMESTAMP="1699311501485" PAGE_START_TIME="1699310036553" DURATION="588.0" EFFECTIVE_PAGE_TIME_DEVIATION="false" EFFECTIVE_PAGE_TIME_DEVIATION_REASON="" EFFECTIVE_PAGE_TIME_DEVIATION_ERROR_TYPE="" EFFECTIVE_PAGE_TIME="588.0" DEVICE_SESSION_ID="aaaa" UI_EVENT_SEQUENCE_NUM="22" PAGE_ENTITY_ID="" PAGE_ENTITY_TYPE="Case" PAGE_CONTEXT="force:objectHomeDesktop" PAGE_URL="/lightning/o/Case/list?filterName=Recent" PAGE_APP_NAME="LightningService" PREVPAGE_ENTITY_ID="aaaa" PREVPAGE_ENTITY_TYPE="Case" PREVPAGE_CONTEXT="one:recordHomeFlexipage2Wrapper" PREVPAGE_URL="/lightning/r/Case/aaaa/view" PREVPAGE_APP_NAME="LightningService" TARGET_UI_ELEMENT="" PARENT_UI_ELEMENT="" GRANDPARENT_UI_ELEMENT="" TIMESTAMP_DERIVED="2023-11-06T22:58:22.108Z" USER_ID_DERIVED="aaaa" CLIENT_IP="xxx.xxx.xxx.xxx" UserAccountId="aaaa" SplunkRetrievedServer="https://aaaa.my.salesforce.com"

[FelixLeh]

As far as I can tell your configuration checks out. The problem could be that the input for salesforce is over an API/Script. Different types of inputs behave differently with the configurations and some even skip certain parts of the pipeline. If it was a normal Monitoring Input and your mentioned configuration is on the indexer everything should work. I have to gather more info myself regarding the transformation of the API inputs to help you further on this. 

[martaBenedetti]

Yes, that could be.

Anyway, thank you very much for the help @FelixLeh