Hi, we are getting a lot of CheckPoint logs, as compare to other sources, so was wondering if there exists any "best practice" in terms of what getting only what is required rather than the entire set of fields.
How can we trimmed off these excessive fields? Do we do it at CheckPoint's end or Splunk's end?
In terms of "best practices" or "minimum requirements" for deciding what to log and what not to log, it heavily depends on the policies and practices of your organization. You may be legally required to keep certain types of logs for certain periods of time. Does your organization consider the Check Point binary-formatted log files generated by your Security Management server(s) to be authoritative over all other forms of firewall logs? If so, you can probably tweak the Check Point data that you're bringing into Splunk. On the other hand, if your organization considers the logs stored in Splunk as authoritative I wouldn't make any changes to what you're doing now without additional approval.
Now, whether a firewall field is considered "excessive" or not is really quite subjective. In my case, I always wanted as much data - from as many fields as possible - out of my firewall logs…even more data than what fw1-loggrabber could provide. For example, firewall cluster failover messages would have been nice to have. (The most recent version of fw1-loggrabber does not support the LEA field that logs firewall failover/failback events.)
As for trimming off fields that you do not want to index, you should perform the trimming within fw1-loggrabber.conf, using the FIELDS configuration stanza. This is easier than configuring Splunk to not index certain key/value pairs within your Check Point logs.
A barebones FIELDS stanza might look something like what I've shown below. Keep in mind that this setting leaves out lots of interesting Check Point data like VPN messages, SecureClient messages, SmartDefense messages, DCE-RPC messages, anti-spoofing warnings, etc.
This configuration should work without any additional modification to props.conf and transforms.conf assuming that you've downloaded one of the Splunk OPSEC apps from splunkbase. (I just checked and these field extractions are also included in the latest Check Point OPSEC 2.0 beta App that was released on 2013-01-03.)