Hi everyone! Here are a few questions from the session (get the full Q&A deck and live recording in the #office-hours Slack channel):

Q1: Is there any best practice to reduce firewall logs drastically? There are options with other tools, but what Splunk can do?

• There are no universal best practices to this issue, but experts discuss a few ways to go about this, including:
  • filter data on source (firewall) 
  • filter data on Syslog server (syslog-ng, rsyslog, SC4S)
  • filter data on Splunk node (per event filtering)
  • filter data on Splunk Edge Processor
  • Fine tuning (anywhere if allowed): expunge timestamp from raw events (will be in _time field anyway)

Q2: Splunk forwarder with more than one outputs: when one of the outputs becomes unavailable, all outputs stop working. Is this bad config?

• This is expected behavior by design. Splunk under the hood is “a series of tubes” (pipeline processors connected with queues) ~ pipeline set: parsingQ -> parsingP -> aggregationQ -> mergingP -> typingQ -> typingP -> indexingQ -> indexingP => output processor (outputs.conf)
• Possible Solutions
  • Multiple forwarders = independent pipeline sets (not affecting one another)
  • More resilient receivers for both target groups: test1 & test2 
    • e.g for Sylog servers on Linux HA cluster (A/S)
    • UDP syslog is non-blocking on Splunk

• Experts also cover some troubleshooting suggestions, important things to keep in mind, and other important question to address before answering this question

Q3: Is it possible to send masked data to an index in Splunk, and unmasked data to another destination?

• A simple way to achieve this would be by leveraging Splunk Edge Processor
• Edge Processor is a data processing solution that works at the edge of your network
• Data can be processed and routed using Edge Processor