Getting Data In

Why is syslog-ng dropping events sent to SC4S's destination d_hec_fmt?

gf13579
Communicator

Searching _internal for source=sc4s shows:

 

 

 

srlssydr01 syslog-ng 174 - [meta sequenceId="32595295"] Message(s) dropped while sending message to destination; driver='d_hec_fmt#0', worker_index='5', time_reopen='10', batch_size='19'

 

 

 

and

 

 

 

srlssydr01 syslog-ng 174 - [meta sequenceId="32594764"] http: handled by response_action; action='drop', url='https://http-inputs-acme.splunkcloud.com:443/services/collector/event', status_code='400', driver='d_hec_fmt#0', location='root generator dest_hec:5:5'

 

 

 

Labels (2)
0 Karma
1 Solution

gf13579
Communicator

This can happen when you're trying to send to an index that doesn't exist. You can confirm this by turning on logging to d_hec_debug in /opt/sc4s/env_file and looking at the index-named folder list in /opt/sc4s/archive/debug and confirming all of those indexes exist.

Create the index or update splunk_metadata.csv to change the destination index for a given source key.

Thanks mbonsack in the sc4s community slack channel for the guidance. Posting here for visibility/googling.

View solution in original post

gf13579
Communicator

This can happen when you're trying to send to an index that doesn't exist. You can confirm this by turning on logging to d_hec_debug in /opt/sc4s/env_file and looking at the index-named folder list in /opt/sc4s/archive/debug and confirming all of those indexes exist.

Create the index or update splunk_metadata.csv to change the destination index for a given source key.

Thanks mbonsack in the sc4s community slack channel for the guidance. Posting here for visibility/googling.

Get Updates on the Splunk Community!

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...

IM Landing Page Filter - Now Available

We’ve added the capability for you to filter across the summary details on the main Infrastructure Monitoring ...

Dynamic Links from Alerts to IM Navigators - New in Observability Cloud

Splunk continues to improve the troubleshooting experience in Observability Cloud with this latest enhancement ...