Hi @klulue,
at first you don't need to put an indexer to perform the work you need but an Heavy Forwarder, pratically there's no difference because an HF is a full Splunk installation where all the data are forwarded to the Indexers and, eventually, locally indexed (duplicated!).
I don't like to have a local copy on the logs (not only for the license consuption!) but to have data in only one point.
This HF will be the syslog and SNMP server; to avoid to have a Single Point of Failure (when the HF is down you loose data) you should use two HFs with a Load Balancer that distributes load between HFs during normal work and manage failures, if you haven't a Load Balancer, you can also use a DNS configuration.
To be sure that the sources send their logs you can run a simple search on Search Heads to know if someone doesn't send but there a problem: you usually cannot be sure that every source send continously logs (unless you're sure there is always a log flow), you should create an heartbeat to solve this problem.
To monitor these sources you have to create a lookup with all the host sources (called e.g. perimeter.csv) containing at least one field (e.g. host) and then run a search like this:
| metasearch index=your_index
| eval host=lower(host)
| stats count BY host
| append [ | inputlookup perimeter.csv | eval host=lower(host), count=0 | fields host count ]
| stats sum(count) AS total BY host
| where total=0
About the secure manner, using syslogs and SNMP you cannot encrypt your data; you could think to put the HFs in the same network of the sources to monitor and then use SSL for the data between the HFs and the Indexers on internet.
Ciao.
Giuseppe
