I have a 6 member search head cluster v6.2.2. A user promoted their dashboard from private to app level visibility. On one server, the dashboard appears with app level visibility and on the other 5 it doesn't appear at all. On one where it doesn't appear in the 'Dashboards' link, it does appear in the 'Settings|User Interface|Views' page--as private. There is no app level version. BTW, the app has r/w permissions for all. It seems mighty tedious if every modification to a dashboard has to be done using the deployer. How could that be even managed? What might I be missing? ... View more

I have about 1300 hosts configured with uni forwarders sending data to a single heavy forwarder. The heavy forwarder doesn't do any processing really (at least yet). It simply turns around and forwards to the indexers in another data center. Essentially, it is functioning as a gateway. Many of the uni forwarders are logging timeout errors trying to connect to the intermediate. fs.max_files is ~793k. And process file descriptor limit is 64k. Typically a dozen or more connections on the intermediate forwarder are in SYN_RECV state. If I telnet to the port (9997) in this case, I get mostly timeouts with an occasional connection. It's acting like it is resource starved. It's running on a 4-core 8G VM. I can't find any splunk tunable connection limits except for the queueSize which I upped to 10MB with no apparent difference. The splunkd seems to float around 60% of a single CPU while in this state. Have I maxed the capability of splunk in this case? I suppose I can add more forwarders and load balance between them. Splunk, what are your recommendations? Thanks, guys! ... View more