We are brand new Splunk Enterprise implementation here at macys.com. We are rolling out 100's of forwarders. I've had no issues until now:
We are deploying linux forwarders and I am now hitting an issue. Sometimes it works completely, but now I see many servers are not recv'ing the deployment dir of Splunk_TA_nix or the
/opt/splunkforwarder/etc/apps/macys_lorain_forwarder_outputs/local/outputs.conf
I've restart splunk svc's, redeploy, etc... several times with no luck. I check the deployment UI and it shows my new client (under Client Tab) as checked in but nothing under Apps.
Another anomaly in this is that we've noticed that it may take hours b/f some servers get the Splunk_TA_nix dir, which is not right of course.
I have the exact steps I use to deploy the linux forwarder if you need to review.
Why is this breaking all of a sudden?
We cannot move ahead w/ 90 more deployments with this current issue. Help is appreciated.
cheers,
Damon
