hello everyone, I saw multiple post regarding this but couldn't really understand the architect behind. We have 3000 forwarders checked in to the server. We have Windows and Linux server classes. We also created apps to push the configuration files to all the forwarders. But now we would like to implement resource monitoring thru the agent. I wrote a batch script that would query the CPU usage and memory every minute, then output to a txt file, and get pushed out to the Splunk indexer. I knew that if I need to run the script on the remote machine, I would need to place the script under $SPLUNK_HOME/etc/apps/MYAPP/bin . Question is, how I can do that? Isn't that as simple as putting the script in the "app" and then restart the splunk agent, then the agents will pick up whenever it's there? Am I right? And what configuration do I need to specifically to make the script run? I am new to Splunk, haven't got chance to take any training as the company did not provide any. I learn as I go. Thank you ... View more

Hello everyone, It seems like I couldn't find any previous answer on this from the community. I have more than 1000 forwarders installed in Windows/Unix servers. I do not have any RDP nor SSH access into those servers due to security reasons. Once in a while, forwarders do not ping back to the server, so I need to access the Splunk "logs" in the directory without having to RDP nor ssh into those servers. I was told that there's already an index which does the internal logging thing and it's doing so in all forwarder agents. So I was trying to run an index=_internal search in the deployment server, but it returned me with just a hostname, and the hostname was the hostname of the deployment server. Am I doing it wrong? I was trying to check the _internal logs from the forwarders. Can anyone shed me some light on this? Thank you ... View more