I am not seeing forwarder client in splunk nmon GUI, but the server is connected to client.checked with netstat in unix/AIX environment
Hi,
Don't know if you checked it, the documentation is provided within the Application, accessible through the icon marker page on top left of the home page.
There is various deployment scenarios described, with all required explanations.
Therefore, in substance, here is what you need to do:
Set the receiving port of your indexer(s)
Your indexers need to have a receiving port activated to receive data from your hosts:
Manager -> sending and receiving -> configure receiving -> new
The default port is 9997
Connect your universal forwarders with your deployment server:
Once your have installed a Universal Forwarder, connect it to your deployment server:
splunk set deploy-poll <IP_address/hostname>:<management_port>
splunk restart
--> Replace with the IP/hostname and splunkd port of your deployment server
Prepare your package for deployment:
On the deployment server:
/opt/splunk/bin/splunk enable deploy-server -auth admin
Extract the content of the nmon/resources/TA_nmon_V*.tar.gz to $SPLUNK_HOME/etc/deploy-apps/TA_nmon
For AIX hosts, you can set up a local and upgrade resilient configuration to use Perl scripts instead of Python (which is the default choice), in deed generally AIX have no Python interpreter installed by default. (and UF don't have Python interpreter embeded)
create a "TA_nmon/local/props.conf" with the following content: (note the .pl instead of .py)
[source::.../*.nmon]
invalid_cause = archive
unarchive_cmd = $SPLUNK_HOME/etc/apps/TA-nmon/bin/nmon2csv.pl
sourcetype = nmon_processing
NO_BINARY_CHECK = true
You will also want to deactivate the Python cleaner to activate the Perl, create a local/inputs.conf with:
[script://./bin/nmon_cleaner.py --cleancsv]
disabled = true
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
[script://./bin/nmon_cleaner.pl --cleancsv]
disabled = false
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
Access to the Forwarder Management page, create a class, add the TA-nmon, set your target hosts
Restart your deployment server
Verify that the TA is being deployed in your UF hosts (in the Forwarder management page, directly in the UF if you want and so on)
Once you are there:
UF hosts should start to generate Nmon data, in any case you will find data in the index:
index=nmon
There is various sourcetypes, notably:
sourcetype=nmon_data --> Performance Data (generated if everything works as expected)
sourcetype=nmon_collect --> Activity of the input nmon_helper.sh script which launches the Nmon data generation
sourcetype=nmon_clean --> Cleaner activity
sourcetype=nmon_config --> Configuration data
If you don't have yet performance data after a few minutes (nmon_data and nmon_config), verify informations in nmon_collect
Also, check the splunkd log of your UF hosts, if they are right connected to your indexers, a search like:
index=_internal hosts=<myhost>
Must return results, and the splunkd events of your UF.
Anyway, in case of trouble, logs are the key, and by the way the essence of Splunk power 🙂
Last but not least, the UF should generate csv data that is being indexed then deleted by Splunk, in TA-nmon/var/csv_repository and config_repository.
This works in batch mode (index and delete), in normal time you should not have data here if the UF is working fine.
You can also debug the converter operation, check the help page or contact me
I hope this helps, as previously said we can exchange by mail if you don't get it to work.
Guilhem
Hi,
Don't know if you checked it, the documentation is provided within the Application, accessible through the icon marker page on top left of the home page.
There is various deployment scenarios described, with all required explanations.
Therefore, in substance, here is what you need to do:
Set the receiving port of your indexer(s)
Your indexers need to have a receiving port activated to receive data from your hosts:
Manager -> sending and receiving -> configure receiving -> new
The default port is 9997
Connect your universal forwarders with your deployment server:
Once your have installed a Universal Forwarder, connect it to your deployment server:
splunk set deploy-poll <IP_address/hostname>:<management_port>
splunk restart
--> Replace with the IP/hostname and splunkd port of your deployment server
Prepare your package for deployment:
On the deployment server:
/opt/splunk/bin/splunk enable deploy-server -auth admin
Extract the content of the nmon/resources/TA_nmon_V*.tar.gz to $SPLUNK_HOME/etc/deploy-apps/TA_nmon
For AIX hosts, you can set up a local and upgrade resilient configuration to use Perl scripts instead of Python (which is the default choice), in deed generally AIX have no Python interpreter installed by default. (and UF don't have Python interpreter embeded)
create a "TA_nmon/local/props.conf" with the following content: (note the .pl instead of .py)
[source::.../*.nmon]
invalid_cause = archive
unarchive_cmd = $SPLUNK_HOME/etc/apps/TA-nmon/bin/nmon2csv.pl
sourcetype = nmon_processing
NO_BINARY_CHECK = true
You will also want to deactivate the Python cleaner to activate the Perl, create a local/inputs.conf with:
[script://./bin/nmon_cleaner.py --cleancsv]
disabled = true
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
[script://./bin/nmon_cleaner.pl --cleancsv]
disabled = false
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
Access to the Forwarder Management page, create a class, add the TA-nmon, set your target hosts
Restart your deployment server
Verify that the TA is being deployed in your UF hosts (in the Forwarder management page, directly in the UF if you want and so on)
Once you are there:
UF hosts should start to generate Nmon data, in any case you will find data in the index:
index=nmon
There is various sourcetypes, notably:
sourcetype=nmon_data --> Performance Data (generated if everything works as expected)
sourcetype=nmon_collect --> Activity of the input nmon_helper.sh script which launches the Nmon data generation
sourcetype=nmon_clean --> Cleaner activity
sourcetype=nmon_config --> Configuration data
If you don't have yet performance data after a few minutes (nmon_data and nmon_config), verify informations in nmon_collect
Also, check the splunkd log of your UF hosts, if they are right connected to your indexers, a search like:
index=_internal hosts=<myhost>
Must return results, and the splunkd events of your UF.
Anyway, in case of trouble, logs are the key, and by the way the essence of Splunk power 🙂
Last but not least, the UF should generate csv data that is being indexed then deleted by Splunk, in TA-nmon/var/csv_repository and config_repository.
This works in batch mode (index and delete), in normal time you should not have data here if the UF is working fine.
You can also debug the converter operation, check the help page or contact me
I hope this helps, as previously said we can exchange by mail if you don't get it to work.
Guilhem
Thanks, I have gone all your procedure, but I cannot see the forwarder client in "splunk app for UNIX" app. when i select this i can only see server instance. no clients.
Hi, I am afraid you seem to be in confusion with the Splunk App for Unix (an official Splunk App)
That's a totally different App !
Oh ok.. Now i can able to get Nmon data. I want to configure email alerts for CPU usage ( >95), mem,Disk,Net work and File system alerts in mail. Do you have any procedure to get email alerts?
No (not yet) but you can build your own requests (take a look at Pivot models within the App) and create your own alert, that's Splunk native functions
I cannot see any contact option to contact you by mail here..
In the App page (https://apps.splunk.com/app/1753) on bottom right, contact
Hi,
Usually AIX won't have a Python interpreter installed, you have to set up a local/props.conf in your deployment to Splunk to use Perl converter and cleaner.
Check the props.conf and documentation of the App.
Splunk logs from your UF will help too.
Finally don't Forget to set the outputs.conf of your deployment
Right, no problem.
Let's exchange by mail, contact me by mail through the contact option of the App page, i will send you further explanations to get the right configuration
Guilhem
Hi I am not able to get exact settings for this, Can you share any link to configure props.conf?