All Apps and Add-ons

NMON Performance Monitor for Unix and Linux Systems: Why am I not seeing the forwarder client in the GUI?

gsrikanth87
Path Finder

I am not seeing forwarder client in splunk nmon GUI, but the server is connected to client.checked with netstat in unix/AIX environment

0 Karma
1 Solution

guilmxm
SplunkTrust
SplunkTrust

Hi,

Don't know if you checked it, the documentation is provided within the Application, accessible through the icon marker page on top left of the home page.

There is various deployment scenarios described, with all required explanations.

Therefore, in substance, here is what you need to do:

Set the receiving port of your indexer(s)

Your indexers need to have a receiving port activated to receive data from your hosts:

Manager -> sending and receiving -> configure receiving -> new

The default port is 9997

Connect your universal forwarders with your deployment server:

Once your have installed a Universal Forwarder, connect it to your deployment server:

splunk set deploy-poll <IP_address/hostname>:<management_port>
splunk restart

--> Replace with the IP/hostname and splunkd port of your deployment server

Prepare your package for deployment:

  • On the deployment server:

    /opt/splunk/bin/splunk enable deploy-server -auth admin

  • Extract the content of the nmon/resources/TA_nmon_V*.tar.gz to $SPLUNK_HOME/etc/deploy-apps/TA_nmon

  • For AIX hosts, you can set up a local and upgrade resilient configuration to use Perl scripts instead of Python (which is the default choice), in deed generally AIX have no Python interpreter installed by default. (and UF don't have Python interpreter embeded)

create a "TA_nmon/local/props.conf" with the following content: (note the .pl instead of .py)

[source::.../*.nmon]

invalid_cause = archive
unarchive_cmd = $SPLUNK_HOME/etc/apps/TA-nmon/bin/nmon2csv.pl
sourcetype = nmon_processing
NO_BINARY_CHECK = true

You will also want to deactivate the Python cleaner to activate the Perl, create a local/inputs.conf with:

[script://./bin/nmon_cleaner.py --cleancsv]
disabled = true
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean

[script://./bin/nmon_cleaner.pl --cleancsv]
disabled = false
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
  • Access to the Forwarder Management page, create a class, add the TA-nmon, set your target hosts

  • Restart your deployment server

  • Verify that the TA is being deployed in your UF hosts (in the Forwarder management page, directly in the UF if you want and so on)

Once you are there:

UF hosts should start to generate Nmon data, in any case you will find data in the index:

index=nmon

There is various sourcetypes, notably:

sourcetype=nmon_data --> Performance Data (generated if everything works as expected)
sourcetype=nmon_collect --> Activity of the input nmon_helper.sh script which launches the Nmon data generation
sourcetype=nmon_clean --> Cleaner activity
sourcetype=nmon_config --> Configuration data

If you don't have yet performance data after a few minutes (nmon_data and nmon_config), verify informations in nmon_collect

Also, check the splunkd log of your UF hosts, if they are right connected to your indexers, a search like:

index=_internal hosts=<myhost>

Must return results, and the splunkd events of your UF.

Anyway, in case of trouble, logs are the key, and by the way the essence of Splunk power 🙂

Last but not least, the UF should generate csv data that is being indexed then deleted by Splunk, in TA-nmon/var/csv_repository and config_repository.

This works in batch mode (index and delete), in normal time you should not have data here if the UF is working fine.

You can also debug the converter operation, check the help page or contact me

I hope this helps, as previously said we can exchange by mail if you don't get it to work.

Guilhem

View solution in original post

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Hi,

Don't know if you checked it, the documentation is provided within the Application, accessible through the icon marker page on top left of the home page.

There is various deployment scenarios described, with all required explanations.

Therefore, in substance, here is what you need to do:

Set the receiving port of your indexer(s)

Your indexers need to have a receiving port activated to receive data from your hosts:

Manager -> sending and receiving -> configure receiving -> new

The default port is 9997

Connect your universal forwarders with your deployment server:

Once your have installed a Universal Forwarder, connect it to your deployment server:

splunk set deploy-poll <IP_address/hostname>:<management_port>
splunk restart

--> Replace with the IP/hostname and splunkd port of your deployment server

Prepare your package for deployment:

  • On the deployment server:

    /opt/splunk/bin/splunk enable deploy-server -auth admin

  • Extract the content of the nmon/resources/TA_nmon_V*.tar.gz to $SPLUNK_HOME/etc/deploy-apps/TA_nmon

  • For AIX hosts, you can set up a local and upgrade resilient configuration to use Perl scripts instead of Python (which is the default choice), in deed generally AIX have no Python interpreter installed by default. (and UF don't have Python interpreter embeded)

create a "TA_nmon/local/props.conf" with the following content: (note the .pl instead of .py)

[source::.../*.nmon]

invalid_cause = archive
unarchive_cmd = $SPLUNK_HOME/etc/apps/TA-nmon/bin/nmon2csv.pl
sourcetype = nmon_processing
NO_BINARY_CHECK = true

You will also want to deactivate the Python cleaner to activate the Perl, create a local/inputs.conf with:

[script://./bin/nmon_cleaner.py --cleancsv]
disabled = true
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean

[script://./bin/nmon_cleaner.pl --cleancsv]
disabled = false
index = nmon
interval = 600
source = nmon_cleaner
sourcetype = nmon_clean
  • Access to the Forwarder Management page, create a class, add the TA-nmon, set your target hosts

  • Restart your deployment server

  • Verify that the TA is being deployed in your UF hosts (in the Forwarder management page, directly in the UF if you want and so on)

Once you are there:

UF hosts should start to generate Nmon data, in any case you will find data in the index:

index=nmon

There is various sourcetypes, notably:

sourcetype=nmon_data --> Performance Data (generated if everything works as expected)
sourcetype=nmon_collect --> Activity of the input nmon_helper.sh script which launches the Nmon data generation
sourcetype=nmon_clean --> Cleaner activity
sourcetype=nmon_config --> Configuration data

If you don't have yet performance data after a few minutes (nmon_data and nmon_config), verify informations in nmon_collect

Also, check the splunkd log of your UF hosts, if they are right connected to your indexers, a search like:

index=_internal hosts=<myhost>

Must return results, and the splunkd events of your UF.

Anyway, in case of trouble, logs are the key, and by the way the essence of Splunk power 🙂

Last but not least, the UF should generate csv data that is being indexed then deleted by Splunk, in TA-nmon/var/csv_repository and config_repository.

This works in batch mode (index and delete), in normal time you should not have data here if the UF is working fine.

You can also debug the converter operation, check the help page or contact me

I hope this helps, as previously said we can exchange by mail if you don't get it to work.

Guilhem

View solution in original post

0 Karma

gsrikanth87
Path Finder

Thanks, I have gone all your procedure, but I cannot see the forwarder client in "splunk app for UNIX" app. when i select this i can only see server instance. no clients.

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Hi, I am afraid you seem to be in confusion with the Splunk App for Unix (an official Splunk App)
That's a totally different App !

gsrikanth87
Path Finder

Oh ok.. Now i can able to get Nmon data. I want to configure email alerts for CPU usage ( >95), mem,Disk,Net work and File system alerts in mail. Do you have any procedure to get email alerts?

0 Karma

guilmxm
SplunkTrust
SplunkTrust

No (not yet) but you can build your own requests (take a look at Pivot models within the App) and create your own alert, that's Splunk native functions

0 Karma

gsrikanth87
Path Finder

I cannot see any contact option to contact you by mail here..

0 Karma

guilmxm
SplunkTrust
SplunkTrust

In the App page (https://apps.splunk.com/app/1753) on bottom right, contact

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Hi,

Usually AIX won't have a Python interpreter installed, you have to set up a local/props.conf in your deployment to Splunk to use Perl converter and cleaner.

Check the props.conf and documentation of the App.

Splunk logs from your UF will help too.
Finally don't Forget to set the outputs.conf of your deployment

0 Karma

guilmxm
SplunkTrust
SplunkTrust

Right, no problem.

Let's exchange by mail, contact me by mail through the contact option of the App page, i will send you further explanations to get the right configuration

Guilhem

0 Karma

gsrikanth87
Path Finder

Hi I am not able to get exact settings for this, Can you share any link to configure props.conf?

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!