All Apps and Add-ons

nmon_data sourcetype missing in Solaris 10 Sparc OS - Splunk UF splunkforwarder-7.3.0

sunnydey728
New Member

We are facing issues with Solaris 10 Sparc OS servers only wherein we are getting lot of internal script errors and also while checking sourcetypes , we do not see nmon_data.

Splunk UF binary installed on Solaris 10 Sparc is splunkforwarder-7.3.0-657388c7a488-SunOS-sparc.tar.

These are the errors that we are getting in Splunkd for those host

Errors:

09-21-2019 04:54:27.560 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh: syntax error at line 188: `count=$' unexpected

09-21-2019 04:57:53.567 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/ta_custom_monitoring_solaris/bin/custom_swap_solaris.sh" ld.so.1: swap: warning: libumem.so: open failed: No such file in secure directories

09-21-2019 05:02:31.565 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nixprocess.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nixprocess.sh: syntax error at line 3: `hostname=$' unexpected

09-21-2019 05:12:07.604 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh: syntax error at line 59: `count=$' unexpected

09-21-2019 05:12:52.012 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/ta_custom_monitoring_all_unix/bin/custom_load_avg_ALLUnix.sh" ld.so.1: uptime: warning: libumem.so: open failed: No such file in secure directories

Please help on this.

0 Karma

sunnydey728
New Member

We have tried installing Splunk UF version 7.2.9 but even with that we are not able to get the desired set of sourcetypes and there are same error occuring in Splunkd.log.

Please check the screenshot for the same.
alt text

0 Karma

ivanreis
Builder

try to run this troubleshoot steps : https://ta-nmon.readthedocs.io/en/latest/troubleshoot.html#
Here is a complete document that author created recently and also have the same troubleshooting steps: https://buildmedia.readthedocs.org/media/pdf/nmon-for-splunk/latest/nmon-for-splunk.pdf
If it did not work, open a case with splunk support and attach the diag file to suport case running on the UF client server and on splunk enterprise.

0 Karma

ivanreis
Builder

Per my verification, the ta-nmon addon does not support version 7.3
COMPATIBILITY
Products: Splunk Enterprise
Splunk Versions: 7.2, 7.1, 7.0, 6.6, 6.5, 6.4, 6.3, 6.2, 6.1, 6.0

further information -> https://splunkbase.splunk.com/app/3248/

Try to deploy a UF version 7.2 instead, and redeploy the addon again.

0 Karma

sunnydey728
New Member

Hello,

We have installed Splunk UF version 7.2.9, still we do not get the desired sourcetypes from the host and getting same errors which we were getting initially.

I am attaching below the logs from splunkd.log having errors and the Splunk UF version info, let me know if there is anything else that needs to be checked.

11-06-2019 10:19:42.292 +0000 INFO HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_172.18.103.249_8089_ai-poc9-pprd.eu.corp.airliquide.com_ai-poc9-pprd_010D7307-0711-4AAC-B679-7AD93A1EDD90
11-06-2019 10:19:44.669 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh: syntax error at line 59: count=$' unexpected
11-06-2019 10:19:50.096 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh: syntax error at line 188:
count=$' unexpected
11-06-2019 10:20:06.597 +0000 INFO TcpOutputProc - Connected to idx=10.151.32.93:9997, pset=0, reuse=0. using ACK.
11-06-2019 10:21:06.400 +0000 INFO TcpOutputProc - Connected to idx=10.151.33.40:9997, pset=0, reuse=0. using ACK.
11-06-2019 10:21:46.468 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nixprocess.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nixprocess.sh: syntax error at line 3: hostname=$' unexpected
11-06-2019 10:22:06.196 +0000 INFO TcpOutputProc - Connected to idx=10.151.33.240:9997, pset=0, reuse=0. using ACK.
11-06-2019 10:22:44.677 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/fifo_consumer.sh: syntax error at line 59:
count=$' unexpected
11-06-2019 10:22:50.105 +0000 ERROR ExecProcessor - message from "/home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh" /home/splunk/splunkforwarder/etc/apps/TA-nmon/bin/nmon_helper.sh: syntax error at line 188: `count=$' unexpected
11-06-2019 10:23:06.002 +0000 INFO TcpOutputProc - Connected to idx=10.151.32.93:9997, pset=0, reuse=0. using ACK.
11-06-2019 10:23:35.900 +0000 INFO TcpOutputProc - Connected to idx=10.151.33.40:9997, pset=0, reuse=0. using ACK.
11-06-2019 10:24:05.800 +0000 INFO TcpOutputProc - Connected to idx=10.151.32.93:9997, pset=0, reuse=0. using ACK.

bash-3.2$ /home/splunk/splunkforwarder/bin/splunk version
Splunk Universal Forwarder 7.2.9 (build 2dc56eaf3546)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...