topic Re: Splunk Add-on for Unix and Linux coming up empty in All Apps and Add-ons

Splunk Add-on for Unix and Linux coming up empty

redc — Fri, 27 Jun 2014 21:49:59 GMT

I'm in the process of setting up the Splunk App for Unix and Linux and the Splunk Add-on for Unix and Linux.

I've installed and configured the App via Splunk Web (which runs on a Windows box) using default settings. I've installed the Add-on on one of my Linux boxes and enabled all of the default inputs using default settings. I've got data flowing into the "os" index.

But...all of the App dashboards are coming up empty/"No results found."

Here's a screenshot of the Hosts dashboard, showing the information for the one Linux host I've configured:

Using the "Process Status" as an example (since it's easy to inspect), I get:
This search has completed and found 5 matching events. However, the transforming commands in the highlighted portion of the following search:

search index=os sourcetype=top host=my-host-name | stats max(pctCPU) as pctCPU max(pctMEM) as pctMEM last(cpuTIME) as cpuTIME by COMMAND, USER | eval CMD=COMMAND | fields CMD, USER, pctCPU, pctMEM, cpuTIME

generated no results.

If I run the search command portion (excluding the stats command and everything after it), I get events that look like this (screenshot #2); I assume this format normal:

Argh! So, what am I missing?

Re: Splunk Add-on for Unix and Linux coming up empty

araitz — Fri, 27 Jun 2014 21:55:59 GMT

Did you follow the prompt from the home page to set up the app? Did you read through the docs on first time configuration?

http://docs.splunk.com/Documentation/UnixApp/latest/User/First-timeconfiguration

Re: Splunk Add-on for Unix and Linux coming up empty

lguinn2 — Fri, 27 Jun 2014 21:57:11 GMT

I don't know what to tell you about the app in general, but I know what is wrong with this particular search!

index=os sourcetype=top host=my-host-name  
| multikv
| stats  max(pctCPU) as pctCPU max(pctMEM) as pctMEM last(cpuTIME) as cpuTIME by COMMAND, USER 
 | eval  CMD=COMMAND  
| fields  CMD, USER, pctCPU, pctMEM, cpuTIME

My only other suggestion is that you check the versions of the app and the add-on - there may be older and newer versions, and you should be sure to use versions of the two that work together...

There is a manual for the app at Splunk App for Linux and Unix

Re: Splunk Add-on for Unix and Linux coming up empty

araitz — Fri, 27 Jun 2014 22:01:21 GMT

@lguinn - the nix TA runs KV_MODE=MULTI automatically, so running multikv explicitly is not required.

Re: Splunk Add-on for Unix and Linux coming up empty

redc — Fri, 27 Jun 2014 22:02:58 GMT

The bottom screenshot is the data that should be filling the "Process Status" portion of the first screenshot, NOT the cpu.sh, vmstat.sh, and df.sh portions in the "Specification" and "System Status" portions. Those three inputs are also sending data to the "os" index but getting the same "transforming commands" generated no results error as above.

Yes, I performed the actions in that document, up to the alerts portion (I'm not ready yet to start alerts flowing), including deleting the auto-created "all_hosts" and "default" category/group in order to configure my own.

Re: Splunk Add-on for Unix and Linux coming up empty

redc — Fri, 27 Jun 2014 22:07:47 GMT

Just for kicks, I tried this by running the search manually and it does generate output where before it doesn't. Sorry @araitz.

However, I find it odd that the app doesn't do this "out of the box" since I would expect there to always be multiple commands/users running on any given server (or at least, for that to be the case more often than not).

But is this the solution? Since I haven't seen what this dashboard should look like when it's working correctly, I'm not sure if this produces the correct results.

Re: Splunk Add-on for Unix and Linux coming up empty

araitz — Mon, 28 Sep 2020 16:57:04 GMT

Well I think you've found your own problem. Doesn't sound like the TA-nix is installed on your search head. Not sure how that can be? Here's the stanza from Splunk_TA_nix/default/props.conf:

 [top]
 SHOULD_LINEMERGE=false
 LINE_BREAKER=(^$|[\r\n]+[\r\n]+)
 TRUNCATE=1000000
 DATETIME_CONFIG = CURRENT
 KV_MODE=multi

Re: Splunk Add-on for Unix and Linux coming up empty

redc — Mon, 28 Sep 2020 16:57:08 GMT

Ah-ha. I'd installed it, but it was disabled. Enabling the Splunk_TA_nix on the search head solved it.

Re: Splunk Add-on for Unix and Linux coming up empty

araitz — Fri, 27 Jun 2014 22:39:22 GMT

See above comment - answers won't let me move it 🙂