Why are dashboards not getting populated with data for Splunk App for Unix and Linux?

[f_luciani]

Splunk 6.1.2 Ubuntu LTS 12.04 Add-on for *Nix 5.0.3 App for Unix and Linux 5.0.1

I have both app and add-on installed but can't see any content in dashboards "Home", "Metrics" and "Hosts" of the app (don't have enough karma to attach the screencap either, sorry). In the top nav bar, between Splunk> logo and login ID (Administrator) I have a "Loading..."; in the left-side dashboard the dropdowns do not show anything, plus the dashboard is empty; in the right side, the smaller dashboard "Recent Unix Headlines" only shows "Waiting for results...". Unix Add-on is not configured for directory inputs because they are huge in this machine, but I have scripted inputs set correctly. The "Settings" view of App for Unix, also, shows me, in "Categories", the host I am working on, so I thought something fishy is going on here...

I could not get results from the searches until I read this question:

http://answers.splunk.com/answers/9138/splunk-unix-app-not-receiving-inputs.html

... so I started the searches with index=os and got the results I wanted from the data I set to be collected. I can have results for top, df, who, etc, as per configuration in the add-on, so it seems the issue is with the app's views. My questions are:

1. Should add-on and app be the same version (presently the app is version 5.0.1 and the add-on is version 5.0.3)?
2. Where should I set the app to use index=os in all searches so I can see the results in the dashboards (for I suspect this is the main reason I cannot have the views displaying any results)?
3. Which logs should I check, since I checked the usual suspects and all I got from splunkd.log was a confirmation that the app had the add-on successfully copied to the destination directory?

[aafogles]

Just my 2 cents. Make sure to check that the sourcetypes you're querying actually have entries. I know my issue is in part because I'm virtual, but only about 8 of the sourcetypes were actually being populated, the rest existed but with no entries, particularly CPU, which is what many of the dashboards default to. Once I pointed my dashboards to sourcetypes that were actually being used, I started seeing stuff.

[jcoates_splunk]

Hi,

1. Not neccesary, the latest versions will work together.
2. Unix App > Settings screen (this is where the "Configure" button takes you on the first time run screen).
3. I don't know what you're trying to find from the logs?

[edit: what the world needs now is another JavaScript WSYWIG markup editor]

[f_luciani]

Hi, jcoates, thanks for the prompt answer!

Regarding the Settings screen, the index=os is already in there, I've proceeded saving it some times just in case to no avail, it is still in there, which is kinda odd, since for all searches I have to manually add it to the beginning of the search. Is it only to be expected, or I am missing something important regarding the whole of Splunk that I should have configured for this app and didn't?

Regarding the logs, I was wondering were should I look to find clues (any clues!) on what is going on with the dashboards inside the app that are not able to show any data. Apart from the logs, are there conf, css, html, js, xml or other config files for this app where I should search for clues on this issue? I will take a look inside the objects of the app to see if I can find anything, because I would like to see what these dashboards show about my system before offering the app's capabilities to other people in the company.

[jcoates_splunk]

Hi, that's really weird... Would it be possible to add that index to Searched by Default as a workaround? Still, that's not how it's supposed to act.

One other thing to try is to check categories (also under settings) -- if you just put everything into one category and group that will clarify if you're running into a grouping problem or an index problem.

[f_luciani]

Agreed, I wasn't expecting misbehaviour from a machine... 😉 Anyway, I've added the index to admin role and it is an acceptable workaround, indeed. I am now able to perform searches without prepending the index=os, which is quite a bonus. The app's dashboards, however, remain blank... Shame, since I would really like to see the Metrics dashboard of my home system, for instance. Swings and runabouts...
I had already organized the servers in groups (each group is named after a country where the hosts are) and there is only one category for now, inside which are the country groups and their respective hosts. I would risk saying this does not seem to be the issue, but one never knows...
Thanks for your help so far, half the troubles are gone, the other half will be taken care in due time.

[araitz]

Did you go through the setup steps as detailed in the documentation? Most folks with this problem in the past did not finish setup.

If you look at the Job activity for the app, do you see errors for the searches?

[f_luciani]

Hi, araitz,

If you are referring to the instructions from the following link:

http://docs.splunk.com/Documentation/UnixApp/5.0.1/User/First-timeconfiguration

...then the answer would be no, I haven't, reason being I've never had the chance to actually see this screen in full. All I have every time I enter the app is a couple blank dashboards with drop-down menus being empty.

Regarding the jobs, they do show up all right in the jobs section of the app, firstly they are empty then I open the search section, run a simple "*" search, reload the jobs page and there the jobs are, no issues apparently. I was wondering whether it would be a good idea to uninstall both app and add-on and re-install them. What would you suggest?

[f_luciani]

Proceeded stopping Splunk, uninstalling the app and add-on, removed app's directories residing inside user's directories, restarted splunk, checked out to be sure no traces of the app ad add-on where left behind, installed app and add-on again and restarted splunk... to no avail. Same thing, I configure the add-on, then go to the app and nothing shows up in the 3 dashboards (home, metrics and hosts). Everything else is working fine, I am still able to run searches within the app and from the main splunk search as well, and I am getting the results fine, only the dashboards refuse to work.

Any ideas?