After trying every mitigation on the forum , why is Splunk still not receiving any data from the in the Asset Discovery App?

[clozach]

Hi,

I'm still not receiving any data from this app. I have nmap installed with permissions to run. I have added +s chmod on the nmap binaries. I have edited my stanza for a simple ping_scan. I have unset LD_LIBRARY_PATH in nmap.sh. I edited line 111 in nmap.sh to nmap $nargs $target #2>/dev/null. I made sure the index was created. I installed it on a single instance. I am running Splunk on Ubuntu. If I check the splunkd.log there is no data pertaining to asset_discovery or nmap.

Please let me know if you have any ideas how to debug.

I appreciate everyone's time!

Thank you.

[mw]

Probably a silly question, but is the input actually enabled?

[clozach]

Hi mw, sorry about the late response. There were a couple of inputs not enabled. After enabling them and trying to run a search once or twice I got these two errors:

Search peer splunk05 has the following message: Received event for unconfigured/disabled/deleted index=asset_discovery with source="source::nmap" host="host::splunk01" sourcetype="sourcetype::ping_scan". So far received events from 1 missing index(es).
10/24/2018, 4:06:41 PM

msg="A script exited abnormally" input="./bin/nmap.sh -A -O" stanza="default" status="exited with code 1"

These were separate errors at different times (around 5 minutes apart). Also, I can see that the asset_discovery index is present and enabled. Any ideas?

[mw]

This is an indication that data is, in fact, being created (i.e. scans are happening), but the associated asset_discovery index is either disabled or doesn't exist on splunk05. I believe I'm correct that you have a distributed setup, and while you've installed the app on the search head, have the input enabled there, and are (correctly) forwarding data from the search tier to your indexers, you haven't ensured that your search peers / indexers have the index as well. The simplest solution would be to deploy the app to the indexers.

[clozach]

Hi mw, this isn't really for you to answer, but I thought I'd ask if you know the answer. Indexers don't have a web interface. So, i'm curious if that every single app we install is it necessary to ssh in and edit the indexes.conf. I feel like there has to be a better way of doing this. This makes adding an index through the web kind of useless when you still need to have ssh access to be able to create an index that'll actually work. I could be understanding something incorrectly, so if you have any insight or best practices for creating an index, I would really appreciate it.

Thank you.

[mw]

Splunk has solutions for that: https://docs.splunk.com/Documentation/Splunk/7.2.0/Indexer/Manageappdeployment

Also, you can use 3rd party tools such as puppet, chef, etc to manage the indexers if you'd like.

[clozach]

Hi mw, I added the index to the remaining machines in the distributed deployment. I stopped receiving the error, but still no data is being populated. Any ideas?