NLP Text Analytics: Search lookup error can locate resources vader_lexicon

[proyleJDS]

I have installed NLP Text Analytics and the other supporting apps.
The app is working fine on an identical search head but not this new search head. The only difference is that the new search head needs to go via a proxy to get to the internet.
I am getting the following error when I search:

sourcetype=aws:firehose:json Attributes.postCallSurvey_feedback="*" Agent.Username="*" | eval customerName='Attributes.customerName',postCallSurvey='Attributes.postCallSurvey_feedback' | spath InitiationMethod | search InitiationMethod=INBOUND OR  InitiationMethod=CALLBACK | table  postCallSurvey
| vader textfield=postCallSurvey full_output=t |  stats avg(sentiment) AS sentiment

4 errors occurred while the search was executing. Therefore, search results might be incomplete. Hide errors.
[ip-10-7-0-188] LookupError at "/opt/splunk/var/run/searchpeers/ip-10-7-2-136-1569282229/apps/nlp-text-analytics/bin/nltk/data.py", line 675 : ********************************************************************** Resource vader_lexicon not found. Please use the NLTK Downloader to obtain the resource: >>> import nltk >>> nltk.download('vader_lexicon')  Searched in: - '/root/nltk_data' - '/usr/share/nltk_data' - '/usr/local/share/nltk_data' - '/usr/lib/nltk_data' - '/usr/local/lib/nltk_data' - '/opt/splunk/nltk_data' - '/opt/splunk/share/nltk_data' - '/opt/splunk/lib/nltk_data' - '/opt/splunk/etc/apps/nlp-text-analytics/bin/nltk_data' - u'' **********************************************************************

There are 4 errors identical and the IP address at the start are indexers in our cluster.

I have been able to successfully run the downloader and all the nltk_data libraries are installed.

python -m nltk.downloader all

[worshamn]

The app comes already bundled with the needed files, so there should not be need to download them. The file it is looking for is found at $SPLUNK_HOME/etc/apps/nlp-text-analytics/bin/nltk_data/sentiment/vader_lexicon.zip

Because the error mentions "searchpeers" it sounds to me like it is expecting to find that file on the indexer's search bundle that is distributed from the search head. That might be an incorrect assumption, but I have seen similar behavior in my environment. The app is intended to be run "locally" (i.e. --> @Configuration(local=True)) but that config doesn't seem to always be honored. See if you have any blacklists in distsearch.conf that may stop that file or directory from coming down to the indexer.

UPDATE: This problem may be fixed after 7.3:
Date resolved Issue number Description
2019-02-15 SPL-159461, SPL-159052 SH is not making use of the latest bundle info from the indexer, during the bundle replication.
https://docs.splunk.com/Documentation/Splunk/7.3.0/ReleaseNotes/Fixedissues#Distributed_search_and_s...

[proyleJDS]

Thanks worshamn, you set me on the right track, I had initially tried distributing the nlp app to my cluster nodes via the master node, but of course that went into etc/slave_apps.
I since copied the nlp app the the indexers under etc/apps and the app has started working and stopped complaining.
The difference between my 2 search heads the working one and the new one that was looking for this app on the indexers is the old one is Splunk 7.1.3 and the new one is 7.3
So for those who are interested there is definitely a difference from 7.3 in the way search works, whether it stays local or not, as my 2 servers have no difference in the config only Splunk version.
So it appears this problem occurred as of 7.3