Shapester App randomly loses existing lookups and fails to load them through UI

[thervig]

I am a currently a student worker and I am just starting to learn how to use Splunk and Linux. My Splunk environment is on a local server as I only use it for dev stuff for now.

After using the Shapester App to draw some custom polygons for a .kmz of my university campus, I would randomly get an internal 500 server error if I were to go the "Manage Lookups" tab. Then, I go back to the shape editor and I find that the UI would no longer have my existing lookup to load (geo_campus_buildings). Sometimes I would also lose the ability to use the UI to load an existing lookup after just powering off my Linux box and coming back the next day. However, I am able to locate the .kmz files I made through Shapester in my terminal so I do not think the files got deleted. I then checked my Splunk's Internal Errors and found no errors with the Shapester app itself. I also checked whether it would load the lookups with different browsers and both Firefox and Chrome had the same issue. Is there something I am missing? Is it the App? Hopefully I gave an ample amount of information and any help to just point me in the right direction would be wonderful.

Thanks

[mporath_splunk · Accepted solution]

Hi thervig,

Based on your description, it looks like there's an error with defining the lookup. You could check $SPLUNK_HOME/var/log/splunk/python.log and see if you see anything.

One known issue is that feature ID names aren't escaped. That means the lookup might fail if you use either > or < in your names.

[thervig]

I did some more digging into part of the problem that might be causing the "Internal 500 server error" and I think it is because of some of the javascripts.
For example, I saw errors from the splunk_web_service:

fileName=/app/geoshapester/components/map/map.js TypeError: Cannot read property 'Control' of undefined 

fileName=/app/geoshapester/components/feature/featureTableView.js TypeError: Backbone.BootstrapModal is not a function

Is it possible that this might be causing the problem?

[mporath_splunk]

This seems more likely to be a consequence of the Internal 500 server errors rather than the cause

We've updated the app (v1.0.3) with some tweaks. Could you try to upgrade and see if that solves the problem?

[thervig]

I updated to 1.0.3 when it came out but I still can't use the UI to load existing lookups I made a week or so ago (just double checked). I don't mind having to make the .kmz's again, I just don't want them to keep disappearing after I put some hours of work into them. Should I perhaps try to reinstall the app and restart splunk?

[mporath_splunk]

Unfortunately you won't be able to recover the lookups you created last week through the app.
Feel free to send me your KMZ to [my username]@splunk.com, and I'll take a look to see if there's anything wrong with it. Might be that the KMZs are fine and just the lookup definition failed.

[thervig]

Hey mporath, thanks for the reply.

I went an checked the python log and I got this message:

-bash-4.3$ tail python.log
OSError: [Errno 2] No such file or directory: '/opt/splunk/etc/apps/geoshapester/lookups'
2015-09-28 15:00:05,484 ERROR [Errno 2] No such file or directory: '/opt/splunk/etc/apps/geoshapester/lookups'
Traceback (most recent call last):
  File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 185, in dispatch
    methodOutput = method()
  File "/opt/splunk/etc/apps/geoshapester/bin/geoshapester_endpoint.py", line 146, in handle_GET
    self.response.write(json.dumps(lookup_files_to_geo_json(lookup_folder())))
  File "/opt/splunk/etc/apps/geoshapester/bin/geoshapester_endpoint.py", line 105, in lookup_files_to_geo_json
    for f in os.listdir(folder):
OSError: [Errno 2] No such file or directory: '/opt/splunk/etc/apps/geoshapester/lookups'

So apparently it is not reading there is a directory when there actually is since I go in and get this:

-bash-4.3$ cd /opt/splunk/etc/apps/geoshapester/lookups
-bash-4.3$ ls
ASU_TEST_POLYGONS.csv  geo_asu_campus_buildings.kmz
geo_asu_buildings.kmz  geo_test.kmz

As for my feature ID names, I am positive I have not used any characters besides letters for them. Hopefully this is helpful information to solve my issue.

Thanks for your time, really appreciate it.