Can not configure any external lookup - can not find executable

[MichalG1]

Hello Team

Splunk 9.4.0. Running as root. All in one.

Seems super simple problem. I am not able to have maxmind lookup working adding Country + City to IP.

root@splunk:/opt/splunk/etc/apps/search/local# cat transforms.conf
[maxmind_lookup]
allow_caching = 1
case_sensitive_match = 1
external_cmd = /opt/splunk/etc/apps/search/bin/geoip_wrapper.sh
fields_list = ip, Country

Tested the script:

root@splunk:/opt/splunk# echo -e "ip\n8.8.8.8" | /opt/splunk/etc/apps/search/bin/geoip_wrapper.sh
ip,Country
8.8.8.8,United States

So seems working fine, but in my search.log getting:

03-16-2025 12:31:09.437 INFO  DispatchStorageManagerInfo [631235 searchOrchestrator] - Successfully created new dispatch directory for search job. sid=828bccc0c4803f0f_tmp dispatch_dir=/opt/splunk/var/run/splunk/dispatch/828bccc0c4803f0f_tmp
03-16-2025 12:31:09.437 INFO  SearchParser [631235 searchOrchestrator] - PARSING: premakeresults
03-16-2025 12:31:09.443 ERROR ExternalProvider [631235 searchOrchestrator] - Could not find '/opt/splunk/etc/apps/search/bin/geoip_wrapper.sh'. It is required for lookup 'maxmind_lookup'.

Permissions are fine:

root@splunk:/opt/splunk# ls -la /opt/splunk/etc/apps/search/bin/geoip_wrapper.sh
-rwxr-xr-x 1 root root 82 Mar 16 12:46 /opt/splunk/etc/apps/search/bin/geoip_wrapper.sh

What am i missing, spend hours already on this....

Also tried direct python script (without wrapper) and the same results. Tried also the path with $SPLUNK_HOME but no change.

For me it looks like a kind of sandboxing ? Maybe i should switch to relative paths ? (tried, did not help)

Thanks,

Michal

[MichalG1]

Thanks, but hmm - that still does not work.

Tried in both search app and my custom app.

I have system permissions for both lookup definition and also script (as per above) @PickleRick 

What am i missing ?

Thanks,

[PickleRick]

1. Just a side note - don't do it in the built-in search app. Create your own one.

2. It's not very well docummented but you have to grant permissions for the script. In your case it would be something like

[bin/geoip_wrapper.sh]
access = read : [*]
export = system

in your metadata/local.meta

[MichalG1]

Thanks, but hmm - that still does not work.

Tried in both search app and my custom app.

I have system permissions for both lookup definition and also script (as per above) @PickleRick 

Still the same error in search.log

Could not find '/opt/splunk/etc/apps/myapp/bin/geoip_wrapper.sh'. It is required for lookup 'maxmind_lookup'.

What am i missing ?

Thanks,

[PickleRick]

I'm also not 100% sure if you can specify absolute path. I'm just not sure.

[MichalG1]

I have tried all possible combinations:

- with bash wrapper and directly python script

- with relative paths (few types)

- with direct paths

And none of those are working 😞 This is really messy

I do not know what to do.....

[MichalG1]

OK, finally i got it !

Working scenario:

[maxmind_lookup]
allow_caching = 1
case_sensitive_match = 1
external_cmd = geoip_lookup.py
fields_list = ip, Country

external_cmd does not work if specific with direct or even relative path.

And also: no need to add anything to ./metadata/local.meta to script permissions, working without those.

But to make sure that lookup could be used correctly from search and reporting app, i had to add to my app (make it global):

[]
access = read : [ * ], write : [ * ]
export = system
version = 9.4.0
modtime = 1742240902.973172000

Thanks for all the hints!

[PickleRick]

Write for everyone is a bit too far 😉

[PickleRick]

To be honest, I also struggled with it a while ago and I have a feeling I got it running in the end but... don't remember how. See https://community.splunk.com/t5/Splunk-Search/Custom-TA-python-script-lookup-issue/m-p/705983

To make things even more "interesting" I'm pretty sure it used to work in my home lab and now I still get the same error as you do.