I've looked through several of the other posts on answers regarding this problem and I think I've tried all the suggestions, so here's my post:
I have a script I put $SPLUNK_HOME/etc/apps/search/bin as below:
splunk@splunk1:/opt/splunk/etc/apps/search/bin$ ll freq.py
-r-xr-xr-x 1 splunk splunk 657 Apr 10 20:33 freq.py*
It runs fine when testing with splunk python:
splunk@splunk1:/opt/splunk/etc/apps/search/bin$ /opt/splunk/bin/splunk cmd python ./freq.py splunk.com
domain,frequency
splunk.com,5.96996388594
I created a transforms.conf in $SPLUNK_HOME/etc/apps/search/local as below:
splunk@splunk1:/opt/splunk/etc/apps/search/local$ cat transforms.conf
[freqserver]
external_cmd = freq.py domain
external_type = external
fields_list = domain, frequency
Made sure it had the right linux permissions and owner:
splunk@splunk1:/opt/splunk/etc/apps/search/local$ ll
total 20
drwx------ 2 splunk splunk 4096 Apr 10 20:56 ./
drwxr-xr-x 10 splunk splunk 4096 Mar 10 21:03 ../
-rw------- 1 splunk splunk 807 Mar 30 00:49 indexes.conf
-rw------- 1 splunk splunk 122 Mar 10 21:49 inputs.conf
-rw------- 1 splunk splunk 101 Apr 10 20:56 transforms.conf
In the lookup definition, for permissions, it says that object should appear in all apps and everyone has read and write permissions.
I performed all the above as the admin of a single instance of Splunk.
I restarted Splunk.
So now I run a search:
index="bro" earliest=-1y sourcetype=bro_dns | fields query | rename query as domain | lookup freqserver domain
but I get the following error:
Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
This is on splunk Version: 8.0.2
I was trying to follow these instructions for creating a new external lookup:
https[:]//docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configureexternallookups
That error is the same error I get if I try a lookup name that does not exist:
index="bro" earliest=-1y sourcetype=bro_dns | fields query | rename query as domain | lookup nonsensename domain
would get the same kind of Could not construct lookup error...
Any suggestions?
see: transforms.conf
external_type = [python|executable|kvstore|geo|geo_hex]
* This setting describes the external lookup type.
* Use 'python' for external lookups that use a python script.
* Use 'executable' for external lookups that use a binary executable, such as a
C++ executable.
* Use 'kvstore' for KV store lookups.
* Use 'geo' for geospatial lookups.
* 'geo_hex' is reserved for the geo_hex H3 lookup.
* Default: python
see: transforms.conf
external_type = [python|executable|kvstore|geo|geo_hex]
* This setting describes the external lookup type.
* Use 'python' for external lookups that use a python script.
* Use 'executable' for external lookups that use a binary executable, such as a
C++ executable.
* Use 'kvstore' for KV store lookups.
* Use 'geo' for geospatial lookups.
* 'geo_hex' is reserved for the geo_hex H3 lookup.
* Default: python
Thanks. This was the answer.
New transforms.conf
[freqserver]
external_cmd = freq.py domain
external_type = python
python.version = python2
fields_list = domain, frequency
Of course, if the script is updated to python3, change the setting accordingly.
thanks @reswob10
I was looking for a place to write and usage "python.version" .
In .conf , I'll write it.
How's search log?
here is the errors in the error log and some surrounding context..
04-11-2020 14:49:49.298 INFO UnifiedSearch - Expanded index search = (index="bro" sourcetype=bro_dns _time>=1554994189.000)
04-11-2020 14:49:49.298 INFO UnifiedSearch - base lispy: [ AND index::bro sourcetype::bro_dns ]
04-11-2020 14:49:49.298 INFO UnifiedSearch - Processed search targeting arguments
04-11-2020 14:49:49.298 ERROR ExternalProvider - Command type 'external' is unsupported for lookup 'freqserver'.
04-11-2020 14:49:49.298 ERROR ExternalProvider - Command type 'external' is unsupported for lookup 'freqserver'.
04-11-2020 14:49:49.298 ERROR LookupProcessor - Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
04-11-2020 14:49:49.298 ERROR SearchPhaseGenerator - Fallback to two phase search failed:Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
04-11-2020 14:49:49.299 ERROR SearchOrchestrator - Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
04-11-2020 14:49:49.299 ERROR SearchStatusEnforcer - sid:1586616589.121 Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
04-11-2020 14:49:49.299 INFO SearchStatusEnforcer - State changed to FAILED due to: Error in 'lookup' command: Could not construct lookup 'freqserver, domain'. See search.log for more details.
04-11-2020 14:49:49.299 INFO SearchStatusEnforcer - Enforcing disk quota = 10485760000