Splunk Search

scripted lookup fails with pooled search heads

tpsplunk
Communicator

I had a scripted lookup working (the dnslookup using external_lookup.py) with a standalone search head. Now that i've implemented search head pooling the lookup is failing with error: "Error in 'lookup' command: Could not find all of the specified lookup fields in the lookup table."

it fails on all members of the search pool, including the original search head where it used to work. although the error seems search related, nothing has changed regarding this search. i'm not sure where to start to figure out the issue. any ideas?

1 Solution

tpsplunk
Communicator

I worked with Masa in support and we found that I because i had moved the external_lookup.py into my mounted searchheadapp\bin directory i needed to run splunk btool fix-dangling. after i did this the lookup works as expected!

View solution in original post

tpsplunk
Communicator

I worked with Masa in support and we found that I because i had moved the external_lookup.py into my mounted searchheadapp\bin directory i needed to run splunk btool fix-dangling. after i did this the lookup works as expected!

View solution in original post

ewoo
Splunk Employee
Splunk Employee

No, you only need to run fix-dangling on one of the search heads; the command "repairs" the confs on shared storage directly.

0 Karma

jdunlea_splunk
Splunk Employee
Splunk Employee

Do you have to run "splunk btool fix-dangling" on EACH search head within the pool???

0 Karma

Masa
Splunk Employee
Splunk Employee

This must be a bug. Support reproduced the issue. We're investigating it.

0 Karma

tpsplunk
Communicator

btw, i did try this with the external_lookup.py copied to '/etc//bin' and following the 4 steps above. it did not work- i still get the same lookup error

0 Karma

tpsplunk
Communicator

Masa- can you provide details on how you updated your transforms.conf and where you moved the external_lookup.py? did you move it to /etc//local/bin/ ?

0 Karma

Masa
Splunk Employee
Splunk Employee

Sorry but the dev team could not reproduce the issue. After re-mounting the mount point, the problem went away. Could it possible to try it?

  1. Stop Search Heads
  2. Unmount the Search Head pooling point
  3. Re-mount the point
  4. Start the Search Heads

In my case, I reconfigured the transforms.conf and moved the external_lookup.py to the search app in the pooling mount point.

0 Karma

the_wolverine
Champion

I seem to be having this issue as well. I'll be contacting support for an update.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.