Splunk Search

scripted lookup fails with pooled search heads

tpsplunk
Communicator

I had a scripted lookup working (the dnslookup using external_lookup.py) with a standalone search head. Now that i've implemented search head pooling the lookup is failing with error: "Error in 'lookup' command: Could not find all of the specified lookup fields in the lookup table."

it fails on all members of the search pool, including the original search head where it used to work. although the error seems search related, nothing has changed regarding this search. i'm not sure where to start to figure out the issue. any ideas?

1 Solution

tpsplunk
Communicator

I worked with Masa in support and we found that I because i had moved the external_lookup.py into my mounted searchheadapp\bin directory i needed to run splunk btool fix-dangling. after i did this the lookup works as expected!

View solution in original post

tpsplunk
Communicator

I worked with Masa in support and we found that I because i had moved the external_lookup.py into my mounted searchheadapp\bin directory i needed to run splunk btool fix-dangling. after i did this the lookup works as expected!

ewoo
Splunk Employee
Splunk Employee

No, you only need to run fix-dangling on one of the search heads; the command "repairs" the confs on shared storage directly.

0 Karma

jdunlea_splunk
Splunk Employee
Splunk Employee

Do you have to run "splunk btool fix-dangling" on EACH search head within the pool???

0 Karma

Masa
Splunk Employee
Splunk Employee

This must be a bug. Support reproduced the issue. We're investigating it.

0 Karma

tpsplunk
Communicator

btw, i did try this with the external_lookup.py copied to '/etc//bin' and following the 4 steps above. it did not work- i still get the same lookup error

0 Karma

tpsplunk
Communicator

Masa- can you provide details on how you updated your transforms.conf and where you moved the external_lookup.py? did you move it to /etc//local/bin/ ?

0 Karma

Masa
Splunk Employee
Splunk Employee

Sorry but the dev team could not reproduce the issue. After re-mounting the mount point, the problem went away. Could it possible to try it?

  1. Stop Search Heads
  2. Unmount the Search Head pooling point
  3. Re-mount the point
  4. Start the Search Heads

In my case, I reconfigured the transforms.conf and moved the external_lookup.py to the search app in the pooling mount point.

0 Karma

the_wolverine
Champion

I seem to be having this issue as well. I'll be contacting support for an update.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...