Splunk Search

lookup files not working even after changing the permissions

Raghav2384
Motivator

Experts,

I am tired of trying to make this work 🙂 . We have two instances, one is a distributed search with (1SH and 3 IDX Search peers) and a second instance (SH Cluster and 3 IDX Cluster - which is new). We have established a connection between New SHCluster members to old (3 Indexer search peers) making the data searchable. Now, i started migrating apps from old SH to new SHC. I have copied everything from lookups folder to the new apps lookups dir. Changed permissions but i get the following errors

idx1:The lookup table 'xyz' does not exist. It is referenced by configuration 'source::xyz '.
idx2:The lookup table 'xyz' does not exist. It is referenced by configuration 'source::xyz '.
idx3:The lookup table 'xyz' does not exist. It is referenced by configuration 'source::xyz '.

Where idx1,idx2,idx3 are old indexers. I have made all the lookups available for all apps with read=* and write=* and i still get the error and few dashboards are not populating.

Just to make sure , i ran |inputlookup "xyz.csv" and sure enough, data populates. Any takers?

Thanks in advance!
Raghav

1 Solution

woodcock
Esteemed Legend

You do not need to deploy this to your indexers (yes, i know that your indexers are complaining) but rather to your search heads. The lookup configuration files are now part of bundle replication from the search head and they must be there to be send over to the indexers. But I think you probably did this correctly. Remember that for automatic lookups there are 3 parts: the part in props.conf that defines the lookup command, the lookup file itself AND the part (it is not saying "lookup file not found) in transforms.conf that glues the other 2 pieces together. It looks like you are missing something like this in your transforms.conf:

[xyz]
filename = xyz.csv

View solution in original post

0 Karma

woodcock
Esteemed Legend

OK, one last guess: namespace collision causing you to pick up an identically-named stanza from a different configuration file. This just happened to me and it was a BEAR to figure out. You can use btool to help you look for duplicates:

/opt/splunk/bin//splunk cmd btool lookups list --debug
/opt/splunk/bin//splunk cmd btool props list --debug
/opt/splunk/bin//splunk cmd btool transforms list --debug
0 Karma

woodcock
Esteemed Legend

You do not need to deploy this to your indexers (yes, i know that your indexers are complaining) but rather to your search heads. The lookup configuration files are now part of bundle replication from the search head and they must be there to be send over to the indexers. But I think you probably did this correctly. Remember that for automatic lookups there are 3 parts: the part in props.conf that defines the lookup command, the lookup file itself AND the part (it is not saying "lookup file not found) in transforms.conf that glues the other 2 pieces together. It looks like you are missing something like this in your transforms.conf:

[xyz]
filename = xyz.csv
0 Karma

Raghav2384
Motivator

I had manually changed all the configs from search app to xyz app and redployed. All good now

0 Karma

Raghav2384
Motivator

I definitely have that entry in my transforms.conf under search app context just like it is on the old search head

0 Karma

woodcock
Esteemed Legend

OK then the only other thing that I can think is that the actual host OS file ownership/permissions are wrong on xyz.csv. Have you checked that?

0 Karma

Raghav2384
Motivator

I sure did.

to be precise,

I have an entry in props.conf as follows

LOOKUP-name = name fieldx AS fieldx OUTPUTNEW fieldy AS fieldy

transforms.conf
[name]
filename = name.csv

and the lookup file in /opt/splunk/etc/apps/search/lookups/ new.csv
Permissions : All Apps, everyone read and write. Like i said, when i run the |inputlookup name.csv, i get the table (So file exists)
i did a grep from the entries and found exact extractions as on old search head in props and transforms. Puzzled

Any other suggestions?

Thanks,
Raghav

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) v3.54.0

The Splunk Threat Research Team (STRT) recently released Enterprise Security Content Update (ESCU) v3.54.0 and ...

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

New Learning Videos on Topics Most Requested by You! Plus This Month’s New Splunk ...

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...