All Apps and Add-ons

Errors After Upgrading Palo Alto Networks App after upgrading to 6.0

veddersg
Explorer

After upgrading the Palo Alto Networks App for Splunk from version 5.4 to 6, I am getting the following error message from each member of the index cluster (3 member cluster).

[idx1] The lookup table 'minemeldfeeds_lookup' does not exist. It is referenced by configuration 'pan:threat'.
[idx1] The lookup table 'minemeldfeeds_lookup' does not exist. It is referenced by configuration 'pan:traffic'.

I have verified the latest SplunkforPaloAltoNetworks and Splunk_TA_paloalto are both installed on each indexer as well as the search head. How do I go about correcting this error?

0 Karma
1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@veddersg You must enable replication for the kvstore collection [minemeldfeeds]

KV Store collections are not bundle-replicated to indexers by default, and lookups run locally on the search head rather than on remote peers. When you enable replication for a KV Store collection, you can run the lookups on your indexers which let you use automatic lookups with your KV Store collections.

Events of sourcetype pan:threat and pan:traffic have automatic lookups configured in props.conf / transforms.conf so the automatic lookup against the minemeldfeeds collection is running on the indexers but the minemeldfeeds kvstore collection doesn't exist there because it is not being replicated from the SH to the indexers by default.

ie:
/opt/splunk/etc/apps/Splunk_TA_paloalto/default/props.conf

LOOKUP-minemeldfeeds_dest_lookup = minemeldfeeds_lookup indicator AS dest_ip OUTPUT value.autofocus_tags AS dest_autofocus_tags

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/transforms.conf
[minemeldfeeds_lookup]
collection = minemeldfeeds
external_type = kvstore
fields_list = indicator, splunk_source, value.confidence, value.first_seen, value.last_seen, value.share_level, value.sources, value.type, value.autofocus_tags, value.autofocus_benign, value.autofocus_grayware, value.autofocus_malware, value.autofocus_num_matching_artifacts, value

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/collections.conf

[minemeldfeeds]
field.indicator = string
field.splunk_source = string
field.value = string

To enable replication for a KV Store collection and allow lookups against that collection to be automatic:

On your search heads create a file called collections.conf in $SPLUNK_HOME/etc/system/local/ (or within an app)
[minemeldfeeds]
replicate = true

Restart Splunk to apply your changes.
Once the next bundle is replicated to your indexers the kvstore collection definition will exist at the indexers and you should see this error message go away the next search you run.

If your indexers are running a version of Splunk Enterprise that is older than 6.3, attempts to run an automatic lookup fail with a "lookup does not exist" error. You must upgrade your indexers to 6.3 or later to use this functionality.

you can learn more about automatic lookups and kvstore collection replication here:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Knowledge/Makeyourlookupautomatic

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@veddersg You must enable replication for the kvstore collection [minemeldfeeds]

KV Store collections are not bundle-replicated to indexers by default, and lookups run locally on the search head rather than on remote peers. When you enable replication for a KV Store collection, you can run the lookups on your indexers which let you use automatic lookups with your KV Store collections.

Events of sourcetype pan:threat and pan:traffic have automatic lookups configured in props.conf / transforms.conf so the automatic lookup against the minemeldfeeds collection is running on the indexers but the minemeldfeeds kvstore collection doesn't exist there because it is not being replicated from the SH to the indexers by default.

ie:
/opt/splunk/etc/apps/Splunk_TA_paloalto/default/props.conf

LOOKUP-minemeldfeeds_dest_lookup = minemeldfeeds_lookup indicator AS dest_ip OUTPUT value.autofocus_tags AS dest_autofocus_tags

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/transforms.conf
[minemeldfeeds_lookup]
collection = minemeldfeeds
external_type = kvstore
fields_list = indicator, splunk_source, value.confidence, value.first_seen, value.last_seen, value.share_level, value.sources, value.type, value.autofocus_tags, value.autofocus_benign, value.autofocus_grayware, value.autofocus_malware, value.autofocus_num_matching_artifacts, value

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/collections.conf

[minemeldfeeds]
field.indicator = string
field.splunk_source = string
field.value = string

To enable replication for a KV Store collection and allow lookups against that collection to be automatic:

On your search heads create a file called collections.conf in $SPLUNK_HOME/etc/system/local/ (or within an app)
[minemeldfeeds]
replicate = true

Restart Splunk to apply your changes.
Once the next bundle is replicated to your indexers the kvstore collection definition will exist at the indexers and you should see this error message go away the next search you run.

If your indexers are running a version of Splunk Enterprise that is older than 6.3, attempts to run an automatic lookup fail with a "lookup does not exist" error. You must upgrade your indexers to 6.3 or later to use this functionality.

you can learn more about automatic lookups and kvstore collection replication here:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Knowledge/Makeyourlookupautomatic

mwarvi
Explorer

This fixed the issue for me, thank you very much.

0 Karma
Get Updates on the Splunk Community!

App Building 101 - Build Your First App!

WATCH RECORDING NOW   Tech Talk: App Dev Edition Splunk has tons of out-of-the-box functionality, and you’ve ...

Introducing support for Amazon Data Firehose in Splunk Edge Processor

We’re excited to announce a powerful update to Splunk Data Management with added support for Amazon Data ...

The Observability Round-Up: September 2024

What’s up Splunk Community! Welcome to the latest edition of the Observability Round-Up, a monthly series in ...