All Apps and Add-ons

Errors After Upgrading Palo Alto Networks App after upgrading to 6.0

veddersg
Explorer

After upgrading the Palo Alto Networks App for Splunk from version 5.4 to 6, I am getting the following error message from each member of the index cluster (3 member cluster).

[idx1] The lookup table 'minemeldfeeds_lookup' does not exist. It is referenced by configuration 'pan:threat'.
[idx1] The lookup table 'minemeldfeeds_lookup' does not exist. It is referenced by configuration 'pan:traffic'.

I have verified the latest SplunkforPaloAltoNetworks and Splunk_TA_paloalto are both installed on each indexer as well as the search head. How do I go about correcting this error?

0 Karma
1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@veddersg You must enable replication for the kvstore collection [minemeldfeeds]

KV Store collections are not bundle-replicated to indexers by default, and lookups run locally on the search head rather than on remote peers. When you enable replication for a KV Store collection, you can run the lookups on your indexers which let you use automatic lookups with your KV Store collections.

Events of sourcetype pan:threat and pan:traffic have automatic lookups configured in props.conf / transforms.conf so the automatic lookup against the minemeldfeeds collection is running on the indexers but the minemeldfeeds kvstore collection doesn't exist there because it is not being replicated from the SH to the indexers by default.

ie:
/opt/splunk/etc/apps/Splunk_TA_paloalto/default/props.conf

LOOKUP-minemeldfeeds_dest_lookup = minemeldfeeds_lookup indicator AS dest_ip OUTPUT value.autofocus_tags AS dest_autofocus_tags

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/transforms.conf
[minemeldfeeds_lookup]
collection = minemeldfeeds
external_type = kvstore
fields_list = indicator, splunk_source, value.confidence, value.first_seen, value.last_seen, value.share_level, value.sources, value.type, value.autofocus_tags, value.autofocus_benign, value.autofocus_grayware, value.autofocus_malware, value.autofocus_num_matching_artifacts, value

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/collections.conf

[minemeldfeeds]
field.indicator = string
field.splunk_source = string
field.value = string

To enable replication for a KV Store collection and allow lookups against that collection to be automatic:

On your search heads create a file called collections.conf in $SPLUNK_HOME/etc/system/local/ (or within an app)
[minemeldfeeds]
replicate = true

Restart Splunk to apply your changes.
Once the next bundle is replicated to your indexers the kvstore collection definition will exist at the indexers and you should see this error message go away the next search you run.

If your indexers are running a version of Splunk Enterprise that is older than 6.3, attempts to run an automatic lookup fail with a "lookup does not exist" error. You must upgrade your indexers to 6.3 or later to use this functionality.

you can learn more about automatic lookups and kvstore collection replication here:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Knowledge/Makeyourlookupautomatic

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@veddersg You must enable replication for the kvstore collection [minemeldfeeds]

KV Store collections are not bundle-replicated to indexers by default, and lookups run locally on the search head rather than on remote peers. When you enable replication for a KV Store collection, you can run the lookups on your indexers which let you use automatic lookups with your KV Store collections.

Events of sourcetype pan:threat and pan:traffic have automatic lookups configured in props.conf / transforms.conf so the automatic lookup against the minemeldfeeds collection is running on the indexers but the minemeldfeeds kvstore collection doesn't exist there because it is not being replicated from the SH to the indexers by default.

ie:
/opt/splunk/etc/apps/Splunk_TA_paloalto/default/props.conf

LOOKUP-minemeldfeeds_dest_lookup = minemeldfeeds_lookup indicator AS dest_ip OUTPUT value.autofocus_tags AS dest_autofocus_tags

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/transforms.conf
[minemeldfeeds_lookup]
collection = minemeldfeeds
external_type = kvstore
fields_list = indicator, splunk_source, value.confidence, value.first_seen, value.last_seen, value.share_level, value.sources, value.type, value.autofocus_tags, value.autofocus_benign, value.autofocus_grayware, value.autofocus_malware, value.autofocus_num_matching_artifacts, value

/opt/splunk/etc/apps/Splunk_TA_paloalto/default/collections.conf

[minemeldfeeds]
field.indicator = string
field.splunk_source = string
field.value = string

To enable replication for a KV Store collection and allow lookups against that collection to be automatic:

On your search heads create a file called collections.conf in $SPLUNK_HOME/etc/system/local/ (or within an app)
[minemeldfeeds]
replicate = true

Restart Splunk to apply your changes.
Once the next bundle is replicated to your indexers the kvstore collection definition will exist at the indexers and you should see this error message go away the next search you run.

If your indexers are running a version of Splunk Enterprise that is older than 6.3, attempts to run an automatic lookup fail with a "lookup does not exist" error. You must upgrade your indexers to 6.3 or later to use this functionality.

you can learn more about automatic lookups and kvstore collection replication here:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Knowledge/Makeyourlookupautomatic

mwarvi
Explorer

This fixed the issue for me, thank you very much.

0 Karma
Get Updates on the Splunk Community!

New Splunk Observability innovations: Deeper visibility and smarter alerting to ...

You asked, we delivered. Splunk Observability Cloud has several new innovations giving you deeper visibility ...

Synthetic Monitoring: Not your Grandma’s Polyester! Tech Talk: DevOps Edition

Register today and join TekStream on Tuesday, February 28 at 11am PT/2pm ET for a demonstration of Splunk ...

Instrumenting Java Websocket Messaging

Instrumenting Java Websocket MessagingThis article is a code-based discussion of passing OpenTelemetry trace ...