topic How to set up a Splunk DB Connect 2 Lookup that is available in the Search App? in All Apps and Add-ons

How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

bshamsian — Thu, 03 Sep 2015 18:57:28 GMT

I am using DB Connect 2 to pull information from database into a lookup table. It is setup correctly but the lookup table is not available in Search app. It looks like it is only available under the DB Connect App. What do I have to do to make it available globally? I found the lookup definition created by DB Connect and changed the permission to global but I still get the following error when I try to use the lookup table in Search App:

Error in 'lookup' command: The lookup table 'db_connect_CarrierLookup' does not exist.

If I run the exact search under DB Connect V2 app it works.

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

bshamsian — Fri, 04 Sep 2015 18:44:27 GMT

DB Connect looks more and more useless every time I try to use it. I had issues before with using DB Connect with outputting data from Splunk - now it looks like lookup using DB Connect is also useless.

According to Splunk Support any data pulled into Splunk using DB Lookup is only available while you are in the DB Connect App context so Lookup tables created by DB Lookup are not available to be used ins searches or alerts or dashboards outside of DB Connect App.

How useless is this feature. If I am importing data using DB Connect to a lookup table it should be obvious that I want to use it in Search App or a dashboard I created under Search App or an alert I setup. What use is it to anyone if it can only be seen under the DB Connect App.

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

somesoni2 — Fri, 04 Sep 2015 18:55:36 GMT

Try these steps

1) Go to Settings->Lookups -> Lookup table files, change the App context to DB Connect and locate your lookup table.
2) In the right most column Actions, click on Move for the record of your lookup table and select Search as application and click on Move.
3) You can see the object moved to Search apps and App context is changed to Search. Optionally, you can change it's Sharing to All apps (by clicking on Permissions under Sharing column).

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

bshamsian — Fri, 04 Sep 2015 19:02:19 GMT

That is the problem - The tables that are created dynamically are not listed there under Lookup Table Files. It is hidden and I have no access to it to change the permission.

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

jcoates_splunk — Mon, 05 Oct 2015 20:05:19 GMT

please try adding these to the local.meta file. this should resolve the issue.

[transforms]
export = system

[searchscripts]
export = system

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

hgehrts_splunk — Tue, 13 Oct 2015 12:04:19 GMT

into which of the local.meta files?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

bshamsian — Thu, 05 Nov 2015 19:11:31 GMT

Same question as above that which local.meta file are you talking about? If you are talking about the user local.meta then I have to note that we are using LDAP and have lots of users and cannot really go and create local.meta for each user. There really should be a better solution for this.

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

steven_swor — Mon, 01 Feb 2016 08:50:10 GMT

Okay, thanks in part to the comment by @jcoates, I was able to get it working.

There are 3 components of a DB lookup (4 for automatic lookups):

dblookup.py (the python script which does the heavy lifting)
a lookup definition defined in transforms.conf
a lookup input defined in inputs.conf

In order to get DB lookups defined in other apps to work, you need (at minimum):

dblookup.py exported globally
your lookup definition exported globally

Here's how I did it.

Export `dblookup.py` globally

Add the following stanza to $SPLUNK_HOME/etc/apps/splunk_app_db_connect/metadata/local.meta to export the dblookup.py script globally (I couldn't figure out a way to do this via the web interface)

[searchscripts/dblookup.py]
export = system

(At this point, I restarted Splunk)

Define your DB lookup

I used the GUI and selected the app context at the end of the wizard. If you've defined it in the wrong place, you should move the lookup definition via the DB Connect app instead of via the normal Splunk settings page (e.g. edit the config and change the app context), because there's also an input that needs to be moved.

Defining the DB lookup will create the following:

in $SPLUNK_HOME/etc/apps//local/transforms.conf:

[<lookup name>]
external_cmd = dblookup.py <lookup name>
fields_list = <comma-separated list of fields you've defined>

in $SPLUNK_HOME/etc/apps//local/inputs.conf:

[mi_lookup://<lookup name>]
... # there's about 16 entries under this stanza. It contains the raw SQL for the lookup, as well as some other bits and pieces used by the gui

In $SPLUNK_HOME/etc/apps//metadata/local.meta:

[transforms/<lookup name>]
...

If you've defined the lookup as an automatic lookup, you'll also have entries in $SPLUNK_HOME/etc/apps//local/props.conf like so:

[<spec>]
LOOKUP-db_connect_<lookup name> = <lookup name> <field> AS <alias> OUTPUTNEW <field>...

and in $SPLUNK_HOME/etc/apps//metadata/local.meta:

[props/<props_spec>/LOOKUP-db_connect_<lookup name>]
.... # note: setting export = system here is optional. The automatic lookups will work if searches are run from within your app context.

Export your lookup definition globally

You can do this via the settings page in Splunk (you want the lookup definition, not the lookup table).

Alternatively, in $SPLUNK_HOME/etc/apps//metadata/local.meta:

[transforms/<lookup name>]
export = system
... # other stuff already added by Splunk

topic How to set up a Splunk DB Connect 2 Lookup that is available in the Search App? in All Apps and Add-ons

How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Re: How to set up a Splunk DB Connect 2 Lookup that is available in the Search App?

Export dblookup.py globally

Define your DB lookup

Export your lookup definition globally

Export `dblookup.py` globally