Updating LookUp Table Data Externally - 'Auto-magically'

[rgcurry]

I am wanting to create a process that will make it really simple and easy for my users to update their lookup table files without having to go into "Manager / Lookups / Lookup table files" to delete then add the new file. Ideally, they'd simply point to their app, the browse to the new file and click a button. Has anyone else wanted to provide that and built something to provide it?

Thought I'd inquire to see what may have already been built that can be shared rather than re-invent it. Or perhaps there is enough interest that we can build a Splunk App for this?

What are your thoughts?

[bmacias84 · Accepted solution]

This can be accomplished using saved searches and state tables with in your apps.

Here is a wiki article using advanced XML: Dynamically_Editing_Lookup_Tables

Also below is a simple search example that will dynamically updates the table each time the populating search runs.

...| table serverName IP  upDown _time | inputlookup append=t serverIP.csv | outputlookup serverIP.csv

Splunk Enterprise Security App uses similar techniques.

Hope this helps or gets your started. Dont forget to vote up and accept answers that help.

[sideview · Accepted solution]

The Sideview Utils app has a little interface called the Lookup Updater. It allows you to

• edit any row in an existing lookup,
  
• add a new row to an existing lookup
• reupload an existing lookup with a new flat file (it must have the same header row).
• delete a row in an existing lookup.

Note that you'll have to get the app from the Sideview site at http://sideviewapps.com/apps/sideview-utils because the older version of the app on Splunkbase doesn't have it.

Also it's recently come up that the tool has a bug with lookups that have owners. If you have a lookup that is simply packaged in an app, it'll have "no owner", in the parlance of the Admin section. However with a lookup that is marked in Manager as having an owner, it'll give you an error starting with the second time you use the tool. I'll get the bug fixed soon but probably not for a few weeks.

[Brian_Hopps]

4/4/2017 I used Sideview Apps lookup Updater and it worked well. Uploaded a new file replacing 40,000 records with 160,000 records.

[meenal901]

Hey..

I just tried Lookup Updater and it's awesome!!
Just a quick question - I want to pass value to the view through URL for pre-populate.
Is is possible?

[rgcurry]

Thank you for this update info on SV, it looks like it can work for most of the updates our users need to make. There are some, albeit rarel cases where the update is to include a new or additional columns so for those situations this won't work. Any chance this can be done? Many of our lookup tables are not used for "Automatic Lookups", they are loaded via "inputlookup" and a search used to extract what is needed; such as to fill a listbox. Your post also provides me with some of the info I need to make this work. Thanks for this and a a great product in Sideview Utilities.

[bmacias84 · Accepted solution]

This can be accomplished using saved searches and state tables with in your apps.

Here is a wiki article using advanced XML: Dynamically_Editing_Lookup_Tables

Also below is a simple search example that will dynamically updates the table each time the populating search runs.

...| table serverName IP  upDown _time | inputlookup append=t serverIP.csv | outputlookup serverIP.csv

Splunk Enterprise Security App uses similar techniques.

Hope this helps or gets your started. Dont forget to vote up and accept answers that help.

[rgcurry]

THis is good info and I already do that for a couple of our lookup tables whose data is in an index without the Splunk data store. However, there are also some lookup tables in which the data source is external to Splunk; I was not clear about that in my post. The wiki article looks like it may provide me with a foundation from which I might be able to get this done. Thanks!