CSV Lookup File From Share

[paddy3883]

Is it possible to set Splunk to read a lookup from a CSV file on a different machine? We have a shared directory internally and ideally we'd let to have our CSV files for the lookups on the share to allow modification without need for uploading or logging directly onto the Splunk server

[nicholasgrabows · Accepted solution]

There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental. Search for importutil on Splunkbase

Here is an example that imports data from the federal reserve economic data website:

|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE

Here is an example that imports data from the Bureau Of Labor Statistics... it uses ftp:

|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes

[nicholasgrabows]

paddy, good luck. If this ends up working for you and if you have a moment please rate the app on splunkbase. Thanks. Good Luck!

[paddy3883]

Many thanks, this looks perfect.

[lguinn2]

I think the following will work, but I'm not able to test it.

To make the English easier, assume that the user account running Splunk (splunkd and splunk web) is named splunkAcct.

First, make sure that splunkAcct can open and read files in the shared directory.

Second, put a link in the proper Splunk directory (lookups) that points to the correct file in the shared directory.
On Linux, use a hard link. On Windows, it's a short cut. Do this as splunkAcct so that the permissions/ownership are appropriate.

If that doesn't work you could write a script that runs on the Splunk server and copies over the files every hour.