Splunk Search

CSV Lookup File From Share

Path Finder

Is it possible to set Splunk to read a lookup from a CSV file on a different machine? We have a shared directory internally and ideally we'd let to have our CSV files for the lookups on the share to allow modification without need for uploading or logging directly onto the Splunk server

Tags (2)
0 Karma
1 Solution

Path Finder

There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental. Search for importutil on Splunkbase

Here is an example that imports data from the federal reserve economic data website:

|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE

Here is an example that imports data from the Bureau Of Labor Statistics... it uses ftp:

|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes

View solution in original post

Path Finder

There is a new splunkbase app called "importutil". It lets you import csv files (or any input) from an http url via the splunk search command line. Also works for ftp. sftp is experimental. Search for importutil on Splunkbase

Here is an example that imports data from the federal reserve economic data website:

|importutil http http://research.stlouisfed.org/fred2/data/PAYEMS.csv
| multikv
| table DATE, VALUE

Here is an example that imports data from the Bureau Of Labor Statistics... it uses ftp:

|importutil ftp ftp://ftp.bls.gov/pub/time.series/ce/ce.data.102.WeeklyEarningsHist
| multikv
| table series_id, year, period, value, footnote_codes

View solution in original post

Path Finder

paddy, good luck. If this ends up working for you and if you have a moment please rate the app on splunkbase. Thanks. Good Luck!

0 Karma

Path Finder

Many thanks, this looks perfect.

0 Karma

Legend

I think the following will work, but I'm not able to test it.

To make the English easier, assume that the user account running Splunk (splunkd and splunk web) is named splunkAcct.

First, make sure that splunkAcct can open and read files in the shared directory.

Second, put a link in the proper Splunk directory (lookups) that points to the correct file in the shared directory.
On Linux, use a hard link. On Windows, it's a short cut. Do this as splunkAcct so that the permissions/ownership are appropriate.

If that doesn't work you could write a script that runs on the Splunk server and copies over the files every hour.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!