topic Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ? in Getting Data In

how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

chaitanya1996 — Mon, 27 Apr 2020 13:52:18 GMT

Hello,
I have a csv file generated by script daily at $SplunkHome\etc\apps\bin\'fuel_stations.csv'. I add manually that CSV file as Lookup table files using "settings> lookups> Lookup table files> add new" to use it for my splunk search |inputlookup fuel_station.csv.

Now I want to automate to update lookup file whenever this csv file in above path is updated.
How do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?
splunk v.6.6.3
Thanks

Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

prachisaxena — Mon, 27 Apr 2020 20:13:06 GMT

Hi,

i would suggest monitoring this file $SplunkHome\etc\apps\bin\'fuel_stations.csv' using the splunk file monitoring. So, that the data in splunk is indexed everytime you update the file.

Then you can schedule a search to this data to lookup csv file using outputlookup command.
index =
| table field1, field2
|outputlookup fuel_station.csv append=(true|false)

Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

MuS — Mon, 27 Apr 2020 20:27:25 GMT

Hi chaitanya1996,

Easiest solution for this is to change the script to output the csv into the correct directory $SPLUNK_HOME\etc\apps\<yourApp>\lookups\fuel_stations.csv this will use the updated lookup file immediately when using | inputlookup afterwards.

This will only work if the script is not running in a Search Head Cluster; if you plan to run it in a search head cluster you can use this app https://splunkbase.splunk.com/app/4649/#/overview

Hope this helps ...

cheers, MuS

Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

chaitanya1996 — Tue, 28 Apr 2020 13:59:23 GMT

Hi @MuS ,

I have fuel_stations.csv file at indexer in Search Head Cluster. I want to update the lookup in search head and do |inputlookup across all search heads.

can you suggest to get this in any possible way.
And can you please elaborate it whether we can use the app you suggested to get this done.

Thanks,
Chaitanya

Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

chaitanya1996 — Tue, 28 Apr 2020 14:07:35 GMT

Hi @prachisaxena,

When we monitor the csv file, if we change the value of field2 in row 2 from 5 to 4.. it gets into splunk.
But if we again change the field2 in row 2 from 4 to 5 , it doesn't get into splunk and iam not able to get the latest updated one due to this.

Thanks,
Chaitanya

Re: how do I get the lookup table to update automatically whenever the CSV file is updated in the specific local file ?

MuS — Tue, 28 Apr 2020 20:10:30 GMT

Hi chaitanya1996,

Lookup files should be created, stored and use on search heads. Just read the details of the app and you will get all details needed to learn how you can update the lookup file in a search head cluster.

cheers, MuS