topic Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed. in Getting Data In

How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

alenseb — Tue, 11 Sep 2012 10:03:38 GMT

Hi All,

I am trying to load a .csv file into splunk, using sourcetype(csv). Upload of data is working fine but the only issue if i change the data in this .csv file, new data get appended to the old data. what i need is to replace the old with new.
Example: if i have 4 lines in the csv initially & if i delete one of its line and add 2 new lines.
My new csv data should contain only 4 lines.
Right now when i am doing this, i am getting 9 lines(4-Old & 5-New) of data.

Please help me with the configurations.

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

MHibbin — Tue, 11 Sep 2012 10:19:02 GMT

Do you not want to do this as a lookup?... http://docs.splunk.com/Documentation/Splunk/latest/User/CreateAndConfigureFieldLookups

If not you should be able to use the dedup command to show the latest event (i.e. the latest csv) depending on how you are indexing it.

MHIbbin

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

Paolo_Prigione — Tue, 11 Sep 2012 10:26:39 GMT

CSVs the way you mean them are treated in a different way than regular log files.
There are 2 basic kinds:

"just CSVs", which are only accessed via "| inputcsv" and "| outputcsv"
lookup CSVs, which are accessed with commands "| lookup", "| inputlookup" and "| outputlookup"

In the former case you can:

load your CSV in $SPLUNK_HOME/var/run/splunk/filename.csv
execute the search:

| inputcsv filename.csv

In the latter you can:

configure it from the Manager and upload the csv file
you will be able to replace the file under this directory $SPLUNK_HOME/etc/apps/APPNAME/lookups/filename.csv.
execute searches like:

.... | lookup configname inputfield OUTPUT outputfield

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

alenseb — Tue, 11 Sep 2012 10:33:34 GMT

I tried the lookup but it didn't serve my purpose since the replacement of data wasn't happening.
Also i need to join this .csv file with another log file.
Therefore wen the data in csv file change, the result of the Join Query also should change.
Any Configuration files i need to look into. I am basically trying to automate it.

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

alenseb — Tue, 11 Sep 2012 10:34:13 GMT

Thanks in advance!!

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

alenseb — Tue, 11 Sep 2012 10:50:57 GMT

Thanks!! Appending worked Just fine!

Can i use this "| inputcsv" command to Join with another log file?

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

Paolo_Prigione — Tue, 11 Sep 2012 10:54:08 GMT

It will probably will be slower than with the "| lookup" command and limited to some 50k results, but | join joinfield [|inputcsv ... | fields + joinfield otherfield] might do

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

jpmackl — Thu, 19 Nov 2015 14:46:52 GMT

Any luck completing this effort? I'm trying to do the same thing. I have tool the produces a CSV report every hour, I would like to pull the data into Splunk in an automated fashion and then build real time dashboards from the data. Thank you for any help!

Re: How to load a CSV file into Splunk & Change the csv file dynamically, so that data is refreshed.

colinmchugo — Fri, 02 Sep 2016 11:10:51 GMT

Any luck with this jpmackl ? I want to do something similar