Importing csv files from directory

[Sasquatchatmars]

Hi all,

I have been trying to monitor a directory with csv files. Let me explain. I have multiple PS scripts running and they are exporting the results to csv files in a directory. I have configured a data input on the corresponding directory and whitelisted the csv files. Which gives me the following in the input.conf file. 

[monitor://C:\Program Files\Splunk\etc\apps\search\bin\Powershell\Results]
disabled = false
index = powershell_scripts
whitelist = \.csv$

Everytime I run a PS script to test if the input works, the script creates the csv file or updates it but it isn't ingested in Splunk. Does someone knows why this could be? 

Thank you,

Sasquatchatmars

[gcusello · Accepted solution]

Hi @Sasquatchatmars,

If the result is always the same, the file isn't indexed twice.

If you could run the PS script from Splunk as scripted inputs, you don't have any problem because the script output is sent directly to Splunk.

For more infos, see at https://docs.splunk.com/Documentation/SplunkCloud/latest/AdvancedDev/ScriptedInputsIntro

Ciao.

Giuseppe

[Sasquatchatmars]

Hi @gcusello,

It doesn't indexes it at all. 

I tried the modular input. Somehow at some points the scripts sees some kind of errors because it is based on a list of servers. These servers are not always working so it generates an error. At that moment the idexing stops and doesn't continue. 

By the way I tried indexing it file by file which works. But what i really want is to monitor all the csv files in the directory without everytime needing to specify the file path in the data inputs.

Thanks,

Bob van Scheijndel

[gcusello]

Hi @Sasquatchatmars,

the content of the files is frequently the same or it's always different?

If it's always the same, Splunk doesn't index twice a file also with a different name.

The filenames are always the same or ther are different?

try to add crcSalt = <SOURCE> to the input stanza and restart forwarder.

Ciao.

Giuseppe

[Sasquatchatmars]

Hi @gcusello,

This did work tahnk you but I found I found an easier way. I just added a TimeStamp column to my csv file so the file changes every time. 

Thank you anyway! 

Sasquatchatmars

[gcusello]

Hi @Sasquatchatmars,

as I said, Splunk reads a file and, if there are differences, indexs the file or the new lines, otherwise it doean't index the file.

Adding a column with timestamp you modify every time the file so splunk understand that has to index it.

Good for you.

Please accept the answer for the other people of Community.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated 😉

[Sasquatchatmars]

Hi @gcusello,

Thanks yes indeed, you said that 😊

Oh sorry I forgot, I'll accept it right away.

Thank you,

Sasquatchatmars