topic Can splunk read logs in CSV format? in Getting Data In

Can splunk read logs in CSV format?

balbano — Tue, 25 May 2010 21:28:57 GMT

Hi, we are currently testing a Palo Alto app sec firewall and are sending some test logs over to the central indexer in CSV format. Can splunk read CSV files?

The splunkd.log file is giving me somewhat of an error stating the following:

05-25-2010 14:21:33.678 ERROR TailingProcessor - Ignoring path due to: File will not be read, is too small to match seekptr checksum (file=PATHTOMYFILES). Last time we saw this initcrc, filename was different. You may wish to use a CRC salt on this source. Consult the documentation or contact Splunk Support for more info.

Let me know. Thanks!

Brian

Re: Can splunk read logs in CSV format?

Nicholas_Key — Tue, 25 May 2010 21:42:48 GMT

sure. please test if it works with

crcSalt = <SOURCE>

in the particular stanza of your app's inputs.conf

Re: Can splunk read logs in CSV format?

balbano — Tue, 25 May 2010 23:00:42 GMT

Doesn't look like it worked. Here is my stanza:

[monitor:///pathtomyfiles]
disabled = false
host =
host_segment = 3
index = myindex
sourcetype = mysourcetype
crcSalt = /pathtomyfiles

Is this correct syntax? not seeing the error message anymore but it still doesn't look like the files are being indexed.

Brian

Re: Can splunk read logs in CSV format?

Lowell — Wed, 26 May 2010 00:26:23 GMT

No, you want the literal value "<SOURCE>". That is a special value with a special meaning. Take a look at http://www.splunk.com/base/Documentation/4.1.2/Admin/Inputsconf

Re: Can splunk read logs in CSV format?

Lowell — Wed, 26 May 2010 00:32:26 GMT

Also, keep in mind that anytime you change crcSalt you are likely to re-index all your files. You should also be aware that setting crcSalt = <SOURCE> means that if you rotate your log files they may be indexed again. Normally splunk detects and handles this automatically, but using <SOURCE> will effectively disable this feature. So you may need to update your rotation scheme to move log files to a different folder, or make sure your monitor is only looking at your non-rotated files. Just a heads up.

Re: Can splunk read logs in CSV format?

balbano — Wed, 26 May 2010 07:05:21 GMT

sweet... that worked!!!!!!!!! Thanks GUYS!!!!!

Re: Can splunk read logs in CSV format?

balbano — Wed, 26 May 2010 07:06:04 GMT

and Lowell since this was a test sample of csv logs the impact was not great but in the future I will definitely keep that in mind!!! Thanks again!!!