topic Re: How to get Splunk to index a small 1.5KB CSV file? in Getting Data In

How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Mon, 23 May 2016 19:14:47 GMT

I am trying to make Splunk read/index a CSV that is of 1.5KB.

I have used the traditional CRCSALT=>SOURCE> tag in the inputs.conf, however, that does not seem to work.

Any ideas, how i can make the file read? It is a once a week feed and it does not process the files.

Re: How to get Splunk to index a small 1.5KB CSV file?

jkat54 — Mon, 23 May 2016 19:43:44 GMT

Should be crcSalt=<SOURCE> capitalization / case sensitivity matters.

Also i think its a typo but there should be a less than sign on left side of SOURCE and greater than sign on right side.

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Mon, 23 May 2016 19:47:15 GMT

Yes. It is a typo. That is what happens when you break your head against such unforeseen issues.

And thus the capslock was ON.

But this did not work. 😞

Re: How to get Splunk to index a small 1.5KB CSV file?

jkat54 — Mon, 23 May 2016 20:12:27 GMT

Can you post the csv file somewhere like pastebin.com?

If its 1.5KB that means its ~ 1500B which means it has well over 256 characters if in ASCII because each ASCII character = 1Byte on disk. So there's enough data to form a proper CRC salt.

Also if you could provide the inputs.conf and props.conf stanzas for the data source maybe there is a config issue there.

Re: How to get Splunk to index a small 1.5KB CSV file?

somesoni2 — Mon, 23 May 2016 20:13:59 GMT

Are you using batch OR monitor in inputs.conf?

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Tue, 12 Jul 2016 15:31:33 GMT

@jkat54

Sorry to get back to you on this after a long time.

Inputs.conf :
[monitor:///Data/scripts/email_listener/logs/username/usernames*.csv]
sourcetype = username
source = username
host = username
index = main
crcSalt=

THere is no props.conf created.

Also assume the CSV contains numbers in one single column. Contains random numbers for around 132 rows.

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Tue, 12 Jul 2016 15:34:17 GMT

[monitor:///Data/scripts/email_listener/logs/username/usernames*.csv]

sourcetype = username
source = username
host = username
index = main
crcSalt = <SOURCE>

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Tue, 12 Jul 2016 15:34:17 GMT

[monitor:///Data/scripts/email_listener/logs/username/usernames*.csv]

sourcetype = username
source = username
host = username
index = main
crcSalt = <SOURCE>

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Tue, 12 Jul 2016 15:36:29 GMT

I am unable to use the crcSalt = [] in the comment box. It is getting invisible.

Re: How to get Splunk to index a small 1.5KB CSV file?

jkat54 — Tue, 12 Jul 2016 16:29:17 GMT

put the 'code' in codeblocks. Highlight it and click the 101010 button.

This will make

 crcSalt=<SOURCE>

show up

Re: How to get Splunk to index a small 1.5KB CSV file?

hardikJsheth — Wed, 13 Jul 2016 05:43:41 GMT

Please add following stanza for your source type in props.conf on indexer. Without this, Splunk will not index csv file correctly.

[sourcetype]
INDEXED_EXTRACTIONS=CSV

Also if the CSV file is already read, splunk will not re read it without clearing fish bucket. To know more about how to clear fishbucket, refer answer from Yann.https://answers.splunk.com/answers/72562/how-to-reindex-data-from-a-forwarder.html

Re: How to get Splunk to index a small 1.5KB CSV file?

vr2312 — Tue, 31 Jan 2017 16:13:58 GMT

Using "initCrcLength = XX" in the inputs.conf fixed the issue.