topic Re: Small CSV file indexing in Getting Data In

Help with small CSV file indexing

rayar — Tue, 13 Sep 2022 13:39:10 GMT

I am trying to index a small CSV file with 2 columns and Size -5.32 KB (5,453 bytes) , Size on Disk - 8.00 KB (8,192 bytes) by Heavy Forwarder

on the forwarder I see that shows 0 files

inputs.conf

[monitor://\\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List\]
disabled = 0
index = websense_large_web_traffic
sourcetype = csv
crcSalt = <SOURCE>
initCrcLength = 512

Re: Small CSV file indexing

gcusello — Tue, 13 Sep 2022 11:48:03 GMT

Hi @rayar,

have you data in Splunk (using a search)?

the dashboard of your screenshot isn't relevant, see in the search dashboard of the Search Head.

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Tue, 13 Sep 2022 12:04:50 GMT

the data is not indexed

I also don't see any activities in index=_*

and the issue that I see that the HF see 0 files under the path

Re: Small CSV file indexing

gcusello — Tue, 13 Sep 2022 12:11:21 GMT

Hi @rayar,

you have to search something like

index=websense_large_web_traffic source="*\<your_file_name>"

not in _* indexes.

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Tue, 13 Sep 2022 12:14:44 GMT

the data is not indexed to the index

also I don't see any events in the internal indexes

what can be the reason HF doesn't recognize filers ?

I copied the same file to my local and was able to index manually

Re: Small CSV file indexing

rayar — Tue, 13 Sep 2022 12:31:02 GMT

just notices that if I add data manually from the HF itself the data is not indexed also

what can be the reason ?

Re: Small CSV file indexing

gcusello — Tue, 13 Sep 2022 12:35:38 GMT

i @rayar,

maybe the filename is missing, please try to use in your inputs.conf:

[monitor://\\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List\*.csv]

or adding at the end of the path the filename with extension.

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Tue, 13 Sep 2022 15:35:17 GMT

[monitor://\\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List\*.csv]
disabled = 0
index = websense_large_web_traffic
sourcetype = csv
crcSalt = <SOURCE>
initCrcLength = 512

still I see 0 in the heavy forwarder

Re: Small CSV file indexing

gcusello — Tue, 13 Sep 2022 15:39:12 GMT

Hi @rayar,

this is a network folder, have the user you're using grants to access this folder?

if you run in a cmd window

dir \\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List\*.csv

have you results?

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Tue, 13 Sep 2022 15:45:36 GMT

C:\Users\issplunk>dir \\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List\*.csv
Volume in drive \\ntnet\filestore1 is SCCM Content
Volume Serial Number is 1EFA-6F4C

Directory of \\ntnet\filestore1\information_security$\ACSC_Websense_Large_Web_Traffic_Exclusion_List

09/12/2022 04:01 PM 5,453 Websense_Lare_Web_Traffic_Exclusion_August_2022.csv
09/13/2022 03:23 PM 5,458 Websense_Lare_Web_Traffic_Exclusion_082022.csv
2 File(s) 10,911 bytes
0 Dir(s) 211,867,983,872 bytes free

C:\Users\issplunk>

Re: Small CSV file indexing

gcusello — Wed, 14 Sep 2022 06:34:16 GMT

Hi @rayar,

please run this last try:

change the name of your file and see if now it's indexed, because Splunk doesn't index a file twice, the only way to do this is using crcSalt and changing a filename.

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Wed, 14 Sep 2022 08:27:46 GMT

Hi
I already tried it before and it still shows 0 files

Re: Small CSV file indexing

gcusello — Wed, 14 Sep 2022 08:32:41 GMT

Hi @rayar,

last try,

could you try to copy your file in a folder without "$" in the path, changing the input stanza to the new folder?

Ciao.

Giuseppe

Re: Small CSV file indexing

rayar — Thu, 15 Sep 2022 05:39:05 GMT

I moved the monitoring to Linux UF and it resolved the issue

thanks

Re: Small CSV file indexing

gcusello — Thu, 15 Sep 2022 07:33:22 GMT

Hi @rayar,

it's always a good idea!

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉