Is is possible to index a small CSV file with only 1 column (both with monitoring and manually)?

[rayar]

I am trying to index a small CSV file with only 1 column (both with monitoring and manually )

is it impossible  ?  

was able to index only after I added additional column 

for monitoring I have defined the below

inputs.conf 

[monitor:///opt/mailboxes_not_created_empid/*.csv]
disabled = 0
sourcetype = csv_current_time
index = mailboxes_not_created_empid
crcSalt = <SOURCE>
initCrcLength = 512

the csv (comma separated ) file is 

Employee_Number

141941

180536

189377

[gcusello]

Hi @rayar,

yes, it's possible, what's the problem you encountered?

Having only one clumn you don't need INDEXED_EXTRACTION.

Ciao.

Giuseppe

[rayar]

no data is indexed into index 

sorry , what do you mean you don't need INDEXED_EXTRACTION ?

[gcusello]

Hi @rayar,

if you don't have data in your index, I need to know, at first, the file to ingest is in the same Splunk server on in another one with a Universal Forwarder?

if it's in the UF, you have at first to check the connection and you can do this in two ways:

check if you already have in the _internal index the UF Splunk internal logs:

index=_internal host=your_host

if you have events, connection is ok otherwise, you have to check the connection using telnet

telnet IP_Server_Splunk 9997

if you don't have the connection, you have do some things:

• to check firewalls routes between UF and Indexer,
• to check local firewalls,
• to check the outputs.conf configuration,
• to enable Log Reaceiving on Indexer.

If instead you have internal Splunk logs, this means that connection is esteblished and you have to check the file reading, you can do this with a CLI command on the server where the file is located:

ls -la /opt/mailboxes_not_created_empid/*.csv

checking also the reading grants.

Tell me your checks.

Ciao.

Giuseppe

[rayar]

its an active UF 

I see the logs in _internal

03-03-2022 13:03:59.915 +0200 DEBUG TailingProcessor - Deferred notification for path='/opt/mailboxes_not_created_empid/Employee_Numbers_15012022_183613.csv'.

.

.

03-03-2022 13:03:56.916 +0200 DEBUG TailingProcessor - File state notification for path='/opt/mailboxes_not_created_empid/Employee_Numbers_15012022_183613.csv' (first time).

.

.

[gcusello]

Hi @rayar,

Please some other details:

• did you located the inputs.conf in the UF?
• Where did you located inputs.conf, in which folder?
• Did you used a Deployment Server or did you manually modified inputs.conf?
• Did you restarted Spluink on UF after inputs.conf modification?

Ciao.

Giuseppe

[rayar]

• did you located the inputs.conf in the UF? - yes 
• Where did you located inputs.conf, in which folder? - /opt/splunkforwarder/etc/apps/search/local/inputs.conf"
• Did you used a Deployment Server or did you manually modified inputs.conf? - manual 
• Did you restarted Spluink on UF after inputs.conf modification? - yes 

Also once I do "Add Data" manually from the SH with csv sourcetype I can't index , only if I add another column

I think the issue is in indexing 1 column csv file 

[gcusello]

Hi @rayar,

I suppose that you checked the localization of the csv files using the ls command I hinted in the first answer.

Did you tried to copy a csv file in another one with a different filename?

Obviously index mailboxes_not_created_empid is created and available.

What does it happen tryng to ingest by GUI on the SH?

Ciao.

Giuseppe

[rayar]

I tried to index the file manually from SH using CSV sourcetype and it got indexed only after I added a new column since its looking for ,

[gcusello]

Hi @rayar,

I triend to load a csv file with only one column.

It runs, using this props.conf on Indexer.

[test_csv]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=AUTO
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Structured
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true

Ciao.

Giuseppe

[gcusello]

Hio @rayar,

sorry I forgot INDEXED_EXTRACTION: it's a way to assign fields to csv or json files, but, having you only one column it could be not relevant.

Ciao.

Giuseppe

[rayar · Accepted solution]

I used the below 

[csv_1_column_test]
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG = CURRENT
KV_MODE = none
LINE_BREAKER = [(\n\r]+)
SHOULD_LINEMERGE = false
category = Structured
description = Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = true
CHARSET = AUTO
HEADER_FIELD_LINE_NUMBER = 1
INDEXED_EXTRACTIONS = csv
NO_BINARY_CHECK = true

[gcusello]

JI @rayar,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉