topic Re: csv file in blob storage in Splunk Search

csv file in blob storage

Skins — Thu, 20 Dec 2018 06:10:49 GMT

I am ingesting from blob storage and have downloaded an example of the file and uploaded to a standalone box and created a new sourcetype and all working as expected.

using INDEXED_EXTRACTIONS = csv

moving to my tierd environment the blob storage is collected via app running on the HF - so i have added the new sourcetype defined there and also on the SH - nothing on the indexing tier.

however searching from the SH tier - the sourcetype is shown but the fields are not extracted.

what could i be missing ?

gratzi

Re: csv file in blob storage

p_gurav — Thu, 27 Dec 2018 08:19:57 GMT

Where you are putting INDEXED_EXTRACTIONS = csv this seeting?

Re: csv file in blob storage

p_gurav — Thu, 27 Dec 2018 09:52:51 GMT

Can you give what setting you configured for sourcetype on HF and SH?

Re: csv file in blob storage

Skins — Tue, 29 Sep 2020 22:33:47 GMT

[mscs:storage:blob:csv]
DATETIME_CONFIG =
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = date
category = Structured
description = csv files from azure blob
disabled = false
pulldown_type = true

Re: csv file in blob storage

Skins — Thu, 03 Jan 2019 05:43:55 GMT

I tried again - and manually downloaded a csv file from blob storage using Azure blob explorer
If i manually add the file to the HF it is indexed using the sourcetype correctly and indexed fileds are shown and searchable from the SH (this is a HF > IDX > SH Scenario)

If i then enable the blob collection again using the mscs app - just get headers

date,level,applicationName,instanceId,eventTickCount,eventId,pid,tid,message,activityId
host =XXXX source =blah/2018/09/16/09/logname.csv sourcetype = mscs:storage:blob:csv

Re: csv file in blob storage

alexstanley — Wed, 28 Aug 2019 04:15:36 GMT

where you able to resolve this issue @Skins ?

Re: csv file in blob storage

rajasekhar14 — Thu, 14 Nov 2019 16:06:30 GMT

hi @Skins

did you resolve this issue?

Re: csv file in blob storage

rfoucault — Wed, 17 Mar 2021 08:35:19 GMT

Hello,

I'm coming to you, I'm trying to implement a BLOB to a splunk like you. I have the same concern that you have found a solution to this problem?

Have a good day