topic Re: How to parse csv (blob) data of azure to microsoft azure addon in Getting Data In

How to parse csv (blob) data of azure to microsoft azure addon

lmjoin — Tue, 23 Apr 2019 14:04:12 GMT

Hello ,

How to parse csv (blob) data of azure to microsoft azure addon. we are not getting data from blobs csv file in right format.

Thanks

Re: How to parse csv (blob) data of azure to microsoft azure addon

lmjoin — Wed, 30 Sep 2020 00:13:10 GMT

getting error:-

4/23/19
2:58:44.918 PM

... 19 lines omitted ...
File "C:\Program Files\Splunk\etc\apps\Splunk_TA_microsoft-cloudservices\bin\splunktamscs\azure\storage_serialization.py", line 125, in _storage_error_handler
return _general_error_handler(http_error)
File "C:\Program Files\Splunk\etc\apps\Splunk_TA_microsoft-cloudservices\bin\splunktamscs\azure\storage_error.py", line 74, in _general_error_handler
raise AzureHttpError(message, http_error.status)
AzureMissingResourceHttpError: The specified resource does not exist.
Show all 27 lines
host = gns3 source = C:\Program Files\Splunk\var\log\splunk\splunk_ta_microsoft-cloudservices_storage_blob_matazc_B64_bWF0YXpj.log sourcetype = mscs:storage:blob:log
4/23/19
2:57:05.917 PM

2019-04-23 14:57:05,917 +0000 log_level=ERROR, pid=3016, tid=ThreadPoolExecutor-0_0, file=mscs_storage_blob_data_collector.py, func_name=collect_data, code_line_no=66 | [stanza_name="asplunk" account_name="appppp" container_name="None" blob_name="None"] Error occurred in collecting data
... 9 lines omitted ...
AzureHttpError: The range specified is invalid for the current size of the resource.
InvalidRangeThe range specified is invalid for the current size of the resource.
RequestId:c1160468-e01e-0091-34e4-f95cfe000000
Time:2019-04-23T14:57:05.8017930Z
Show all 14 lines
host = gns3 source = C:\Program Files\Splunk\var\log\splunk\splunk_ta_microsoft-cloudservices_storage_blob_asplunk_B64_YXNwbHVuaw==.log sourcetype = mscs:storage:blob:log
4/23/19
2:57:05.571 PM

... 19 lines omitted ...
File "C:\Program Files\Splunk\etc\apps\Splunk_TA_microsoft-cloudservices\bin\splunktamscs\azure\storage_serialization.py", line 125, in _storage_error_handler
return _general_error_handler(http_error)
File "C:\Program Files\Splunk\etc\apps\Splunk_TA_microsoft-cloudservices\bin\splunktamscs\azure\storage_error.py", line 74, in _general_error_handler
raise AzureHttpError(message, http_error.status)
AzureMissingResourceHttpError: The specified resource does not exist.
Show all 27 lines
host = gns3 source = C:\Program Files\Splunk\var\log\splunk\splunk_ta_microsoft-cloudservices_storage_blob_matazc_B64_bWF0YXpj.log sourcetype = mscs:storage:blob:log

Re: How to parse csv (blob) data of azure to microsoft azure addon

p_gurav — Wed, 24 Apr 2019 13:50:32 GMT

This may help:
https://www.splunk.com/blog/2017/08/18/splunking-microsoft-cloud-data-part-2.html

Re: How to parse csv (blob) data of azure to microsoft azure addon

sahilverma — Thu, 09 Jan 2020 18:20:39 GMT

Hello,

Were you able to fix this; We have azure app logs in CSV stored in blob.
with mscs:storage:blob its not working.
Lemme know the sourcetype if you have got it fixed.

thx

Re: How to parse csv (blob) data of azure to microsoft azure addon

sguez — Mon, 23 Aug 2021 10:05:06 GMT

Did anyone managed to resolve that issue, we are facing the same problem seem very basic function that is not working as expected, the linked mention guide is useless as it does not solve the issue

tried to use an existing source type like

csv

mscs:storage:blob:csv

trying to create a custom and somehow force header read with all the available options all are seem to be ignored , very frustrating loading simple 5 line + header file onto blob storage and although we see the file in splunk as comma delimited it never auto extract the fileds