Splunk Add-on for Microsoft Cloud Services: How to edit props.conf for Azure Storage Blob multi-event JSON ingestion?

[pkeller]

Using the Splunk Add-On for Microsoft Cloud Services to pull Azure data ... I'm having some difficulty indexing Azure Storage Blobs ... Anyone have any suggestions for props.conf on the sourcetype? I'm currently doing this, but events are still being broken in random places.

[mscs:storage:blob]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)\{
TRUNCATE = 0
KV_MODE = json

I'd welcome any better suggestions.

props.conf has been deployed to the heavy forwarder, all indexers, and the search head cluster (although I'm not sure that it's necessary there)

Splunk version HF - 6.4.3, Indexers - 6.4.3, Search 6.4.3

[aaraneta_splunk]

@pkeller - Did the answer provided by MuS help provide a working solution to your question? If yes, please don't forget to resolve this post by clicking "Accept". If no, please leave a comment with more feedback. Thanks!

[MuS · Accepted solution]

Hi pkeller,

just had the same troubles and made it work with this props.conf

[mscs:storage:blob]
KV_MODE = json
MAX_TIMESTAMP_LOOKAHEAD = 800
NO_BINARY_CHECK = true
category = Structured
description = A variant of the JSON source type, with support for Azure NGS blob format
disabled = false
pulldown_type = true
BREAK_ONLY_BEFORE_DATE = false
LINE_BREAKER = (,[\r\n]+\s+)\{
SEDCMD-mscs:storage:blob-StripHeader = \{\s+"records":\s+\[\s+

Deployed it to the indexers and the search heads.

Hope this helps ...

cheers, MuS

[hossyee]

I think that it is necessary to delete the trailer as well.
[mscs:storage:blob]
SEDCMD-removetrailer = s/]\s+}// 　　　<< ADD