All Apps and Add-ons
Highlighted

How to differentiate different sourcetypes when ingesting from blob storage?

Path Finder

I have some blob storage and in there are different files that I need to ingest and apply different source types to.

eg.
some are error.log files
some are web access logs
some are other logs

How do I do this ?

Gratzi.

0 Karma
Highlighted

Re: How to differentiate different sourcetypes when ingesting from blob storage?

Builder

Hi,

I'm also playing around with this.
In the SplunkTAmicrosoft-cloudservices/inputs I have created different inputs for each container name.
Each Container Name gets an own sourcetype.
alt text

0 Karma
Highlighted

Re: How to differentiate different sourcetypes when ingesting from blob storage?

Path Finder

hi ..thanks.

I only have one container with all my logs in .

The only thing i can think of is sourcetype overrides - so i label my input with : mscs:storage:blob:logs

And then identify each sourcetype (as each log has a different name convention) using regex and sourcetype overrides on the HF where the MSCS app is installed.

Unless there is a better way?

gratzi

0 Karma
Highlighted

Re: How to differentiate different sourcetypes when ingesting from blob storage?

Builder

Yes, I would use regex in props and transform to split up in specific sourcetypes in this case. Unless there is a better way indeed...

0 Karma
Highlighted

Re: How to differentiate different sourcetypes when ingesting from blob storage?

Path Finder

OK i went with creating several inputs but use the 'blob list' section to only ingest that log :

Input1:
Bloblist = filetypeA.logs
sourcetype = mscs:storage:blob:fileA

Input2:
Bloblist = filetypeB.logs
sourcetype = mscs:storage:blob:fileB

and so on ..