I have some blob storage and in there are different files that I need to ingest and apply different source types to.
some are error.log files
some are web access logs
some are other logs
How do I do this ?
I only have one container with all my logs in .
The only thing i can think of is sourcetype overrides - so i label my input with : mscs:storage:blob:logs
And then identify each sourcetype (as each log has a different name convention) using regex and sourcetype overrides on the HF where the MSCS app is installed.
Unless there is a better way?
Yes, I would use regex in props and transform to split up in specific sourcetypes in this case. Unless there is a better way indeed...
OK i went with creating several inputs but use the 'blob list' section to only ingest that log :
Bloblist = filetypeA.logs
sourcetype = mscs:storage:blob:fileA
Bloblist = filetypeB.logs
sourcetype = mscs:storage:blob:fileB
and so on ..