Hello @mgherman HTTP Event Collector expects all the values to be prepared in the right format. Fluent-bit does not do that, connect for kubernetes does solve this problem. We can also offer our solution https://www.outcoldsolutions.com/ with the pre-build dashboards and alerts https://splunkbase.splunk.com/app/3743/, it solves the problem of forwarding logs in the expected format. You can also define fields extraction on the source level by specifying the annotations, see https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#extracting-fields-from-the-container-logs We have also a performance comparison between collectord (our solution), fluent-bit and fluentd https://www.outcoldsolutions.com/blog/2018-11-19-performance-collectord-fluentd-fluentbit/ ... View more

To answer my own question: adding a regex that matched everything to the transforms.conf has it now working... though I'm sure there must be a better way to do this. ... View more

As if by magic. Thanks again. ... View more

It appears to have worked fine, and subsequent bucket rolls have dropped them into the correct directory. Thanks for the response. ... View more

Im not sure where you have found the above option, it might work, but is definitely not documented.. Anyhow, a while back i submitted a bug about .csv files and oneshot not respecting the props/transforms but i have not tried it with regular log files (note csv use headers) Question is: Does this have to happen at index time? Can you not use props/transforms in order to make a search time field extraction instead? That would be a lot cheaper (resource wise) for the indexer and faster. Also, you would not have to worry about if it works with oneshot or not. Check the docs for more info on search time field extractions ... View more

I don't think this is what you want to do, though the specific answer to how to alias a sourcetype is given later. It seems to me that you simply want to specify a sourcetype for a set of input files. Normally, you can simply specify one when you create the input, either in the Manager GUI, or with sourcetype = mysourcetype in inputs.conf, or with a sourcetype stanza based on source in props.conf. If you were using a Splunk forwarder that would be it. If not, you may have to use a TRANSFORM stanza to modify/set the sourcetype at index time, much as with host names: http://www.splunk.com/base/Documentation/latest/Admin/Overridedefaulthostassignments You can rename sourcetypes in 4.x. props.conf.spec says: rename = <string> * Renames <sourcetype> as <string> * With renaming, you can search for the sourcetype with sourcetype=<string> * To search for the original sourcetype without renaming, use the field _sourcetype therefore, for example: [myoldsourcetype] rename = mynewsourcetype ... View more