The collect command has a marker option which can be "A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.". However, I can't find any good examples of where markers are used and parseable. I've tried comma and space delimited key value pairs, but I can't get those parsed in the index search results. As an example,
is an attempt to collect hourly stats on VPN connects and disconnects by country. I've not been able to find an incantation that would let me use things like search index=mysummary summary_type=vpn to pull out VPN-related summary index stats. I see the key-value pairs in the result, but they're not parsed as fields. What am I missing here in terms of how to construct the marker argument to collect?
After a moderate amount of experimentation, I found what works, which is that there has to be a comma and a space between the key=value pairs. And if the value contains spaces or commas, it needs to be escape quoted. And the stream should have an | addinfo included for the manual population of the index (backfill):
... | stats count by _time,vpn_action,country_source | addinfo | collect index=mysummary marker="summary_type=vpn, summary_span=3600, summary_method=bucket, search_name=\"vpn starts and stops\""
Using a search of this form gives me fields I can use when I do a search (e.g. search index=mysummary). The | addinfo ensures that the search results contain fields that specify when the search was run to populate these particular index values. I think that gets added automatically for scheduled searches. But if you're doing a manual search to backfill the index, throw the | addinfo into the stream before the collect.
In the category of "error between keyboard and monitor", I thought I'd tried this before posting a question, but I hadn't (quite). The space after the comma is necessary. I do wonder why space-separated doesn't work, since space separated key=value pairs are automatically parsed at search time for _raw fields otherwise.