topic Re: Why am I not able to get any dynamic content using the collect command with the marker option? in Knowledge Management

Why am I not able to get any dynamic content using the collect command with the marker option?

bjoernjensen — Thu, 26 Mar 2015 23:26:10 GMT

Hey,

I am trying to use collect together with the marker-Option. Unfortunately I am not able to get any dynamic content for the marker string:

index=_internal file=* | head 10 | table _time file | collect  index=test_temp marker=file

yields into _raw entries like this:

03/26/2015 23:59:27 +0100, info_search_time=1427410768.113, file=shelper, file

What I would like of course is the content of the field file and not the string file. I have already tried:
... marker='file'
... marker=\'file\'
... marker=\\'file\\'
... marker=\\\'file\\\'

But the marker is always set to the string.

I thought of using map-command. But this is very ugly since map starts a search for each event going into map (maxsearches could be adjusted, but .... naah)

I also tried to create a macro mycollect(2):

collect index=$index$ marker=$marker$

But the same result for either

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,file)`

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,'file')`

So, has anyone an idea?
Thanks in advance!

Re: Why am I not able to get any dynamic content using the collect command with the marker option?

packman — Fri, 27 Mar 2015 01:54:57 GMT

I'm having the same issue, I get no error in the search but the data is never indexed if I use a field value instead of just text.

Re: Why am I not able to get any dynamic content using the collect command with the marker option?

bjoernjensen — Mon, 28 Sep 2020 19:18:20 GMT

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

Re: Why am I not able to get any dynamic content using the collect command with the marker option?

masonmorales — Fri, 27 Mar 2015 23:33:02 GMT

Right, defining it as a field using an eval is the way to go for your use case. As you saw in the docs, marker has to be a string, which is useful for separating out different sets of data in the summary index.

For example:

| collect index="mysummaryindex" marker="report=top_sales"

Allows me to search on that data using:

index=mysummaryindex report=top_sales