Knowledge Management

Why am I not able to get any dynamic content using the collect command with the marker option?

bjoernjensen
Contributor

Hey,

I am trying to use collect together with the marker-Option. Unfortunately I am not able to get any dynamic content for the marker string:

index=_internal file=* | head 10 | table _time file | collect  index=test_temp marker=file

yields into _raw entries like this:

03/26/2015 23:59:27 +0100, info_search_time=1427410768.113, file=shelper, file

What I would like of course is the content of the field file and not the string file. I have already tried:
... marker='file'
... marker=\'file\'
... marker=\\'file\\'
... marker=\\\'file\\\'

But the marker is always set to the string.

I thought of using map-command. But this is very ugly since map starts a search for each event going into map (maxsearches could be adjusted, but .... naah)

I also tried to create a macro mycollect(2):

collect index=$index$ marker=$marker$

But the same result for either

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,file)`

or

index=_internal file=* | head 10 | table _time file | `mycollect(temp_test,'file')`

So, has anyone an idea?
Thanks in advance!

1 Solution

bjoernjensen
Contributor

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

View solution in original post

bjoernjensen
Contributor

I eventually made it work. I read the docs and there it states:

marker
Syntax: marker=<string>
Description: A string, usually of key-value pairs, to append to each event written out. Optional, default is empty.

Since this is not working with dynamic content (e.g. field content), I will either table my dynamic marker field before collect:
index=_internal file=* | head 10 | eval myMarker="some usefull stuff" | table _time file myMarker | collect index=test_temp

or append my marker string to to the content field _raw beginning with , :
index=_internal | head 10 | eval _raw=_raw.", myKey = isSearchable, andWill = beExtracted" | collect index=test_temp

Hope that helps!

masonmorales
Influencer

Right, defining it as a field using an eval is the way to go for your use case. As you saw in the docs, marker has to be a string, which is useful for separating out different sets of data in the summary index.

For example:

| collect index="mysummaryindex" marker="report=top_sales"

Allows me to search on that data using:

index=mysummaryindex report=top_sales

packman
Engager

I'm having the same issue, I get no error in the search but the data is never indexed if I use a field value instead of just text.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...