How to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself?

[Gian89]

Hello community,

on my desk, I have a pretty edgy request that is giving me quite a headache.

I would need to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself.

Here you can find a simple ad hoc example:

| makeresults 
| eval letter1="A", letter2="B", letter3="C" 
| eval variabile="NewSourcetype"
| eval _raw=_time + ": " + _raw
| collect index=garbage sourcetype=variabile

Problem is that the event is stored under  sourcetype=variabile instead of sourcetype=NewSourcetype.
Any idea how to manage such a situation?

Thanks in advance for your kind support.

[mbjerkeland_spl]

In case @PickleRick 's suggestion wasn't clear, you can do this:

| makeresults count=5
| eval n=(random() % 10)
| eval sourcetype="something" . n
| fields - n
| collect index=your_summary_index output_format=hec

 It will respect the sourcetype set, in this case a value between something0 to something9

[PickleRick]

OK, if you read the docs for the collect command, you come across this passage:

[...]

output_format

Syntax: output_format=[raw | hec]

Description: Specifies the output format for the summary indexing. If set to raw, uses the traditional non-structured log style summary indexing stash output format.If set to hec, it generates HTTP Event Collector (HEC) JSON formatted output:

• All fields are automatically indexed when the stash file is indexed.
• The file that is written to the var/spool/splunk path ends in .stash_hec instead of .stash.
• Allows the source, sourcetype, and host from the original data to be used directly in the summary index. Does not re-map these fields to the extract_host/extracted_sourcetype/... path.
• The index and splunk_server fields in the original data are ignored.
• You cannot use the addtime, host, marker, source, or the sourcetype options when output_format=hec.

[...]

So you should format the event before calling the collect, setting up proper host/source/sourcetype values (which in this case can be evaluated dynamically as any other field). And then just collect with output_format=hec.

[Gian89 · Accepted solution]

Not the best approach but it is working:

| makeresults
| eval letter1="A", letter2="B", letter3="C"
| append
[| makeresults
| eval letter1="D", letter2="E", letter3="F"]
| eval _raw=_time + ": " + _raw
| appendpipe
[| where letter1="A"
| collect index="trash" sourcetype=testA
| where false() ]
| appendpipe
[| where letter1="D"
| collect index="trash" sourcetype=testD
| where false() 

Originally posted in this thread:
https://community.splunk.com/t5/Knowledge-Management/collect-index-quot-based-on-values-quot/m-p/473...