Reporting

How to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself?

Gian89
Explorer

Hello community,

on my desk, I have a pretty edgy request that is giving me quite a headache.

I would need to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself.

Here you can find a simple ad hoc example:

| makeresults 
| eval letter1="A", letter2="B", letter3="C"
| eval variabile="NewSourcetype"
| eval _raw=_time + ": " + _raw
| collect index=garbage sourcetype=variabile

Problem is that the event is stored under  sourcetype=variabile instead of sourcetype=NewSourcetype.
Any idea how to manage such a situation?

Thanks in advance for your kind support.

Labels (1)
0 Karma
1 Solution

Gian89
Explorer

Not the best approach but it is working:

| makeresults
| eval letter1="A", letter2="B", letter3="C"
| append
[| makeresults
| eval letter1="D", letter2="E", letter3="F"]
| eval _raw=_time + ": " + _raw
| appendpipe
[| where letter1="A"
| collect index="trash" sourcetype=testA
| where false() ]
| appendpipe
[| where letter1="D"
| collect index="trash" sourcetype=testD
| where false()

Originally posted in this thread:
https://community.splunk.com/t5/Knowledge-Management/collect-index-quot-based-on-values-quot/m-p/473...

 

View solution in original post

0 Karma

PickleRick
Ultra Champion

OK, if you read the docs for the collect command, you come across this passage:

[...]

output_format

Syntax: output_format=[raw | hec]

Description: Specifies the output format for the summary indexing. If set to raw, uses the traditional non-structured log style summary indexing stash output format.If set to hec, it generates HTTP Event Collector (HEC) JSON formatted output:

  • All fields are automatically indexed when the stash file is indexed.
  • The file that is written to the var/spool/splunk path ends in .stash_hec instead of .stash.
  • Allows the source, sourcetype, and host from the original data to be used directly in the summary index. Does not re-map these fields to the extract_host/extracted_sourcetype/... path.
  • The index and splunk_server fields in the original data are ignored.
  • You cannot use the addtime, host, marker, source, or the sourcetype options when output_format=hec.

[...]

So you should format the event before calling the collect, setting up proper host/source/sourcetype values (which in this case can be evaluated dynamically as any other field). And then just collect with output_format=hec.

 

Gian89
Explorer

Not the best approach but it is working:

| makeresults
| eval letter1="A", letter2="B", letter3="C"
| append
[| makeresults
| eval letter1="D", letter2="E", letter3="F"]
| eval _raw=_time + ": " + _raw
| appendpipe
[| where letter1="A"
| collect index="trash" sourcetype=testA
| where false() ]
| appendpipe
[| where letter1="D"
| collect index="trash" sourcetype=testD
| where false()

Originally posted in this thread:
https://community.splunk.com/t5/Knowledge-Management/collect-index-quot-based-on-values-quot/m-p/473...

 

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...