Reporting

How to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself?

Gian89
Explorer

Hello community,

on my desk, I have a pretty edgy request that is giving me quite a headache.

I would need to collect (with | collect) the output of a search in a new sourcetype created dynamically within the search itself.

Here you can find a simple ad hoc example:

| makeresults 
| eval letter1="A", letter2="B", letter3="C"
| eval variabile="NewSourcetype"
| eval _raw=_time + ": " + _raw
| collect index=garbage sourcetype=variabile

Problem is that the event is stored under  sourcetype=variabile instead of sourcetype=NewSourcetype.
Any idea how to manage such a situation?

Thanks in advance for your kind support.

Labels (1)
0 Karma
1 Solution

Gian89
Explorer

Not the best approach but it is working:

| makeresults
| eval letter1="A", letter2="B", letter3="C"
| append
[| makeresults
| eval letter1="D", letter2="E", letter3="F"]
| eval _raw=_time + ": " + _raw
| appendpipe
[| where letter1="A"
| collect index="trash" sourcetype=testA
| where false() ]
| appendpipe
[| where letter1="D"
| collect index="trash" sourcetype=testD
| where false()

Originally posted in this thread:
https://community.splunk.com/t5/Knowledge-Management/collect-index-quot-based-on-values-quot/m-p/473...

 

View solution in original post

0 Karma

PickleRick
Ultra Champion

OK, if you read the docs for the collect command, you come across this passage:

[...]

output_format

Syntax: output_format=[raw | hec]

Description: Specifies the output format for the summary indexing. If set to raw, uses the traditional non-structured log style summary indexing stash output format.If set to hec, it generates HTTP Event Collector (HEC) JSON formatted output:

  • All fields are automatically indexed when the stash file is indexed.
  • The file that is written to the var/spool/splunk path ends in .stash_hec instead of .stash.
  • Allows the source, sourcetype, and host from the original data to be used directly in the summary index. Does not re-map these fields to the extract_host/extracted_sourcetype/... path.
  • The index and splunk_server fields in the original data are ignored.
  • You cannot use the addtime, host, marker, source, or the sourcetype options when output_format=hec.

[...]

So you should format the event before calling the collect, setting up proper host/source/sourcetype values (which in this case can be evaluated dynamically as any other field). And then just collect with output_format=hec.

 

Gian89
Explorer

Not the best approach but it is working:

| makeresults
| eval letter1="A", letter2="B", letter3="C"
| append
[| makeresults
| eval letter1="D", letter2="E", letter3="F"]
| eval _raw=_time + ": " + _raw
| appendpipe
[| where letter1="A"
| collect index="trash" sourcetype=testA
| where false() ]
| appendpipe
[| where letter1="D"
| collect index="trash" sourcetype=testD
| where false()

Originally posted in this thread:
https://community.splunk.com/t5/Knowledge-Management/collect-index-quot-based-on-values-quot/m-p/473...

 

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...