Knowledge Management

"Collect" command doesnt work with "All time (Real-Time)"

DavidGirsvaldas
Explorer

Hi,
I have a use case where I need to check for incomming events with measurements, combine and modify them and save as a new event. This is different from a simple summary as I might need to apply math to its values. What Im trying to use now is "Collect" command:

index=main (host="host1" AND MeasurementChannel=1) OR (host="host2" AND MeasurementChannel=2) | stats latest(MeanValue) as sumMean latest(Timestamp) as latestTImestamps latest(_time) as _time by MeasurementChannel| stats sum(sumMean) as MeanValue latest(latestTImestamps) as Timestamp latest(_time) as _time | eval Alias="myCustomChannel" | collect index=main host=host1

This search works fine when executed as non-real time, but when I set time interval to "All Time(Real-time)" nothings gets collected. in documentation for Collect command (http://docs.splunk.com/Documentation/Splunk/7.1.2/SearchReference/Collect) it says

The collect command also works with real-time searches that have a time range of All time.

Tags (2)
0 Karma

inventsekar
Ultra Champion

collect command Description....
Adds the results of a search to a summary index that you specify. You must create the summary index before you invoke the collect command.

index=main ....................| collect index=main host=host1
here you are collecting the events from main index and sending it again to main index.

probably you should re-write your query to... (beforehand, you have to create this mainCollectSummary index )
index=main ....................| collect index=mainCollectSummary

0 Karma

DavidGirsvaldas
Explorer

thank you for reply.
I also tried collecting events to "summary" index and it behaved in a same way. It worked with non-real time queries. Also my use case actually requires events to be saved in a same index.
"You must create the summary index before you invoke the collect command."- the way I read it, is that Splunk says search will not create a new index automatically by running search and so index should be created prior.
"Adds the results of a search to a summary index that you specify"- as far as I know Real-Time searches never finishes so therefore they do not produce results. So I wouldnt expect it to work, but the Collect command documentation clearly states that command works with All-time(real time) which confuses me

0 Karma

inventsekar
Ultra Champion

maybe, update the authorize.conf file that gives a way to grant/remove this collect command from a user...

[capability::run_collect]
* Lets a user run the collect command.

http://docs.splunk.com/Documentation/Splunk/latest/Admin/Authorizeconf

0 Karma

DavidGirsvaldas
Explorer

The documentation page points to Splunk version 7.1.2 meanwhile Im using a bit older 7.0.2. This capability doesnt appear as option in my version and gets ignored if set in config files. However I doubt this is an issue since Im able to successfully use Collect command as long as it is not real time.

0 Karma

inventsekar
Ultra Champion

ok, are you able to run other real time searches?!?!

[capability::rtsearch]
* Lets a user run realtime searches.

0 Karma

DavidGirsvaldas
Explorer

yes, they all work as expected. Im currently running it all using Admin role. rtsearch is enabled.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...