topic Re: Alternate command for collect in Splunk Cloud Platform

Is there an alternate command for collect?

knanaiah001 — Mon, 14 Aug 2023 17:03:18 GMT

Hi ,

Do we have any command in splunk which does similar functionality like "Collect " command.
Can someone suggest on this?

Re: Alternate command for collect

isoutamo — Thu, 29 Oct 2020 16:46:38 GMT

Can you describe your issue where you are needing this?

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 08:41:48 GMT

We cannot use "collect" command
Please see my example search:

index="main"
|table host index tag
|collect index="custom_index"

It didn't work. No results were collected in "custom_index"

Re: Alternate command for collect

ITWhisperer — Sun, 13 Aug 2023 09:18:51 GMT

Do you have permissions to read the index? Do you have permissions to run the collect command? What search are you using to check whether events have been written?

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 09:49:38 GMT

We have a new index named "threathunting" and having permission to collect the results. We use splunk admin account.
How do we check that we have permission to run collect command?

Re: Alternate command for collect

PickleRick — Sun, 13 Aug 2023 09:57:57 GMT

Does your original search return any results at all? (Without the collect part)

It doesn't seem right or at least doesn't seem like good practice to have events in the main index.

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 10:02:47 GMT

Yes, the original command without |collect is working fine and gets results.

index="main"
|table host index tag

Till this command, it shows results as table.

Re: Alternate command for collect

ITWhisperer — Sun, 13 Aug 2023 10:10:24 GMT

What search are you using to get the results from the summary index?

What does the job inspector say about the search you use to collect the events? (It should say something about successfully writing the results to stash.

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 10:22:48 GMT

Yes, the original command without |collect is working fine and gets results.

index="main"
|table host index tag
|head 10

Till this command, it shows the results with table. When I put the collect command, the search run has no error, still showing the head 10 results.

|collect index="threathunting"

But, the results are not collected in the "threathunting" index.

I already created a empty "threathunting" index, get permission and accessible index.

So, the head 10 results should be collected in "threathunting" index. Yet, it can't.

Re: Alternate command for collect

ITWhisperer — Sun, 13 Aug 2023 10:31:11 GMT

After you run the search with the collect command, even if there is no error, there should still be a message in the job. What messages do you get when you click on the job button

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 10:36:58 GMT

Successfully wrote file to '/opt/splunk/var/spool/splunk/b90a7184a4568807_events.stash_new'.

Re: Alternate command for collect

ITWhisperer — Sun, 13 Aug 2023 10:46:08 GMT

If you go to settings -> indexes, what does it say about your index?

Re: Alternate command for collect

bimatomsoc — Sun, 13 Aug 2023 11:12:43 GMT

Setting>Indexes>threathunting

It's enabled, deployed in the Global sharing permissions app. It also has home path for db.

Re: Alternate command for collect

ITWhisperer — Sun, 13 Aug 2023 12:00:52 GMT

Looks like no events in the index - do you have any errors in the splunkd.log?

Re: Alternate command for collect

bowesmana — Mon, 14 Aug 2023 02:59:22 GMT

Is this a clustered environment? Do you have access to the file system and is the stash file still there?

Check the _internal index for any evidence of that stash file

Re: Alternate command for collect

bimatomsoc — Mon, 14 Aug 2023 10:38:46 GMT

Yes, it was found in _internal index.

Also, this is a cluster environment. We have access to file system but there is no *_events.stash_new file under the /opt/splunk/var/spool/splunk/ directory.

Re: Alternate command for collect

bimatomsoc — Mon, 14 Aug 2023 10:40:15 GMT

Yes, "threathunting" index is empty and created to collect search results.
No error in splunkd.log

Re: Alternate command for collect

bowesmana — Tue, 15 Aug 2023 02:00:42 GMT

If this is clustered and your 'index' is showing no events, then I suspect there may be something going on with one index being on the search head and one on the indexer and you have an issue with data being ingested to one index rather than the other.

That sounds like an admin issue and I am not sure how to verify that.

Did you create the threathunting index through the UI on the search head - if so, that means the index is on the SH, not the indexers. Creating indexes on the indexers is not done through the UI, it needs to be set up in the conf files for the indexers.

If this is the case, I suspect the data is being "ingested" to a threathunting index that does not exist on the indexer, as you created it on the search head.