topic Re: Why is collect command not working? in Knowledge Management

Why is collect command not working?

rschmelzle_noda — Wed, 30 Sep 2020 03:01:43 GMT

I have an instance of Splunk Enterprise installed where my search head and indexer are running on the same server. I installed and configured the Splunk Forwarder for Windows on a Windows server with a syntax error causing events to be sent to an incorrect index. I tried following the support articles for using the "collect" command to copy events from one index to another but that does not seem to be working. Additionally I double checked the syntax of the collect command directly from the Splunk documentation for the collect command and it appears to be correct. However, when I run the following search and collect my data is not copied to the destination index:

host="hostname" sourcetype="source_type" index="source_index" | collect index="destination_index" sourcetype="source_type" host="hostname"

For my particular use case, my host and sourcetype should be the same for the data in the source and destination index. I only with to copy the events to the new destination index where after I will delete them from the original index.

Is there anything I am missing here? Thanks and please let me know if anyone has any insight!

Re: Why is collect command not working?

uagrawal_splunk — Wed, 30 Sep 2020 03:02:02 GMT

Have you created your destination_index in the indexer? I tried the same query of yours and it works for me, the events are copied to my new destination_index.
If the destination_index is not available then you will get below message :
Received event for unconfigured/disabled/deleted index='test' with source="source" host="my_host" sourcetype="my_sourcetype". So far received events from 1 missing index(es).

Re: Why is collect command not working?

rschmelzle_noda — Wed, 20 Nov 2019 16:58:58 GMT

Thanks for the reply. When I got to Settings --> Data --> Indexes my index is present. Any other thoughts or suggestions?

Re: Why is collect command not working?

uagrawal_splunk — Wed, 20 Nov 2019 17:08:04 GMT

Are you getting any error message or anything? What happens after you hit the above command.

Re: Why is collect command not working?

rschmelzle_noda — Wed, 20 Nov 2019 17:10:16 GMT

When I run it nothing happens at all. I run it in the search box from the web UI and nothing at all happens. Upon searching for the events that should be copied in the destination index I do not see them. I do still see the events in the source index untouched.

Maybe I ran it with a small syntax error?

Re: Why is collect command not working?

uagrawal_splunk — Wed, 30 Sep 2020 03:02:30 GMT

I don't think there is a syntax error.
Because this query works for me. I am seeing events on my new_index. I used below query:
host="host_name" sourcetype="My_sourcetype" index="test" | collect index="new_index" host="host_name" sourcetype="My_sourcetype"

Re: Why is collect command not working?

rschmelzle_noda — Wed, 20 Nov 2019 20:11:48 GMT

Thanks, I may have typed it in wrong the first time, but I will try it again and see if it works. I appreciate your feedback!

Re: Why is collect command not working?

uagrawal_splunk — Fri, 22 Nov 2019 15:36:06 GMT

Are you able to copy the data in the destination index?

Re: Why is collect command not working?

rschmelzle_noda — Fri, 22 Nov 2019 15:51:44 GMT

Yes, I had incorrectly formatted my original search and collect. After looking back through my documentation there was a small syntax error in the initial execution of my command causing the issue.

Thanks for your help and sorry for the silly mistake!

Re: Why is collect command not working?

uagrawal_splunk — Fri, 22 Nov 2019 16:01:05 GMT

No problem, I am writing the collect command in answers for others.

Re: Why is collect command not working?

uagrawal_splunk — Wed, 30 Sep 2020 03:03:48 GMT

The collect command stated in the question is correct and it will indexed the data in new index:
host="host_name" sourcetype="My_sourcetype" index="test" | collect index="new_index" host="host_name" sourcetype="My_sourcetype"

Re: Why is collect command not working?

rschmelzle_noda — Fri, 22 Nov 2019 16:02:28 GMT

Thanks again!

Re: Why is collect command not working?

uagrawal_splunk — Fri, 22 Nov 2019 17:11:19 GMT

No problem. Glad to help.