it was a simple regex. regex was not the issue. as the same regex worked with other sourcetypes, but not stash. These commands worked for me. I am still figuring it out how to retain the original host, source, sourcetype | extract auto=t I wanted to merge data from one index into another for a use case. My understanding is collect command does the work. It is also documented here. https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Collect Copying events to a different index
You can use the collect command to copy search results to another index. Construct a search that returns the data you want to copy, and pipe the results to the collect command. For example:
index=foo | ... | collect index=bar
This search writes the results into the bar index. The sourcetype is changed to stash.
You can specify a sourcetype with the collect command. However, specifying a sourcetype counts against your license, as if you indexed the data again. We can probably keep the original sourcetypes and host, and source values too. But license usage will become an issue since the amount of data is in TBs. I think I saw a thread about how you can append source, sourcetype, host to _raw. I am still looking for it. | eval _raw=_raw.orig_host=$host..orig_source=$source Once it is done, I can use the destination index like index=destinationindex|eval host=orig_host ...|extract auto=t and I will have a backup index data without consuming more license usage. It also says: data is stored under: The file that is written to the var/spool/splunk path ends in .stash_hec instead of .stash. while the saved results from normal searches are stored under var/run/splunk/dispatch. So, splunk should not replicate the artifacts from spool/splunk to other search head cluster members. I can test it out though. So, that should rule out the results getting replicated across search peers and having duplicate events? If I do schedule a search with collect command, for this use case, should it be run in fast mode, or verbose mode? or it doesn't matter, most likely scheduled searches always run in fast mode? and where will the results from scheduled search for a collect command get stored, under dispatch or spool? What fields does collect command collect from source index? thanks
... View more