Knowledge Management

Is collect command working correctly ?

dreadangel
Path Finder
index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | table *  | collect index=inx_copy_data

Straightforward task - to select events, filter them, add some new fields, copy the results to another index.
But in inx_copy_data not all data is copied - I can't find the new fields (new_field, new_field_id) nor the old ones (old_field, old_field_id).
Do I miss something?

Tags (1)
0 Karma
1 Solution

DavidHourani
Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

View solution in original post

DavidHourani
Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

dreadangel
Path Finder

so the big problem is due to _raw field, correct? Is any way to avoid this? Or I should omit _raw fields from collect command ?

0 Karma

DavidHourani
Super Champion

yeah, sure you can omit it to avoid having the same info multiple times. It's actually best to replace that * with the exact fields you need

0 Karma

dreadangel
Path Finder

thank you!!

0 Karma
Get Updates on the Splunk Community!

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...

Index This | What goes away as soon as you talk about it?

May 2025 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this month’s ...

What's New in Splunk Observability Cloud and Splunk AppDynamics - May 2025

This month, we’re delivering several new innovations in Splunk Observability Cloud and Splunk AppDynamics ...