Is collect command working correctly ?

[dreadangel]

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | table *  | collect index=inx_copy_data

Straightforward task - to select events, filter them, add some new fields, copy the results to another index.
But in inx_copy_data not all data is copied - I can't find the new fields (new_field, new_field_id) nor the old ones (old_field, old_field_id).
Do I miss something?

[DavidHourani · Accepted solution]

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

[dreadangel]

so the big problem is due to _raw field, correct? Is any way to avoid this? Or I should omit _raw fields from collect command ?

[DavidHourani]

yeah, sure you can omit it to avoid having the same info multiple times. It's actually best to replace that * with the exact fields you need

[dreadangel]

thank you!!