Knowledge Management

Is collect command working correctly ?

Path Finder
index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | table *  | collect index=inx_copy_data

Straightforward task - to select events, filter them, add some new fields, copy the results to another index.
But in inx_copy_data not all data is copied - I can't find the new fields (new_field, new_field_id) nor the old ones (old_field, old_field_id).
Do I miss something?

Tags (1)
0 Karma
1 Solution

Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

View solution in original post

Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

View solution in original post

Path Finder

so the big problem is due to _raw field, correct? Is any way to avoid this? Or I should omit _raw fields from collect command ?

0 Karma

Super Champion

yeah, sure you can omit it to avoid having the same info multiple times. It's actually best to replace that * with the exact fields you need

0 Karma

Path Finder

thank you!!

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!