Knowledge Management

Is collect command working correctly ?

dreadangel
Path Finder
index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | table *  | collect index=inx_copy_data

Straightforward task - to select events, filter them, add some new fields, copy the results to another index.
But in inx_copy_data not all data is copied - I can't find the new fields (new_field, new_field_id) nor the old ones (old_field, old_field_id).
Do I miss something?

Tags (1)
0 Karma
1 Solution

DavidHourani
Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

View solution in original post

DavidHourani
Super Champion

Hi @dreadangel

Use stats instead of table * because using tabledoesn't transform the results and collect would still get the raw data.

something like this should do the trick :

index=* sourcetype="..."  ... | eval new_field="new_value-".old_field, new_field_id="[some new id]".old_field_id | stats values(*) as * by _time  | collect index=inx_copy_data

Reference here :
https://docs.splunk.com/Documentation/Splunk/7.2.6/SearchReference/Collect

Note:
Summary indexing counts against ur license so careful when using it.

Cheers,
David

dreadangel
Path Finder

so the big problem is due to _raw field, correct? Is any way to avoid this? Or I should omit _raw fields from collect command ?

0 Karma

DavidHourani
Super Champion

yeah, sure you can omit it to avoid having the same info multiple times. It's actually best to replace that * with the exact fields you need

0 Karma

dreadangel
Path Finder

thank you!!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...