How can I preserver original fields in a summary index?

[joydeep741]

When I summary index, my data's original fields are lost.
How can I preserve original fields in a Summary Index?

[woodcock]

That is how it is supposed to work. The summary part means just that. It only saves the fields that are present in your final results and even then, some, like host are overridden and moved to orig_* values like orig_host. If you need the fields then you must preserver then through to the end of your search results.

[wrighke]

One option is to prepend the original fields before sending to the summary index...

| foreach _* *
    [| eval orig_<<MATCHSTR>>=<<FIELD>>]

(You may want to only prepend the fields you actually need, and which get overwritten, e.g. host, source, sourcetype, instead of _* *)

[ngatchasandra]

Hi joy,

This must be due to the fact that your field extractions are applied to the sourcetype given to the _raw indexed data . I think if you try to run your query in the search app using | collect index="summaryIndex", the fields are there.

To fix this, try to recreate your field extraction regex's to also be applied to a sourcetype of ''application''.

[joydeep741]

1. “Original Search Query” : Before summary indexing Query searches raw data of last one day (yesterday’s data) :

index=dotcom_odin
sourcetype="odin_ws_access"

| eval host_sourcetype= host+"_"+sourcetype
| timechart span=5m count as "Requests" by host_sourcetype limit=100

1. INDEX POPULATING SEARCH :-

index=dotcom_odin
sourcetype="odin_ws_access"
| eval host_sourcetype= host+"_"+sourcetype
| sitimechart span=5m count as "Requests" by host_sourcetype limit=100

After the summary index is complete I see all my original fields lost. I want to use them in queries. Is there a way to preserve them in summary index ?

[jbjerke_splunk]

Hi joy

The stats command above would only have two fields: Requests and host_sourcetype . These are the only fields created by that search. Which other fields are you expecting to see?

[jbjerke_splunk]

Hi joydeep41
Can you share what search you are running where you can't see the original fields? I summary search would typically end with a summarizing command like stats or timechart and not raw data. The meta data fields (source, sourcetype etc.) should get the "original_" prefix added to them.

j

[joydeep741]

1. “Original Search Query” : Before summary indexing Query searches raw data of last one day (yesterday’s data) :

index=dotcom_odin
sourcetype="odin_ws_access"

| eval host_sourcetype= host+"_"+sourcetype
| timechart span=5m count as "Requests" by host_sourcetype limit=100

1. INDEX POPULATING SEARCH :-

index=dotcom_odin
sourcetype="odin_ws_access"
| eval host_sourcetype= host+"_"+sourcetype
| sitimechart span=5m count as "Requests" by host_sourcetype limit=100

After the summary index is complete I see all my original fields lost. I want to use them in queries. Is there a way to preserve them in summary index ?