According to this page
stuff that is sent to a summary index has its fields renamed, the sourcetyp gets renamed to stash and the orig. sourcetype should be called origsourcetype.
Using version 6.0.3 I get the sourcetype field renamed for stuff sent to the summary index but there is no origsourcetype field.
Are there any special requisites to get those fields ? Anything I need to do or are not allowed to do ?
What I've learned with summary indexing and transactions is there are some inconsistencies in the collection of the data ... It turned out I have to do the transaction in the summary index, not before because my main concern with the duration ... I am collecting VPN logs. so I ended up using a join command to gather my events, used collect to push them into the summary index and then I run the transaction command using the startswith/endswith and eval on the fields.. that solved a big part of my problem.
| join sessionid [ search agency=sales] .... this gathers all my relevant events with a sessionid field and narrows the grouping down by doing a search for the field agency=sales
I also found out that I had to create all new field extractions using the sourcetype=stash because that is what the summary index uses ... other summary indexes will also use the same sourcetype=stash too, although I think you can force a different name ... luckily I didn't need to figure that out.
Maybe this is useful, maybe not ... but thought I would try ... Oh, and remember to make sure your in smartmode instead of fast mode for field discovery
The sourcetype of a summary event will always be "stash_new".
And a field "orig_sourcetype" will be created only if such a field is necessary in your summary generation search.
<mysearch> | sistats count by sourcetype
will silently replace sourcetype by "origsourcetype" and revert it back when you retrieve the summarized result with the symmetrical search
`index=summary searchname=mysummarysearch | stats count by sourcetype`
However a summary populating search like
<mysearch> | sistats count by host
will not create the field "orig_summary" because there is no need for it in the summarization.
"fields created by the eval statements", it depends if the fields are renamed BEFORE or AFTER the summary command.
index=* | rename host AS myhost | sistats count by myhost will be summarized with the field myhost
index=* | sistats count by host | rename host AS myhost will be summarized at the "sistats" level, therefore any renaming will not be saved, and the field will be "host"
this is the same for renamed commands in the summary command.
index=* | sistats count AS mycount by host the renaming of the count field will not be saved either.
The search itself works and provides the fields that I want / need but saving the results to a summary index will neither get me the name / newname fields nor the origsourcetype field that I think should be automatically created.
index=someindex sourcetype=auditd | sort msg desc | transaction name maxspan=2s startswith=(syscall=open AND a1=c1) endswith=(syscall=unlink) keeporphans=true | where isnull(eventcount) | eval name=if(isnull(origname),name,origname) | eval newname=if(isnull(newname),"",newname)
I expected the sourcetype field that splunk automatically generates for every event to be renamed to orig_sourcetype acc. to the mentioned webpage but that does not happen.
My query indeed DOES generate some fields but with unsed names and they are dumped alltogether anyway and are therefor NOT in the summary index 😞
Does your query, result of which you're saving/sending to summary index, generates field sourcetype? During summary indexing following fields are being overwritten and if these fields are present in the summary index query result, they will get renamed to orig_fieldname.