We recently upgraded to from 7.1.2 to 8.0.3 on on-prem Splunk Enterprise. A previously working saved search is no longer returning the correct results.
| transaction session_id maxspan=30s
Looking into it looks like the transaction
command is no longer closing connections when the maxspan
(30s) value is hit. This leaves all transactions open and then the search ends when it hits the default of 5000.
I need to create transactions out of 650000 entries (two or three lines each), so needless to say this search no longer functions. I can confirm this behavior, by:
| stats count by closed_txn
shows all the transactions returned as closed_txn=0
maxopentxn=5500
to the transaction
command causes the number of returned results to go from 5000 to 5500maxevents=2
only closes some of the eventsclosed_txn
, eventcount
, count
0 1 1041
0 2 4458
1 2 1654
Transactions are supposed to close when:
The 'closed_txn
' field is set to '1
' if one of the following conditions is met: maxevents
, maxpause
, maxspan
, startswith
.
https://docs.splunk.com/Documentation/Splunk/8.0.3/SearchReference/Transaction -> Memory control options -> keepevicted
@andrewcg which are the fields you want to stitch together after transaction command?
You should ideally try stats
for your correlation instead of transaction
. There are several examples of stats for transaction on Splunk Answers.
<yourCurrentSearch>
| fields _time, session_id, field1, field2, field3
| eval Time=_time
| bin _time span=30s
| stats count as eventcount list(*) as list_* by _time session_id
| eval maxTime=mvindex(list_Time,0),minTime=mvindex(list_Time,eventcount-1), duration=maxTime-minTime
| fields - minTime maxTime
Following is a runanywhere example based on Splunk's _internal index just to generate some data and correlate using component
(similar to session_id
)
index=_internal sourcetype=splunkd
| fields _time, component, log_level, date_hour, date_minute, date_second
| eval Time=_time
| bin _time span=30s
| stats count as eventcount list(*) as list_* by component _time
| eval maxTime=mvindex(list_Time,0),minTime=mvindex(list_Time,eventcount-1), duration=maxTime-minTime
| fields - minTime maxTime
Please try out and confirm!
@andrewcg which are the fields you want to stitch together after transaction command?
You should ideally try stats
for your correlation instead of transaction
. There are several examples of stats for transaction on Splunk Answers.
<yourCurrentSearch>
| fields _time, session_id, field1, field2, field3
| eval Time=_time
| bin _time span=30s
| stats count as eventcount list(*) as list_* by _time session_id
| eval maxTime=mvindex(list_Time,0),minTime=mvindex(list_Time,eventcount-1), duration=maxTime-minTime
| fields - minTime maxTime
Following is a runanywhere example based on Splunk's _internal index just to generate some data and correlate using component
(similar to session_id
)
index=_internal sourcetype=splunkd
| fields _time, component, log_level, date_hour, date_minute, date_second
| eval Time=_time
| bin _time span=30s
| stats count as eventcount list(*) as list_* by component _time
| eval maxTime=mvindex(list_Time,0),minTime=mvindex(list_Time,eventcount-1), duration=maxTime-minTime
| fields - minTime maxTime
Please try out and confirm!
I totally forgot to follow up on this issue, thank you for commenting on this issue. Using `stats` was the recommendation from the Splunk support. Not only does using `stats` result in all the entries being reflected in the results, it is so much faster.
Worked perfectly for me. I could not picture in my mind how the transaction command could be duplicated with a stats command based on the Splunk documentation, your example illustrated it for me. I now have a faster and more importantly functional search. Thank you.
I was able to drop the _time bin part as the session_id is unique enough to not be duplicated in a day and I was only using the maxspan to close open transactions. My search went from:
transaction session_id maxspan=30s
to
| stats count as eventcount list(mac) AS mac, list(hostname) AS hostname, list(nas_ip) AS nas_ip, list(nas_port) AS nas_port, list(nas_id) AS nas_id, list(attr_value) AS attr_value, list(profiles) AS profiles by session_id
That allowed me to keep all the rest of the search on either side the same.
I prefer transaction
when for scenarios where I need to drilldown with a particular session_id
or for more complex scenarios where things like maxpause
and other conditions are in place. A good guide to Event Grouping and Correlation is this Splunk Documentation: https://docs.splunk.com/Documentation/Splunk/latest/Search/Abouteventcorrelation
However, there is more to correlation and commands like union, multisearch also needs to be considered under specific circumstances.