Assistance with Complex Event Correlation Using transaction in SPL

[oussama1]

Hello,

I'm currently working on a Splunk query designed to identify and correlate specific error events leading up to system reboots or similar critical events within our logs. My goal is to track sequences where any of several error signatures occurs shortly before a system reboot or a related event, such as a kernel panic or cold restart. These error signatures include "EDAC UE errors," "Uncorrected errors," and "Uncorrected (Non-Fatal) errors," among others.

Here's the SPL query I've been refining:

index IN (xxxx) sourcetype IN ("xxxx")
("EDAC* UE*" OR "* Uncorrected error *" OR "* Uncorrected (Non-Fatal) error *" OR "reboot" OR "*Kernel panic* UE *" OR "* UE ColdRestart*")
| append [| eval search=if("true" ="true", "index IN (xxx) sourcetype IN (xxxxxx) shelf IN (*) card IN (*)", "*")]
| transaction source keeporphans=true keepevicted=true startswith="*EDAC* UE*" OR "* Uncorrected error *" OR "* Uncorrected (Non-Fatal) error *" endswith="reboot" OR "*Kernel panic* UE *" OR "* UE ColdRestart*" maxspan=300s
| search closed_txn = 1
| sort 0_time
| search message!="*reboot*"
| table tj_timestamp, system, ne, message

My primary question revolves around the use of the `transaction` command, specifically the `startswith` and `endswith` parameters. I aim to use multiple conditions (error signatures) to start a transaction and multiple conditions (types of reboots) to end a transaction. Does the `transaction` command support using logical operators such as OR and AND within `startswith` and `endswith` parameters to achieve this? If not, could you advise on how best to structure my query to accommodate these multiple conditions for initiating and concluding transactions?

I'm looking to ensure that my query can capture any of the specified start conditions leading to any of the specified end conditions within a reasonable time frame (maxspan=300s), but I've encountered difficulties getting the expected results. Your expertise on the best practices for structuring such queries or any insights on what I might be doing wrong would be greatly appreciated.

Thank you for your time and assistance.

[PickleRick]

Adding to what's already been said, your search terms are very ineffective.

1. Searching for "* something *" makes no sense since space is a major segmenter and you can just search for "something".

2. Searching for terms wildcarded at the beginning (like "*something") is very ineffective since Splunk cannot use its internal index structures and has to dig through the raw data of all the events to find matches to your search terms.

[bowesmana]

Also note that transaction will be slow and can, if you have a lot of data, just give you wrong results, as it can be affected by memory usage and just discard results.

transaction can often be avoided through stats, although it does take some extra steps to get your data to a state where you can use stats, but it's not obvious how you would do that here.

[bowesmana]

I am not quite sure what you're trying to do with the append command - it doesn't look valid to me, but it doesn't appear to make sense anyway, as you can just add the additional index search to your existing primary search with an OR between the two conditions.

[yuanliu]

The documentation on transaction answers most of these questions.  One thing to note is if you use quotation marks with any of those options, you will need to escape them.  For example,

startswith="\"*EDAC* UE*\" OR \"* Uncorrected error *\" OR \"* Uncorrected (Non-Fatal) error *\""