Splunk Search

Trouble with REX command on a multi-line event

like2splunk
Explorer

Hello,
I'm running a streamstats command that prints out a series of previously-searched events. There are often more than one "ERROR" events within each group. As such, I want to rex the entire ERROR message (composed of multiple lines). Below is an example ERROR event (in BOLD).

2017-03-08 10:34:34,067 [ WARN] {Application Queue} (com.iba.tcs.beam.bds.devices.impl.gateway.rpc.ScanningControllerProxy) - ScanningController failure: NECU Transitioned to Error State NECU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83 FCU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83 RCU Error: [0x1] Threshold Violation : Timeslice: 162 Submap: 83 (X_VOLT_SEC_FB: -1.858963 V MapThresholdLow: -6.005e-02 MapThresholdHigh: 4.256e-01) SGCU Error: [0x10] _FilteringAbsolute : Timeslice: 159 Submap: 83 (MIN_CHARGE_PRIM: 6.159e-10 C AbsoluteThresholdLow: 7.119e-10 AbsoluteThresholdHigh: 7.569e-10)

As you can see, there are multiple lines for a single timestamp. I want to rex everything after the "ScanningController failure:" string. So the result would simply look like this:

NECU Transitioned to Error State NECU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83 FCU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83 RCU Error: [0x1] Threshold Violation : Timeslice: 162 Submap: 83 (X_VOLT_SEC_FB: -1.858963 V MapThresholdLow: -6.005e-02 MapThresholdHigh: 4.256e-01) SGCU Error: [0x10] _FilteringAbsolute : Timeslice: 159 Submap: 83 (MIN_CHARGE_PRIM: 6.159e-10 C AbsoluteThresholdLow: 7.119e-10 AbsoluteThresholdHigh: 7.569e-10)

How do I do this? I tried the following but it does not work: | rex "Transitioned to Error State: .?(?<_error_msg>.?)$"

It would also be nice to extract that timestamp as well and place it in a variable if someone can help me do so!

Thank you in advance!

Tags (1)
0 Karma
1 Solution

Ravan
Path Finder

Can you try

rex "^(?P<time>[^\,]*)\,.*ScanningController failure\:\s(?<error_msg>.*)$"

View solution in original post

0 Karma

alemarzu
Motivator

Hey there,

  1. This should grab all the errors per event into one single field. | rex "Transitioned\sto\sError\sState\s+(?<ALL_ERROR_LINES>[\n\r\s\S\d]+)"
  2. If you want to extract those errors individually. | rex max_match=10 "^(?<AA>[A-Z]+\sError:\s[^\n]+)"
  3. Or something more granular like field=value (ie: error_type=NECU msg="[0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83"), something like this should work.

props.conf

[your_sourcetype]
REPORT-multi_errors = multi_error_values

transforms.conf

[multi_error_values]
REGEX = ^(?<_KEY_1>[A-Z]+\sError):\s(?<_VAL_1>[^\n]+)
REPEAT_MATCH = true
CLEAN_KEYS = 1

Hope it helps.

0 Karma

Ravan
Path Finder

Can you try

rex "^(?P<time>[^\,]*)\,.*ScanningController failure\:\s(?<error_msg>.*)$"
0 Karma

like2splunk
Explorer

All I get from your rex is the following:

"NECU Transitioned to Error State" (this corresponds to the first line only. I need the remaining four lines as well. How do I grab those?

FYI, the logfile looks like this:

2017-03-08 10:34:34,067 [ WARN] {Application Queue} (com.iba.tcs.beam.bds.devices.impl.gateway.rpc.ScanningControllerProxy) - ScanningController failure: NECU Transitioned to Error State

NECU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83

FCU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83

RCU Error: [0x1] Threshold Violation : Timeslice: 162 Submap: 83 (X_VOLT_SEC_FB: -1.858963 V MapThresholdLow: -6.005e-02 MapThresholdHigh: 4.256e-01)

SGCU Error: [0x10] _FilteringAbsolute : Timeslice: 159 Submap: 83 (MIN_CHARGE_PRIM: 6.159e-10 C AbsoluteThresholdLow: 7.119e-10 AbsoluteThresholdHigh: 7.569e-10)

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try this regex string

ScanningController failure:\s(?<error_msg>.*)

The timestamp is already in a field called _time.

BTW, you shouldn't start your field names with an underscore. Such fields names are reserved by Splunk.

---
If this reply helps you, Karma would be appreciated.
0 Karma

like2splunk
Explorer

All I get from your rex is the following:

"NECU Transitioned to Error State" (this corresponds to the first line only. I need the remaining four lines as well. How do I grab those?

FYI, the logfile looks like this:

2017-03-08 10:34:34,067 [ WARN] {Application Queue} (com.iba.tcs.beam.bds.devices.impl.gateway.rpc.ScanningControllerProxy) - ScanningController failure: NECU Transitioned to Error State

NECU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83

FCU Error: [0x0] _SynchronizationSGCUTimeout : Timeslice: 163 Submap: 83

RCU Error: [0x1] Threshold Violation : Timeslice: 162 Submap: 83 (X_VOLT_SEC_FB: -1.858963 V MapThresholdLow: -6.005e-02 MapThresholdHigh: 4.256e-01)

SGCU Error: [0x10] _FilteringAbsolute : Timeslice: 159 Submap: 83 (MIN_CHARGE_PRIM: 6.159e-10 C AbsoluteThresholdLow: 7.119e-10 AbsoluteThresholdHigh: 7.569e-10)

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Try adding the single line flag:

(?s)ScanningController failure:\s(?<error_msg>.*)
---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...