Splunk Search

Splunk Extraction

ppatkar
Path Finder

I have Splunk events like below & would like to extract the reason for failure.

Event 1 :
FILE_READER[1]: TT19472 Fatal data error processing file '/default/folder/ingest/amr_ca_sf_items_658721_US.out'.
Field length overflow(s) in record 2355, field 17, 'COUNT_DESC'. Expected 300 bytes, field contained 307 bytes.
FILE_READER[1]: TT19015 TPT Exit code set to 12.

Event 2 :
$FILE_READER<1>: DataConnector Producer operator Instances: 1
$FILE_READER<1>: ECI operator ID: '$FILE_READER-18808'
$FILE_READER<1>: Operator instance 1 processing file '/default/folder/ingest/amr_ca_sf_items_658721_US.out'.
$FILE_READER<1>: TT19472 Fatal data error processing file '/default/folder/ingest/amr_ca_sf_items_658721_US.out'.
Field length overflow(s) in record 1, field 1, '"ORDER"'. Expected 20 bytes, field contained 841 bytes.
$FILE_READER<1>: TT19015 TPT Exit code set to 12.

Event 3 :
FILE_READER<1>: TT19434 pmAttach failed. General failure (34): '!ERROR! dlopen failed: /default/folder/installations/lib/axm.so: cannot open shared object file: No such file or directory'
FILE_READER<1>: TT19302 Fatal error loading access module.
FILE_READER<1>: TT19015 TPT Exit code set to 12.

Event 4 :
FILE_READER<1>: TT19134 !ERROR! Fatal data error processing file '/default/folder/ingest/rpv0410_12123_1.out.gz'. Delimited Data Parsing error: Too many columns in row 246.
FILE_READER<1>: TT19015 TPT Exit code set to 12.

Event 5 :
FILE_WRITER<1>: TT19434 pmWrite failed. General failure (34): 'pmunxWBuf: fwrite byte count error (No space left on device)'
FILE_WRITER<1>: TT19306 Fatal error writing data.
FILE_WRITER<1>: TT19015 TPT Exit code set to 12.

Reason for failure should look like below :
1: Field length overflow(s) in record 2355, field 17, 'COUNT_DESC'. Expected 300 bytes, field contained 307 bytes.
2 : Field length overflow(s) in record 1, field 1, '"ORDER"'. Expected 20 bytes, field contained 841 bytes.
3 : Fatal error loading access module or '!ERROR! dlopen failed: /default/folder/installations/lib/axm.so: cannot open shared object file: No such file or directory'
4 : Parsing error: Too many columns in row 246.
5 : Fatal error writing data or General failure (34): 'WBuf: fwrite byte count error (No space left on device)'

If someone can guide on a way to extract this , it will be very helpful .

Thanks.

Tags (1)
0 Karma
1 Solution

dmarling
Builder

This regex should grab all of your use cases when I tested it in regex101:

| rex "(Fatal data error processing file '[^\']+'\.\n?\s?|General failure \(\d+\): )(?<Exception>[^\n\$]+)"

Here's the link to the regex101 test example: https://regex101.com/r/0V46z8/1

alt text

It captures a little more than you want on your last three examples which can be trimmed off by adding this after your extraction:

| rex mode=sed field=Exception "s/FILE_(READER|WRITER)[^\e]+//g"

There's probably a little better way to prevent that extra FILE_READER/WRITER data at the end. Let me know if this works for you.

If this comment/answer was helpful, please up vote it. Thank you.

View solution in original post

0 Karma

dmarling
Builder

This regex should grab all of your use cases when I tested it in regex101:

| rex "(Fatal data error processing file '[^\']+'\.\n?\s?|General failure \(\d+\): )(?<Exception>[^\n\$]+)"

Here's the link to the regex101 test example: https://regex101.com/r/0V46z8/1

alt text

It captures a little more than you want on your last three examples which can be trimmed off by adding this after your extraction:

| rex mode=sed field=Exception "s/FILE_(READER|WRITER)[^\e]+//g"

There's probably a little better way to prevent that extra FILE_READER/WRITER data at the end. Let me know if this works for you.

If this comment/answer was helpful, please up vote it. Thank you.
0 Karma

ppatkar
Path Finder

Thank you @dmarling

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...