Splunk Search

How to edit my regular expression to extract a field from my sample data?

Path Finder

Hey Fellow Splunkers

I'm looking to possibly create a regular expression that can be used to extract a field. The data associated with the field that I'm attempting to extract is a complex string with numerical values including quotes example:

EXAMPLE DATA:
Aug 10 10:10:40 HOST SUPERTROOPERS: 10-08-2016 10:55:15 WARNING 555 ERRORS "THE ERROR IS 1 MORE THAN EXPECTED"

WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
WARNING  210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
WARNING  122 Errors " SOME ADDITIONAL 1 TEXT"

The Regular Expression I generated:

rex field=ERROR ^(?:[^ \n]* ){5}(?P<ERROR>\s+"+*+")

the last part of the regular expression s+"*") is complex, I'm wandering how can i say all words, spaces, and numbers within " " to be associated with my field Errors.

I think this is saying new field name ERROR new line, non capture, except new line, and any value, 5 characters for in ERROR space than then this part would be to include everything within quotes?

My Goal is to simply create the field Error = quotes, words, spaces, numbers, and some more quotes.?

0 Karma

Path Finder

I think this Regular Expression may work, but I cannot figure out the error.

rex field=WARNING "*\s+Anomalies\s+(?+[^"]+)\\"

Sample Data:

 WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
 WARNING  210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
 WARNING  122 Errors " SOME ADDITIONAL 1 TEXT"

Referencing WARNING field.

rex field=WARNING

Indicating ALL variations and one to some spaces

"*\s+

data along this path after Anomalies

"+Anomalies"

One to Many spaces "AND"

\s+

Call this field warning

(?

AND include " everything inside AND

+[^"]+)

Stop when you see "

\\"

I'm thinking the concept may be accurate, but the syntax for sure is incorrect. I'm not sure what is not working.

0 Karma

Path Finder

Hey Fellow Splunkers

Still no luck, I wasn't able to utilize any regular expression to solve my issue. So I decided to break it up.

Sample Data:

WARNING 344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
WARNING 210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
WARNING 122 Errors " SOME ADDITIONAL 1 TEXT"

Warning 344
Warning 210
Warning 122

Warning = Field

I've decided to create a second Field as Errors. However, I'm now having issues correlating Errors to the complex string with quotes.

rex "Errors\s?\((?[^\)]+)" I believe this is emphasis on parenthesis not quotes.

0 Karma

Motivator

You can also use Splunk's erex command. If you give it the field and some sample data it will write the rex for you.

http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Erex

0 Karma

Motivator

whups,

Couple of typo's, use this one

index=blah sourcetype=blah blah | erex {your data field name goes here} examples="344 Errors THE ERROR IS 1 MORE THAN EXPECTED, 210 Errors THE ERROR IS 5 LESS THAN EXPECTED"
0 Karma

Motivator

Try this

index=blah sourctype=blah blah | erex {your data field name goes here}  examples="344 Errors "THE ERROR IS 1 MORE THAN EXPECTED, THE ERROR IS 5 LESS THAN EXPECTED"

Once it is done running click the JOB drop down, you should see a generated rex command that should work for your needs.

0 Karma

SplunkTrust
SplunkTrust

This will extract the first word as the error level and the remaining text in the 'message' field.

(?<level>\w+)\s+(?<message>.*)
---
If this reply helps you, an upvote would be appreciated.
0 Karma

SplunkTrust
SplunkTrust

Here's an update based on your latest comment. It will extract "Warning", the number following it, and the text in quotes.

... | rex "(?<level>\w+)\s+(?<errorCount>\d+)\s+Errors\s+\"(?<message>.*)\"" | ...
---
If this reply helps you, an upvote would be appreciated.
0 Karma

SplunkTrust
SplunkTrust

So you want to extract the entire line of text which can have numbers, words, and quotes right?

If so then we will need more sample data to build a good regular expression.. We ideally need to know if there is anything before or after this text and if its at the beginning or end of the line.. Also will it be a pattern? will it have WARNING|INFO|ERROR or will it always be warning? Will it also have a digit(s) after the warning message and will it always have text in the quotes or will it not have quotes sometimes?

Path Finder

Sorry, Skoelpin

I'm sorry, I forgot to provide some sample data

Aug 10 09:53:40 SomeHost SuperTrooper: 2-04-2015 1:4:15 WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"

I created a simple field extraction for WARNING. Ideally I had wanted to use WARNING for the field and 344 Errors "THE ERROR IS 1 MORE THAN EXPECTED" as the value, but I wasn't able to figure that out.

So I would like to extract the field ERRORS and have multiple complex strings within quotes.

0 Karma

SplunkTrust
SplunkTrust

What fields do you want to extract from that example?

---
If this reply helps you, an upvote would be appreciated.