Splunk Search

How to edit my regular expression to extract a field from my sample data?

asarran
Path Finder

Hey Fellow Splunkers

I'm looking to possibly create a regular expression that can be used to extract a field. The data associated with the field that I'm attempting to extract is a complex string with numerical values including quotes example:

EXAMPLE DATA:
Aug 10 10:10:40 HOST SUPERTROOPERS: 10-08-2016 10:55:15 WARNING 555 ERRORS "THE ERROR IS 1 MORE THAN EXPECTED"

WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
WARNING  210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
WARNING  122 Errors " SOME ADDITIONAL 1 TEXT"

The Regular Expression I generated:

rex field=ERROR ^(?:[^ \n]* ){5}(?P<ERROR>\s+"+*+")

the last part of the regular expression s+"*") is complex, I'm wandering how can i say all words, spaces, and numbers within " " to be associated with my field Errors.

I think this is saying new field name ERROR new line, non capture, except new line, and any value, 5 characters for in ERROR space than then this part would be to include everything within quotes?

My Goal is to simply create the field Error = quotes, words, spaces, numbers, and some more quotes.?

0 Karma

asarran
Path Finder

I think this Regular Expression may work, but I cannot figure out the error.

rex field=WARNING "*\s+Anomalies\s+(?+[^"]+)\\"

Sample Data:

 WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
 WARNING  210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
 WARNING  122 Errors " SOME ADDITIONAL 1 TEXT"

Referencing WARNING field.

rex field=WARNING

Indicating ALL variations and one to some spaces

"*\s+

data along this path after Anomalies

"+Anomalies"

One to Many spaces "AND"

\s+

Call this field warning

(?

AND include " everything inside AND

+[^"]+)

Stop when you see "

\\"

I'm thinking the concept may be accurate, but the syntax for sure is incorrect. I'm not sure what is not working.

0 Karma

asarran
Path Finder

Hey Fellow Splunkers

Still no luck, I wasn't able to utilize any regular expression to solve my issue. So I decided to break it up.

Sample Data:

WARNING 344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"
WARNING 210 Errors "THE ERROR IS 5 LESS THAN EXPECTED"
WARNING 122 Errors " SOME ADDITIONAL 1 TEXT"

Warning 344
Warning 210
Warning 122

Warning = Field

I've decided to create a second Field as Errors. However, I'm now having issues correlating Errors to the complex string with quotes.

rex "Errors\s?\((?[^\)]+)" I believe this is emphasis on parenthesis not quotes.

0 Karma

dbcase
Motivator

You can also use Splunk's erex command. If you give it the field and some sample data it will write the rex for you.

http://docs.splunk.com/Documentation/Splunk/6.4.2/SearchReference/Erex

0 Karma

dbcase
Motivator

whups,

Couple of typo's, use this one

index=blah sourcetype=blah blah | erex {your data field name goes here} examples="344 Errors THE ERROR IS 1 MORE THAN EXPECTED, 210 Errors THE ERROR IS 5 LESS THAN EXPECTED"
0 Karma

dbcase
Motivator

Try this

index=blah sourctype=blah blah | erex {your data field name goes here}  examples="344 Errors "THE ERROR IS 1 MORE THAN EXPECTED, THE ERROR IS 5 LESS THAN EXPECTED"

Once it is done running click the JOB drop down, you should see a generated rex command that should work for your needs.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This will extract the first word as the error level and the remaining text in the 'message' field.

(?<level>\w+)\s+(?<message>.*)
---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Here's an update based on your latest comment. It will extract "Warning", the number following it, and the text in quotes.

... | rex "(?<level>\w+)\s+(?<errorCount>\d+)\s+Errors\s+\"(?<message>.*)\"" | ...
---
If this reply helps you, Karma would be appreciated.
0 Karma

skoelpin
SplunkTrust
SplunkTrust

So you want to extract the entire line of text which can have numbers, words, and quotes right?

If so then we will need more sample data to build a good regular expression.. We ideally need to know if there is anything before or after this text and if its at the beginning or end of the line.. Also will it be a pattern? will it have WARNING|INFO|ERROR or will it always be warning? Will it also have a digit(s) after the warning message and will it always have text in the quotes or will it not have quotes sometimes?

asarran
Path Finder

Sorry, Skoelpin

I'm sorry, I forgot to provide some sample data

Aug 10 09:53:40 SomeHost SuperTrooper: 2-04-2015 1:4:15 WARNING  344 Errors "THE ERROR IS 1 MORE THAN EXPECTED"

I created a simple field extraction for WARNING. Ideally I had wanted to use WARNING for the field and 344 Errors "THE ERROR IS 1 MORE THAN EXPECTED" as the value, but I wasn't able to figure that out.

So I would like to extract the field ERRORS and have multiple complex strings within quotes.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What fields do you want to extract from that example?

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...