topic Re: Help with field extraction for specific start line for log in Splunk Search

Help with field extraction for specific start line for log

joesrepsolc — Fri, 09 Aug 2019 15:18:58 GMT

Trying to extract the value of the 1st WORD in line 3 of each log (i.e. FAILURE or SUCCESS) and put that into a field extraction called "Status". The 3rd line will start with that word, then be followed by a colon, then a space.
REGEX skills still lacking. Trying to learn. Help anyone???

>Informational  August 9, 2019 9:45:40 AM CDT
Transaction "Name of Transaction" Summary:
FAILURE: Message failed at August 09, 2019 09:45:40 AM
Inbound transaction
Error scheduling linked transaction (TransactionName): Attempt to run TransactionName as linked job but it has 'Do Not Run' set.

>Informational  August 9, 2019 9:50:02 AM CDT
Transaction "Name of Transaction" Summary:
SUCCESS: TransactionName was successful at August 09, 2019 09:50:02 AM
Outbound transaction
Source files FTP'd from SOURCE to DESTINATION

Re: Help with field extraction for specific start line for log

gcusello — Fri, 09 Aug 2019 15:27:11 GMT

Hi joesrepsolc,
if your values are only SUCCESS or FAILURE, you could use something like this

(?ms)(?<my_field>FAILURE|SUCCESS):

test it at https://regex101.com/r/UEahRa/1

Bye.
Giuseppe

Re: Help with field extraction for specific start line for log

jpolvino — Fri, 09 Aug 2019 15:43:20 GMT

This will give you whatever is after "Summary:" followed by 1 more more spaces:

| rex "Summary:\s+(?<Status>[^:]+):"

See link text

Re: Help with field extraction for specific start line for log

joesrepsolc — Fri, 09 Aug 2019 15:56:02 GMT

WOW. that was fast and works perfect! dissecting the regex to understand. Always impressed on how helpful everyone is on this site. So thank you!

I was stuck on using the carrot ^ at the beginning of the line, guess that would still work though, just learning more about regex use in Splunk.

Re: Help with field extraction for specific start line for log

joesrepsolc — Fri, 09 Aug 2019 15:58:24 GMT

This works great for inline searches too. I was trying not to have to do this in every search though, so the field extraction method is working best for me. Still appreciate the quick answers too jpolvino.

I see the logic you're using too. So anything after the "Summary: " up to the next line up to the ":" right? Man, I need to understand this stuff better.

Thank you.

Re: Help with field extraction for specific start line for log

michael_schmidt — Fri, 09 Aug 2019 16:17:13 GMT

Regex101.com as shown in the links they provided will be your godsend when it comes to creating and testing regexes.

Re: Help with field extraction for specific start line for log

jpolvino — Fri, 09 Aug 2019 17:00:15 GMT

Glad Giuseppe's search worked for you. The strategy I used above is handy for may searches, where you are looking for a bunch of characters that are NOT something. In my example, the [^:]+ means "match 1 more more characters that does not include colon." This strategy is invaluable when working with delimiters such as space, comma, double quote, etc.

Re: Help with field extraction for specific start line for log

joesrepsolc — Fri, 09 Aug 2019 17:34:24 GMT

i am using that, just hard to figure out which commands/syntax works. When I copy/paste in the ones from this helpful community, they always work perfectly (annoying... haha). It's just getting to the point to know when to use what solution. I'll get there!!!

Re: Help with field extraction for specific start line for log

joesrepsolc — Fri, 09 Aug 2019 17:35:11 GMT

love it. I still get mixed up between characters, and whole words. Still learning. But this logic is very helpful. Thanks!

Re: Help with field extraction for specific start line for log

gcusello — Sat, 10 Aug 2019 06:29:28 GMT

Always at your disposal!
Bye and see next time.
Giuseppe