topic Re: help to extract four fields in a log file with regex in Splunk Search

help to extract four fields in a log file with regex

jip31 — Tue, 26 Mar 2019 14:35:04 GMT

I want to extract the four fields after the text in yellow color and in the same line except the last field where I need to have the fields after ../
Could you help me to do this please?

Re: help to extract four fields in a log file with regex

nickhills — Tue, 26 Mar 2019 14:46:00 GMT

Wow, that's a horrid log!
Do you have this data in Splunk already - how does it look, does it get indexed as one event, or broken into multiple events?

If you can put a copy/paste-able version of the log that would help too.

Re: help to extract four fields in a log file with regex

FrankVl — Tue, 26 Mar 2019 14:48:11 GMT

Any chance you can provide sample data as text, with the sensitive values replaced with representative placeholders? This screenshot is making it a bit difficult to see what exactly the regex should be and also it requires anyone who wants to write a testcase for your question to have to type all that sample data manually from the screenshot.

Re: help to extract four fields in a log file with regex

jip31 — Wed, 27 Mar 2019 05:52:18 GMT

here is the logs
I want to extract fields in yellow (see screenshot) and to display them in a table
thanks

06/09/2018 - 14:21:24 -- Installation of **
14:21:29 ./ Check Product XXXXXXXX
../ Completed
.../ Not installed
* 14:21:29 ./ Check Disk Size ( must be > to 15 MO)
* ../ Completed
* .../ Installation user context: XXXXXXXX
14:21:30 ./ Starting 'XXXXXXXX
../ Completed
* 14:22:13 ./ Starting 'XXXXXXXX Synchro Step 1
* ../ Completed
* 14:22:42 ./ Starting 'XXXXXXXX Synchro Step 2
* ../ Failed Error code : ****
* 14:23:01 ./ Starting 'XXXXXXXX Synchro Step 3
* ../ Failed Error code : 268435509
14:23:01 ./ Installation Status
../ Completed
06/09/2018 - 14:23:01 -- End of installation of ****
14:23:08 ./ Check Product XXXXXXXX installation Status
../ ****
.../ *********

Re: help to extract four fields in a log file with regex

FrankVl — Wed, 27 Mar 2019 09:44:33 GMT

Providing the solution as rex commands, shouldn't be too hard to translate that to props/transforms if needed.

| rex "Installation\s+of\s+(?<Installation_of>\S+)"
| rex "Failed\s+Error\s+code\s+:\s+(?<Failed_Error_code>\S+)"
| rex "End\s+of\s+installation\s+of\s+(?<End_of_installation_of>\S+)"
| rex "Check\s+Product\s+\S+\s+installation\s+Status\s+\.+\/\s+(?<installation_status>\S+)"
| rex "Check\s+Product\s+\S+\s+installation\s+Status\s+\.+\/\s+\S+\s+\.+\/\s+(?<installation_status2>\S+)"

Not sure how exactly you like to have those 2 bits of the installation status extracted, I put them into separate fields now. You can easily merge those if you want.

Re: help to extract four fields in a log file with regex

jip31 — Tue, 29 Sep 2020 23:49:51 GMT

Hi
For the first rex perfect
For the second almost perfect : sometimes, I have results which are not error code : *, 1 or error code but with a minus sign before
and for the other rex, any results

and furthermore I would like to do something complex
In my table, I just want to have a specific line by host and to values the result of the fields below preferably without doubloon
Installation_of Failed_Error_code installation_status installation_status2

thanks for you help

Re: help to extract four fields in a log file with regex

FrankVl — Thu, 28 Mar 2019 08:26:48 GMT

I'm sorry, I can't make much sense of your comment.

For the second regex: \S+ should also match if there is a - before the error code. Please share better sample data, or better explain what is wrong with the suggested regex.

What do you mean by "and for the other rex, any results"?

It helps if you could provide a made up sample of what you want the output to look like.

Re: help to extract four fields in a log file with regex

jip31 — Thu, 28 Mar 2019 09:16:31 GMT

by What do you mean by "and for the other rex, any results" I mean that the regex extract anything event if I have result in the log
I added an example of what I want in the question
for the moment I just want to use 2 fields

Re: help to extract four fields in a log file with regex

FrankVl — Thu, 28 Mar 2019 09:32:48 GMT

You mean does not extract anything?

OK, so get the extractions working and then do a | stats list(Installation_of) as Installation_of list(Failed_Error_code) as Failed_Error_code by host. Note: use list, not values, otherwise each multivalue field is lexicographically sorted and you loose the relation between installation and error code.

If you need further help getting the extractions working, please be more clear in what the problem is with the suggested regex and provide more representative sample data (especially of the cases where the regex currently fails).