topic Re: What will be the regex for the below? in Splunk Search

What will be the regex for the below?

abhi04 — Wed, 13 Jun 2018 13:02:15 GMT

I need a regex to capture Parameters list i.e. Name, Category, Publisher, Version,Build Release, Branch, Packaging Date,Size and their respective values as x11/session/xauth, System/X11, solaris , 1.0.7, 5.11,
0.175.3.0.0.30.1483, 0.175.3.0.0.30.1483, August 21, 2015 05:37:42 PM, 100.68 kB respectively and give all values when using below query:

table Parameter values

Below is the sample data.

Name: x11/session/xauth
Summary: xauth - X authority file utility
Description: The xauth program is used to edit and display the
authorization information used in connecting to the X server.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.0.7
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:37:42 PM
Size: 100.68 kB
FMRI: pkg://solaris/x11/session/xauth@1.0.7,5.11-0.175.3.0.0.30.1483:20150821T173742Z

Re: What will be the regex for the below?

jodyfsu — Wed, 13 Jun 2018 13:17:01 GMT

Assuming the event will have the same formatting as above:

search
| rex "Name\:\s(?<NAME>.[^\n]+)\sSummary\:\s(?<SUM>.[^\n]+)\sDescription\:\s(?<DES>.[^\.]+)\.\sCategory\:\s(?<CAT>.[^\n]+)\sState\:\s(?<STATE>.[^\n]+)\sPublisher\:\s(?<PUB>.[^\n]+)\sVersion\:\s(?<VER>.[^\n]+)\sBuild\sRelease\:(?<BUILD>.[^\n]+)\sBranch\:\s(?<BRANCH>.[^\n]+)\sPackaging\sDate\:(?<PDATE>.[^\n]+)\sSize\:(?<SIZE>.[^\n]+)\sFMRI\:\s(?<FMRI>.[^\n]+)"
| table NAME SUM DES CAT STATE PUB VER BUILD BRANCH PDATE FMRI

Let us know if you need more.

Re: What will be the regex for the below?

FrankVl — Wed, 13 Jun 2018 13:20:37 GMT

I’d suggest to use props and transforms to set up individual extractions for each field. That way the order of the fields does not matter.

Might even work like this (not tested), which takes the part befor the first : as the key and the part after (until end of line) as the value and keeps matching that for each line of the event.

Props.conf

[yoursourcetype]
REPORT-extract-my-fields = extractmyfields

Transforms.conf

[extractmyfields]
REGEX = (?m)^([^:]+):\s+(.*)$
FORMAT = $1::$2

Re: What will be the regex for the below?

abhi04 — Wed, 13 Jun 2018 13:25:10 GMT

Hi jodyfsu,

I want the different parameters name to be captured in a single variable and their values in a separate variable. So we will have below list of Parameters and their values.

table Parameter Values

Re: What will be the regex for the below?

lacastillo — Wed, 13 Jun 2018 13:43:00 GMT

Make sure that your REPORT-extract-my-fields value in props.conf matches the stanza in Transforms.conf

in props.conf
REPORT-extract-my-fields = extractmyfields <---

in transforms.conf
[extractmyfields]

Re: What will be the regex for the below?

jodyfsu — Wed, 13 Jun 2018 13:47:32 GMT

Sorry I missed that earlier. Here is what I came up with:

search
| rex "(?<P1>Name\:)\s(?<NAME>.[^\n]+)\s(?<P2>Summary\:)\s(?<SUM>.[^\n]+)\s(?<P3>Description\:)\s(?<DES>.[^\.]+)\.\s(?<P4>Category\:)\s(?<CAT>.[^\n]+)\s(?<P5>State\:)\s(?<STATE>.[^\n]+)\s(?<P6>Publisher\:)\s(?<PUB>.[^\n]+)\s(?<P7>Version\:)\s(?<VER>.[^\n]+)\s(?<P8>Build\sRelease\:)\s(?<BUILD>.[^\n]+)\s(?<P9>Branch\:)\s(?<BRANCH>.[^\n]+)\s(?<P10>Packaging\sDate\:)\s(?<PDATE>.[^\n]+)\s(?<P11>Size\:)\s(?<SIZE>.[^\n]+)\s(?<P12>FMRI\:)\s(?<FMRI>.[^\n]+)"
| table P1 NAME P2 SUM P3 DES P4 CAT P5 STATE P6 PUB P7 VER P8 BUILD P9 BRANCH P10 PDATE P11 SIZE P12 FMRI

Re: What will be the regex for the below?

FrankVl — Wed, 13 Jun 2018 13:49:40 GMT

Not sure why you would want that (as you will effectively loose a proper connection between parameter and its value), but you could try this:

| rex max_match=0 "(?m)^(?<Parameters>[^:]+):\s+(?<Values>.*)$"

Re: What will be the regex for the below?

FrankVl — Wed, 13 Jun 2018 13:52:01 GMT

Thanks for catching that. Fixed it in my answer 🙂

Re: What will be the regex for the below?

abhi04 — Thu, 14 Jun 2018 10:59:33 GMT

Hi @FrankVI

This does not work. I want to use below command

chart limit=0 values(abc) over Parameter by Server_Name

So I need all parameters name in the "Parameter" and their values in the "abc"

Re: What will be the regex for the below?

FrankVl — Thu, 14 Jun 2018 11:04:33 GMT

Then just replace Values in my regex with abc?

If I misunderstood what you want, please provide some mockup of the output you are looking for, based on your example.

Re: What will be the regex for the below?

abhi04 — Thu, 14 Jun 2018 11:41:08 GMT

Below is the mock-up.The regex you provided does not even after replacing.

Parameter. Server1 server2

Name. X11/session/xauth. X11/library/toolkit
Category. System/x11. System/core

Re: What will be the regex for the below?

FrankVl — Thu, 14 Jun 2018 11:57:02 GMT

Well, earlier you stated you wanted all parameter values in 1 variable and the values in another. That is exactly what my regex does. But as mentioned: that is probably not very useful, as it will be very difficult to work with those multivalued fields.

Better extract each parameter and its value as separate fields (as suggested by @jodyfsu) and then create a stats command that captures the values() of each field by Server_Name.

Re: What will be the regex for the below?

abhi04 — Wed, 20 Jun 2018 07:13:20 GMT

Yes @FrankVI, I agree with you but then if I use stats command to list the values, will I be able to compare those values for different server and display message "same" "not same"
if any of the parameter values for any one server is different. If yes, how?

Re: What will be the regex for the below?

FrankVl — Wed, 20 Jun 2018 07:44:28 GMT

Let's cover that in the separate question you posted for that: https://answers.splunk.com/answers/665799/how-to-compare-more-that-50-column-values-for-a-sp.html

Otherwise things get confusing 🙂

Re: What will be the regex for the below?

abhi04 — Fri, 22 Jun 2018 12:28:09 GMT

@FrankVI thats what my mock table shows. "Name" and "Category" are different parameters stored in "Parameter" variable.
"X11/session/xauth" ,"X11/library/toolkit","System/x11" and "System/core" are the values to be stored in "abc"

Re: What will be the regex for the below?

abhi04 — Mon, 02 Jul 2018 13:36:48 GMT

@FrankVI
The below regex works in the regex site101 but not in splunk, it is only catching the parameter as Name and Description and not others. Any idea why?

| rex max_match=0 "(?m)^(?[^:]+):\s+(?.*)$"

Re: What will be the regex for the below?

FrankVl — Mon, 02 Jul 2018 13:55:43 GMT

Can you please post the query as code using the 101010 button or by wrapping it in ` characters? Now I think some parts have disappeared.

Re: What will be the regex for the below?

abhi04 — Wed, 04 Jul 2018 10:39:34 GMT

Hi @jodyfsu,

what if the description have more than one full stop.The one you provided above works for only a single sentence in description. Below is the sample data:

Name: x11/library/toolkit/libxt
Summary: libXt - X Toolkit Intrinsics library
Description: The X Toolkit Intrinsics are a programming library tailored
to the special requirements of user interface construction
within a network window system, specifically the X Window
System. The X Toolkit Intrinsics and a widget set such as the
Athena Widgets (Xaw) or Motif (Xm) make up an X Toolkit.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.1.4
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:36:35 PM
Size: 2.86 MB
FMRI: pkg://solaris/x11/library/toolkit/libxt@1.1.4,5.11-0.175.3.0.0.30.1483:20150821T173635Z

Re: What will be the regex for the below?

jodyfsu — Wed, 04 Jul 2018 11:31:31 GMT

Hello @abhi04, so here is an updated regex:

| rex "Name\:\s(?<NAME>.[^\n]+)\nSummary\:\s(?<SUM>.[^\n]+)\nDescription\:\s(?<DES>.+\n.+\n.+\n.+\n.+)\nCategory\:\s(?<CAT>.[^\n]+)\nState\:\s(?<STATE>.[^\n]+)\nPublisher\:\s(?<PUB>.[^\n]+)\nVersion\:\s(?<VER>.[^\n]+)\nBuild\sRelease\:(?<BUILD>.[^\n]+)\nBranch\:\s(?<BRANCH>.[^\n]+)\nPackaging\sDate\:(?<PDATE>.[^\n]+)\nSize\:(?<SIZE>.[^\n]+)\nFMRI\:\s(?<FMRI>.[^\n]+)"

Assuming the number of line breaks remains the same, the above will work. If it varies on each event we will need to come up with another plan for the Description field. In the DES line each "\n" is a line break or carriage return. If the DES field varies we may need to figure out a way to do some OR "|" lookups.

Re: What will be the regex for the below?

abhi04 — Wed, 04 Jul 2018 11:45:54 GMT

The number of line breaks is not same in Description field.