What will be the regex for the below?

[abhi04]

I need a regex to capture Parameters list i.e. Name, Category, Publisher, Version,Build Release, Branch, Packaging Date,Size and their respective values as x11/session/xauth, System/X11, solaris , 1.0.7, 5.11,
0.175.3.0.0.30.1483, 0.175.3.0.0.30.1483, August 21, 2015 05:37:42 PM, 100.68 kB respectively and give all values when using below query:

table Parameter values

Below is the sample data.

Name: x11/session/xauth
Summary: xauth - X authority file utility
Description: The xauth program is used to edit and display the
authorization information used in connecting to the X server.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.0.7
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:37:42 PM
Size: 100.68 kB
FMRI: pkg://solaris/x11/session/xauth@1.0.7,5.11-0.175.3.0.0.30.1483:20150821T173742Z

[FrankVl]

I’d suggest to use props and transforms to set up individual extractions for each field. That way the order of the fields does not matter.

Might even work like this (not tested), which takes the part befor the first : as the key and the part after (until end of line) as the value and keeps matching that for each line of the event.

Props.conf

[yoursourcetype]
REPORT-extract-my-fields = extractmyfields

Transforms.conf

[extractmyfields]
REGEX = (?m)^([^:]+):\s+(.*)$
FORMAT = $1::$2

[lacastillo]

Make sure that your REPORT-extract-my-fields value in props.conf matches the stanza in Transforms.conf

in props.conf
REPORT-extract-my-fields = extractmyfields <---

in transforms.conf
[extractmyfields]

[FrankVl]

Thanks for catching that. Fixed it in my answer 🙂

[jodyfsu]

Assuming the event will have the same formatting as above:

search
| rex "Name\:\s(?<NAME>.[^\n]+)\sSummary\:\s(?<SUM>.[^\n]+)\sDescription\:\s(?<DES>.[^\.]+)\.\sCategory\:\s(?<CAT>.[^\n]+)\sState\:\s(?<STATE>.[^\n]+)\sPublisher\:\s(?<PUB>.[^\n]+)\sVersion\:\s(?<VER>.[^\n]+)\sBuild\sRelease\:(?<BUILD>.[^\n]+)\sBranch\:\s(?<BRANCH>.[^\n]+)\sPackaging\sDate\:(?<PDATE>.[^\n]+)\sSize\:(?<SIZE>.[^\n]+)\sFMRI\:\s(?<FMRI>.[^\n]+)"
| table NAME SUM DES CAT STATE PUB VER BUILD BRANCH PDATE FMRI

Let us know if you need more.

[jodyfsu]

Hello @abhi04, so here is an updated regex:

| rex "Name\:\s(?<NAME>.[^\n]+)\nSummary\:\s(?<SUM>.[^\n]+)\nDescription\:\s(?<DES>.+\n.+\n.+\n.+\n.+)\nCategory\:\s(?<CAT>.[^\n]+)\nState\:\s(?<STATE>.[^\n]+)\nPublisher\:\s(?<PUB>.[^\n]+)\nVersion\:\s(?<VER>.[^\n]+)\nBuild\sRelease\:(?<BUILD>.[^\n]+)\nBranch\:\s(?<BRANCH>.[^\n]+)\nPackaging\sDate\:(?<PDATE>.[^\n]+)\nSize\:(?<SIZE>.[^\n]+)\nFMRI\:\s(?<FMRI>.[^\n]+)"

Assuming the number of line breaks remains the same, the above will work. If it varies on each event we will need to come up with another plan for the Description field. In the DES line each "\n" is a line break or carriage return. If the DES field varies we may need to figure out a way to do some OR "|" lookups.

[abhi04]

The number of line breaks is not same in Description field.

[jodyfsu]

🙂 Okay, I think I have it:

| rex "Name\:\s(?<NAME>.[^\n]+)\nSummary\:\s(?<SUM>.[^\n]+)\nDescription\:\s(?<DES>(?:\s|.)*?(?=Category))Category\:\s(?<CAT>.[^\n]+)\nState\:\s(?<STATE>.[^\n]+)\nPublisher\:\s(?<PUB>.[^\n]+)\nVersion\:\s(?<VER>.[^\n]+)\nBuild\sRelease\:(?<BUILD>.[^\n]+)\nBranch\:\s(?<BRANCH>.[^\n]+)\nPackaging\sDate\:(?<PDATE>.[^\n]+)\nSize\:(?<SIZE>.[^\n]+)\nFMRI\:\s(?<FMRI>.[^\n]+)"

Let me know if there are issues.

[abhi04]

Hi @jodyfsu,

what if the description have more than one full stop.The one you provided above works for only a single sentence in description. Below is the sample data:

Name: x11/library/toolkit/libxt
Summary: libXt - X Toolkit Intrinsics library
Description: The X Toolkit Intrinsics are a programming library tailored
to the special requirements of user interface construction
within a network window system, specifically the X Window
System. The X Toolkit Intrinsics and a widget set such as the
Athena Widgets (Xaw) or Motif (Xm) make up an X Toolkit.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.1.4
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:36:35 PM
Size: 2.86 MB
FMRI: pkg://solaris/x11/library/toolkit/libxt@1.1.4,5.11-0.175.3.0.0.30.1483:20150821T173635Z

[abhi04]

Hi jodyfsu,

I want the different parameters name to be captured in a single variable and their values in a separate variable. So we will have below list of Parameters and their values.

table Parameter Values

[FrankVl]

Not sure why you would want that (as you will effectively loose a proper connection between parameter and its value), but you could try this:

| rex max_match=0 "(?m)^(?<Parameters>[^:]+):\s+(?<Values>.*)$"

[abhi04]

@FrankVI
The below regex works in the regex site101 but not in splunk, it is only catching the parameter as Name and Description and not others. Any idea why?

| rex max_match=0 "(?m)^(?[^:]+):\s+(?.*)$"

[FrankVl]

Can you please post the query as code using the 101010 button or by wrapping it in ` characters? Now I think some parts have disappeared.

[abhi04]

Hi @FrankVI

This does not work. I want to use below command

chart limit=0 values(abc) over Parameter by Server_Name

So I need all parameters name in the "Parameter" and their values in the "abc"

[FrankVl]

Then just replace Values in my regex with abc?

If I misunderstood what you want, please provide some mockup of the output you are looking for, based on your example.

[abhi04]

Below is the mock-up.The regex you provided does not even after replacing.

Parameter. Server1 server2

Name. X11/session/xauth. X11/library/toolkit
Category. System/x11. System/core

[FrankVl]

Well, earlier you stated you wanted all parameter values in 1 variable and the values in another. That is exactly what my regex does. But as mentioned: that is probably not very useful, as it will be very difficult to work with those multivalued fields.

Better extract each parameter and its value as separate fields (as suggested by @jodyfsu) and then create a stats command that captures the values() of each field by Server_Name.

[abhi04]

Yes @FrankVI, I agree with you but then if I use stats command to list the values, will I be able to compare those values for different server and display message "same" "not same"
if any of the parameter values for any one server is different. If yes, how?

[FrankVl]

Let's cover that in the separate question you posted for that: https://answers.splunk.com/answers/665799/how-to-compare-more-that-50-column-values-for-a-sp.html

Otherwise things get confusing 🙂

[abhi04]

@FrankVI thats what my mock table shows. "Name" and "Category" are different parameters stored in "Parameter" variable.
"X11/session/xauth" ,"X11/library/toolkit","System/x11" and "System/core" are the values to be stored in "abc"

[jodyfsu]

Sorry I missed that earlier. Here is what I came up with:

search
| rex "(?<P1>Name\:)\s(?<NAME>.[^\n]+)\s(?<P2>Summary\:)\s(?<SUM>.[^\n]+)\s(?<P3>Description\:)\s(?<DES>.[^\.]+)\.\s(?<P4>Category\:)\s(?<CAT>.[^\n]+)\s(?<P5>State\:)\s(?<STATE>.[^\n]+)\s(?<P6>Publisher\:)\s(?<PUB>.[^\n]+)\s(?<P7>Version\:)\s(?<VER>.[^\n]+)\s(?<P8>Build\sRelease\:)\s(?<BUILD>.[^\n]+)\s(?<P9>Branch\:)\s(?<BRANCH>.[^\n]+)\s(?<P10>Packaging\sDate\:)\s(?<PDATE>.[^\n]+)\s(?<P11>Size\:)\s(?<SIZE>.[^\n]+)\s(?<P12>FMRI\:)\s(?<FMRI>.[^\n]+)"
| table P1 NAME P2 SUM P3 DES P4 CAT P5 STATE P6 PUB P7 VER P8 BUILD P9 BRANCH P10 PDATE P11 SIZE P12 FMRI