Splunk Search

What will be the regex for the below?

abhi04
Communicator

I need a regex to capture Parameters list i.e. Name, Category, Publisher, Version,Build Release, Branch, Packaging Date,Size and their respective values as x11/session/xauth, System/X11, solaris , 1.0.7, 5.11,
0.175.3.0.0.30.1483, 0.175.3.0.0.30.1483, August 21, 2015 05:37:42 PM, 100.68 kB respectively and give all values when using below query:

table Parameter values

Below is the sample data.

Name: x11/session/xauth
Summary: xauth - X authority file utility
Description: The xauth program is used to edit and display the
authorization information used in connecting to the X server.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.0.7
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:37:42 PM
Size: 100.68 kB
FMRI: pkg://solaris/x11/session/xauth@1.0.7,5.11-0.175.3.0.0.30.1483:20150821T173742Z

Tags (1)
0 Karma

FrankVl
Ultra Champion

I’d suggest to use props and transforms to set up individual extractions for each field. That way the order of the fields does not matter.

Might even work like this (not tested), which takes the part befor the first : as the key and the part after (until end of line) as the value and keeps matching that for each line of the event.

Props.conf

[yoursourcetype]
REPORT-extract-my-fields = extractmyfields

Transforms.conf

[extractmyfields]
REGEX = (?m)^([^:]+):\s+(.*)$
FORMAT = $1::$2

lacastillo
Path Finder

Make sure that your REPORT-extract-my-fields value in props.conf matches the stanza in Transforms.conf

in props.conf
REPORT-extract-my-fields = extractmyfields <---

in transforms.conf
[extractmyfields]

FrankVl
Ultra Champion

Thanks for catching that. Fixed it in my answer 🙂

0 Karma

jodyfsu
Path Finder

Assuming the event will have the same formatting as above:

search
| rex "Name\:\s(?<NAME>.[^\n]+)\sSummary\:\s(?<SUM>.[^\n]+)\sDescription\:\s(?<DES>.[^\.]+)\.\sCategory\:\s(?<CAT>.[^\n]+)\sState\:\s(?<STATE>.[^\n]+)\sPublisher\:\s(?<PUB>.[^\n]+)\sVersion\:\s(?<VER>.[^\n]+)\sBuild\sRelease\:(?<BUILD>.[^\n]+)\sBranch\:\s(?<BRANCH>.[^\n]+)\sPackaging\sDate\:(?<PDATE>.[^\n]+)\sSize\:(?<SIZE>.[^\n]+)\sFMRI\:\s(?<FMRI>.[^\n]+)"
| table NAME SUM DES CAT STATE PUB VER BUILD BRANCH PDATE FMRI

Let us know if you need more.

0 Karma

jodyfsu
Path Finder

Hello @abhi04, so here is an updated regex:

| rex "Name\:\s(?<NAME>.[^\n]+)\nSummary\:\s(?<SUM>.[^\n]+)\nDescription\:\s(?<DES>.+\n.+\n.+\n.+\n.+)\nCategory\:\s(?<CAT>.[^\n]+)\nState\:\s(?<STATE>.[^\n]+)\nPublisher\:\s(?<PUB>.[^\n]+)\nVersion\:\s(?<VER>.[^\n]+)\nBuild\sRelease\:(?<BUILD>.[^\n]+)\nBranch\:\s(?<BRANCH>.[^\n]+)\nPackaging\sDate\:(?<PDATE>.[^\n]+)\nSize\:(?<SIZE>.[^\n]+)\nFMRI\:\s(?<FMRI>.[^\n]+)"

Assuming the number of line breaks remains the same, the above will work. If it varies on each event we will need to come up with another plan for the Description field. In the DES line each "\n" is a line break or carriage return. If the DES field varies we may need to figure out a way to do some OR "|" lookups.

0 Karma

abhi04
Communicator

The number of line breaks is not same in Description field.

0 Karma

jodyfsu
Path Finder

🙂 Okay, I think I have it:

| rex "Name\:\s(?<NAME>.[^\n]+)\nSummary\:\s(?<SUM>.[^\n]+)\nDescription\:\s(?<DES>(?:\s|.)*?(?=Category))Category\:\s(?<CAT>.[^\n]+)\nState\:\s(?<STATE>.[^\n]+)\nPublisher\:\s(?<PUB>.[^\n]+)\nVersion\:\s(?<VER>.[^\n]+)\nBuild\sRelease\:(?<BUILD>.[^\n]+)\nBranch\:\s(?<BRANCH>.[^\n]+)\nPackaging\sDate\:(?<PDATE>.[^\n]+)\nSize\:(?<SIZE>.[^\n]+)\nFMRI\:\s(?<FMRI>.[^\n]+)"

Let me know if there are issues.

0 Karma

abhi04
Communicator

Hi @jodyfsu,

what if the description have more than one full stop.The one you provided above works for only a single sentence in description. Below is the sample data:

Name: x11/library/toolkit/libxt
Summary: libXt - X Toolkit Intrinsics library
Description: The X Toolkit Intrinsics are a programming library tailored
to the special requirements of user interface construction
within a network window system, specifically the X Window
System. The X Toolkit Intrinsics and a widget set such as the
Athena Widgets (Xaw) or Motif (Xm) make up an X Toolkit.
Category: System/X11
State: Installed
Publisher: solaris
Version: 1.1.4
Build Release: 5.11
Branch: 0.175.3.0.0.30.1483
Packaging Date: August 21, 2015 05:36:35 PM
Size: 2.86 MB
FMRI: pkg://solaris/x11/library/toolkit/libxt@1.1.4,5.11-0.175.3.0.0.30.1483:20150821T173635Z

0 Karma

abhi04
Communicator

Hi jodyfsu,

I want the different parameters name to be captured in a single variable and their values in a separate variable. So we will have below list of Parameters and their values.

table Parameter Values

FrankVl
Ultra Champion

Not sure why you would want that (as you will effectively loose a proper connection between parameter and its value), but you could try this:

| rex max_match=0 "(?m)^(?<Parameters>[^:]+):\s+(?<Values>.*)$"
0 Karma

abhi04
Communicator

@FrankVI
The below regex works in the regex site101 but not in splunk, it is only catching the parameter as Name and Description and not others. Any idea why?

| rex max_match=0 "(?m)^(?[^:]+):\s+(?.*)$"

0 Karma

FrankVl
Ultra Champion

Can you please post the query as code using the 101010 button or by wrapping it in ` characters? Now I think some parts have disappeared.

0 Karma

abhi04
Communicator

Hi @FrankVI

This does not work. I want to use below command

chart limit=0 values(abc) over Parameter by Server_Name

So I need all parameters name in the "Parameter" and their values in the "abc"

0 Karma

FrankVl
Ultra Champion

Then just replace Values in my regex with abc?

If I misunderstood what you want, please provide some mockup of the output you are looking for, based on your example.

0 Karma

abhi04
Communicator

Below is the mock-up.The regex you provided does not even after replacing.

Parameter. Server1 server2

Name. X11/session/xauth. X11/library/toolkit
Category. System/x11. System/core

0 Karma

FrankVl
Ultra Champion

Well, earlier you stated you wanted all parameter values in 1 variable and the values in another. That is exactly what my regex does. But as mentioned: that is probably not very useful, as it will be very difficult to work with those multivalued fields.

Better extract each parameter and its value as separate fields (as suggested by @jodyfsu) and then create a stats command that captures the values() of each field by Server_Name.

0 Karma

abhi04
Communicator

Yes @FrankVI, I agree with you but then if I use stats command to list the values, will I be able to compare those values for different server and display message "same" "not same"
if any of the parameter values for any one server is different. If yes, how?

0 Karma

FrankVl
Ultra Champion

Let's cover that in the separate question you posted for that: https://answers.splunk.com/answers/665799/how-to-compare-more-that-50-column-values-for-a-sp.html

Otherwise things get confusing 🙂

0 Karma

abhi04
Communicator

@FrankVI thats what my mock table shows. "Name" and "Category" are different parameters stored in "Parameter" variable.
"X11/session/xauth" ,"X11/library/toolkit","System/x11" and "System/core" are the values to be stored in "abc"

0 Karma

jodyfsu
Path Finder

Sorry I missed that earlier. Here is what I came up with:

search
| rex "(?<P1>Name\:)\s(?<NAME>.[^\n]+)\s(?<P2>Summary\:)\s(?<SUM>.[^\n]+)\s(?<P3>Description\:)\s(?<DES>.[^\.]+)\.\s(?<P4>Category\:)\s(?<CAT>.[^\n]+)\s(?<P5>State\:)\s(?<STATE>.[^\n]+)\s(?<P6>Publisher\:)\s(?<PUB>.[^\n]+)\s(?<P7>Version\:)\s(?<VER>.[^\n]+)\s(?<P8>Build\sRelease\:)\s(?<BUILD>.[^\n]+)\s(?<P9>Branch\:)\s(?<BRANCH>.[^\n]+)\s(?<P10>Packaging\sDate\:)\s(?<PDATE>.[^\n]+)\s(?<P11>Size\:)\s(?<SIZE>.[^\n]+)\s(?<P12>FMRI\:)\s(?<FMRI>.[^\n]+)"
| table P1 NAME P2 SUM P3 DES P4 CAT P5 STATE P6 PUB P7 VER P8 BUILD P9 BRANCH P10 PDATE P11 SIZE P12 FMRI
0 Karma
Get Updates on the Splunk Community!

Splunk is Nurturing Tomorrow’s Cybersecurity Leaders Today

Meet Carol Wright. She leads the Splunk Academic Alliance program at Splunk. The Splunk Academic Alliance ...

Part 2: A Guide to Maximizing Splunk IT Service Intelligence

Welcome to the second segment of our guide. In Part 1, we covered the essentials of getting started with ITSI ...

Part 1: A Guide to Maximizing Splunk IT Service Intelligence

As modern IT environments continue to grow in complexity and speed, the ability to efficiently manage and ...