Splunk Search

Field Extaction from regex not displaying Character

pmcfadden91
Path Finder

Hi, I have an issue for extracting values. It extracts 7 out of the 8 characters I need to catch.
I currently have this:

 index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?P<FIELDNAME>**[i|n|p|o|r|a|j])**" | dedup FIELDNAME | table FIELDNAME

It does return all the characters listed in the regex onto the statistics table. However, when I add the character "b"( rex field=_raw "(?i)\..*? (?P&lt;FIELDNAME&gt;**[i|n|b|p|o|r|a|j]**)"), it only displays "a" and "b" in stats table. How do I correct?

0 Karma
1 Solution

landen99
Motivator

On your sample data, the following regex extracts the letters:

\*{2}(?P<FIELDNAME>[^\*]+)\*{2}

MATCH 1
FIELDNAME [150-157] b s i r
MATCH 2
FIELDNAME [339-346] b n p r
MATCH 3
FIELDNAME [539-546] a n i r
MATCH 4
FIELDNAME [756-763] a s o r

' It allows me to extract the combinations as individual characters into a field until I add I add the character "b" '

If I understood correctly about the part of matching everything until you get to a "b", the regex becomes:

\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}

MATCH 1
FIELDNAME [539-546] a n i r
MATCH 2
FIELDNAME [756-763] a s o r

You can then use makemv command to create a multivalue field of each string of single characters.

| rex "\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}"
| makemv FIELDNAME delim=" " allowempty=t

View solution in original post

0 Karma

landen99
Motivator

On your sample data, the following regex extracts the letters:

\*{2}(?P<FIELDNAME>[^\*]+)\*{2}

MATCH 1
FIELDNAME [150-157] b s i r
MATCH 2
FIELDNAME [339-346] b n p r
MATCH 3
FIELDNAME [539-546] a n i r
MATCH 4
FIELDNAME [756-763] a s o r

' It allows me to extract the combinations as individual characters into a field until I add I add the character "b" '

If I understood correctly about the part of matching everything until you get to a "b", the regex becomes:

\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}

MATCH 1
FIELDNAME [539-546] a n i r
MATCH 2
FIELDNAME [756-763] a s o r

You can then use makemv command to create a multivalue field of each string of single characters.

| rex "\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}"
| makemv FIELDNAME delim=" " allowempty=t
0 Karma

pmcfadden91
Path Finder

This worked when I plugged it in and tweaked a bit. Thanks!

0 Karma

landen99
Motivator

No problem. I used regex101.com to craft the regex. Try it out if you like.

0 Karma

pmcfadden91
Path Finder

Below are some log examples:

Fri Jul 17 14:22:15 2015 0 139.149.36.161 10032 /sbclocal/InternalSecureFileTransfer/users/ETDIT/gmiprod2skctest/chi/global_idt/comm/idts_00104.zip **b s i r** gmiprod2skctest ssh 0 *

Fri Jul 17 11:28:10 2015 0 localhost 0 /sbclocal/InternalSecureFileTransfer/users/ETDIT/gmi_test_ftp/gmi_to_skc/gmi/SKCU02/asia_memo/account_types.zip **b n p r** gmi_test_ftp folder 0 *

Fri Jul 17 11:03:29 2015 8 151.191.80.226 3341520 /sbclocal/InternalSecureFileTransfer/users/ETDIT/tpt_ftp_test/outbox/rexuat/ldn/GMI_OBS_IRSCME_POS_20150716.txt **a n i r** tpt_ftp_test ftp 0 *

Fri Jul 17 11:03:29 2015 1 rex_ldn_uat1_cmp.ldn.swissbank.com 3341520 /sbclocal/InternalSecureFileTransfer/users/ETDIT/tpt_ftp_test/outbox/rexuat/ldn/GMI_OBS_IRSCME_POS_20150716.txt **a s o r** tpt_ftp_test ssh 0 *

It allows me to extract the combinations as individual characters into a field until I add I add the character "b". Also I was wondering if there is a way to extract different combinations(as shown in examples) and the single characters of the combos all into one field? I have a regex that pulls all combinations now. I just wanted to combine them.

 index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?P<FIELDNAME>\w+\s+\w+\s+\w+\s+\w+)\s+\w+" | dedup FIELDNAME | table FIELDNAME
0 Karma

landen99
Motivator

Your regex indicates that there may be a pattern like ".stuff **a**" We really need to see the raw data and the desired result before we can understand the best regex for it. I am thinking that the following may extract you results much better:

index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\*{2}(?<FIELDNAME>[inbporaj])\*{2}" | dedup FIELDNAME | table FIELDNAME
0 Karma

woodcock
Esteemed Legend

It looks like your character class is inefficient and you are not escaping your asterisks; try this:

index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?<FIELDNAME>\*\*[inbporaj])\*\*" | dedup FIELDNAME | table FIELDNAME
0 Karma

MuS
SplunkTrust
SplunkTrust

can you provide some log samples?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...