Splunk Search

Field Extaction from regex not displaying Character

Path Finder

Hi, I have an issue for extracting values. It extracts 7 out of the 8 characters I need to catch.
I currently have this:

 index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?P<FIELDNAME>**[i|n|p|o|r|a|j])**" | dedup FIELDNAME | table FIELDNAME

It does return all the characters listed in the regex onto the statistics table. However, when I add the character "b"( rex field=_raw "(?i)\..*? (?P&lt;FIELDNAME&gt;**[i|n|b|p|o|r|a|j]**)"), it only displays "a" and "b" in stats table. How do I correct?

0 Karma
1 Solution

Motivator

On your sample data, the following regex extracts the letters:

\*{2}(?P<FIELDNAME>[^\*]+)\*{2}

MATCH 1
FIELDNAME [150-157] b s i r
MATCH 2
FIELDNAME [339-346] b n p r
MATCH 3
FIELDNAME [539-546] a n i r
MATCH 4
FIELDNAME [756-763] a s o r

' It allows me to extract the combinations as individual characters into a field until I add I add the character "b" '

If I understood correctly about the part of matching everything until you get to a "b", the regex becomes:

\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}

MATCH 1
FIELDNAME [539-546] a n i r
MATCH 2
FIELDNAME [756-763] a s o r

You can then use makemv command to create a multivalue field of each string of single characters.

| rex "\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}"
| makemv FIELDNAME delim=" " allowempty=t

View solution in original post

0 Karma

Motivator

On your sample data, the following regex extracts the letters:

\*{2}(?P<FIELDNAME>[^\*]+)\*{2}

MATCH 1
FIELDNAME [150-157] b s i r
MATCH 2
FIELDNAME [339-346] b n p r
MATCH 3
FIELDNAME [539-546] a n i r
MATCH 4
FIELDNAME [756-763] a s o r

' It allows me to extract the combinations as individual characters into a field until I add I add the character "b" '

If I understood correctly about the part of matching everything until you get to a "b", the regex becomes:

\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}

MATCH 1
FIELDNAME [539-546] a n i r
MATCH 2
FIELDNAME [756-763] a s o r

You can then use makemv command to create a multivalue field of each string of single characters.

| rex "\*{2}(?P<FIELDNAME>[^b\*]+)\*{2}"
| makemv FIELDNAME delim=" " allowempty=t

View solution in original post

0 Karma

Path Finder

This worked when I plugged it in and tweaked a bit. Thanks!

0 Karma

Motivator

No problem. I used regex101.com to craft the regex. Try it out if you like.

0 Karma

Path Finder

Below are some log examples:

Fri Jul 17 14:22:15 2015 0 139.149.36.161 10032 /sbclocal/InternalSecureFileTransfer/users/ETDIT/gmiprod2skctest/chi/global_idt/comm/idts_00104.zip **b s i r** gmiprod2skctest ssh 0 *

Fri Jul 17 11:28:10 2015 0 localhost 0 /sbclocal/InternalSecureFileTransfer/users/ETDIT/gmi_test_ftp/gmi_to_skc/gmi/SKCU02/asia_memo/account_types.zip **b n p r** gmi_test_ftp folder 0 *

Fri Jul 17 11:03:29 2015 8 151.191.80.226 3341520 /sbclocal/InternalSecureFileTransfer/users/ETDIT/tpt_ftp_test/outbox/rexuat/ldn/GMI_OBS_IRSCME_POS_20150716.txt **a n i r** tpt_ftp_test ftp 0 *

Fri Jul 17 11:03:29 2015 1 rex_ldn_uat1_cmp.ldn.swissbank.com 3341520 /sbclocal/InternalSecureFileTransfer/users/ETDIT/tpt_ftp_test/outbox/rexuat/ldn/GMI_OBS_IRSCME_POS_20150716.txt **a s o r** tpt_ftp_test ssh 0 *

It allows me to extract the combinations as individual characters into a field until I add I add the character "b". Also I was wondering if there is a way to extract different combinations(as shown in examples) and the single characters of the combos all into one field? I have a regex that pulls all combinations now. I just wanted to combine them.

 index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?P<FIELDNAME>\w+\s+\w+\s+\w+\s+\w+)\s+\w+" | dedup FIELDNAME | table FIELDNAME
0 Karma

Motivator

Your regex indicates that there may be a pattern like ".stuff **a**" We really need to see the raw data and the desired result before we can understand the best regex for it. I am thinking that the following may extract you results much better:

index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\*{2}(?<FIELDNAME>[inbporaj])\*{2}" | dedup FIELDNAME | table FIELDNAME
0 Karma

Esteemed Legend

It looks like your character class is inefficient and you are not escaping your asterisks; try this:

index=gfs_etd_mft  source="/opt/TMWD/SecureTransport/var/logs/xferlog" | rex field=_raw "(?i)\..*? (?<FIELDNAME>\*\*[inbporaj])\*\*" | dedup FIELDNAME | table FIELDNAME
0 Karma

SplunkTrust
SplunkTrust

can you provide some log samples?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!