Splunk Search

Extract multiple values from a single existing field, using regex

hketer
Path Finder

Hey,
I have this event.
as you can see there is field named cs1.
I need to create new field lets say cs_1 and extract the values

  • FTP - using unsecured file transfer
  • FTP plaintext user & password

output for cs_1 will be

                 cs_1

FTP - using unsecured file transfer

FTP plaintext user & password

alt text

please assist:)

Tags (2)
0 Karma

woodcock
Esteemed Legend

Like this:

| makeresults 
| eval raw="cs1=['5050447193422847;;FTP - using unsecured file transfer;;;;;;', '2742510939747842;;FTP plaintext user & password;;;;;;']:::cs1=['4163302463635179;;SMB2 login failure;;;;;;']:::cs1=['4277463150217252;;Stop;;;;;;']:::cs1=['5050447193422847;;FTP - using unsecured file transfer;;;;;;']"
| makemv delim=":::" raw
| mvexpand raw
| rename raw AS _raw
| kv
| table cs1

| rename COMMENT AS "Everything above generates sample event data; everything below is your solution"

| rex field=cs1 mode=sed "s/^\[// s/\]$//"
| makemv delim="', '" cs1
| rex field=cs1 mode=sed "s/^'// s/'$//"
| rex field=cs1 "^(?:[^;]*;){2}(?<cs_1>[^;]+)"

hketer
Path Finder

Hi! 🙂
Thank you for your help! I appreciate it!!

I'll explain myself again

I have 4 different events that contain the field:

cs1=['5050447193422847;;FTP - using unsecured file transfer;;;;;;', '2742510939747842;;FTP plaintext user & password;;;;;;']
cs1=['4163302463635179;;SMB2 login failure;;;;;;']
cs1=['4277463150217252;;Stop;;;;;;']
cs1=['5050447193422847;;FTP - using unsecured file transfer;;;;;;']

as you can see in one of the events I have 2 FTP messages in the same field.
I need to extract (not in search) the values to a new field, for example, cs_1
the field now will contain 4 different values

cs1=FTP - using unsecured file transfer
cs1=FTP plaintext user & password
cs1=SMB2 login failure
cs1=Stop

or

cs1=FTP - using unsecured file transfer
cs1=FTP - using unsecured file transfer , FTP plaintext user & password
cs1=SMB2 login failure
cs1=Stop

haven't decided yet but if someone can help me with one of them it would be amazing!!!! 🙂
Thank YOU!

0 Karma

woodcock
Esteemed Legend

Like this:

.... | rex max_match=0 field=my_raw ";(?<cs_1>FTP[^;]+)" 
| table cs_1
| mvexpand cs_1
0 Karma

damann
Communicator

What about using max_match=0 in your rex ?

|makeresults
|eval my_raw="04-29-2019 11:03:53 Local7.Debug xx CEF:0 [somehost] 6.5.7|a12345|black plicy|1|app=ftp cs1=['50512345;;FTP - using unsecured file transfer;;;;;;','272510;;FTP plaintext user & password;;;;;;'] cs1Label=Reason ]"
 | rex max_match=0 field=my_raw ";(?<cs_1>FTP[^;]+)"
 | table cs_1

This gives me a nice multivalue Field for cs_1

hketer
Path Finder

Hi,
Thanks for the answer!
I've tried it and yet its not the result I'm looking for.

the 2 FTP lines are in the same cell and I need it to be separate

0 Karma

DavidHourani
Super Champion

Hi @hketer,

Extracting a field from another can be done using props and transforms and shown here : https://answers.splunk.com/answers/493083/how-to-transformregex-on-already-extracted-field-w.html

If you don't want to go through this hassle you can use a calculated field which is easier to setup as shown in this answer here : https://answers.splunk.com/answers/177878/how-to-extract-a-field-from-within-the-value-of-an.html

Let me know which part of cs1 you want to extract and I can help you with the regexp if needed.

Cheers,
David

hketer
Path Finder

I'm trying to extract like you sent in the first link.
Thanks I'll try it again

0 Karma

DavidHourani
Super Champion

Awesome ! Let me know if you hit a stonewall with the config files, happy to help out and feel free to accept the answer if it helped !
Cheers

0 Karma

koshyk
Super Champion

Please try

|makeresults
|eval my_raw="04-29-2019 11:03:53 Local7.Debug xx CEF:0 [somehost] 6.5.7|a12345|black plicy|1|app=ftp cs1=['50512345;;FTP - using unsecured file transfer;;;;;;','272510;;FTP plaintext user & password;;;;;;'] cs1Label=Reason ]"
| rex field=my_raw "\[\'.+?\;\;(?<cs_1_ftpA>FTP[^\;]+).+?(?<cs_1_ftpB>FTP[^\;]+)"

Regex101 demo => https://regex101.com/r/azZv9k/1/

Also if you need to club the messages into a single Key

|makeresults
|eval my_raw="04-29-2019 11:03:53 Local7.Debug xx CEF:0 [somehost] 6.5.7|a12345|black plicy|1|app=ftp cs1=['50512345;;FTP - using unsecured file transfer;;;;;;','272510;;FTP plaintext user & password;;;;;;'] cs1Label=Reason ]"
| rex field=my_raw "\[\'.+?\;\;(?<cs_1_ftpA>FTP[^\;]+).+?(?<cs_1_ftpB>FTP[^\;]+)"
| eval cs1 = cs_1_ftpA + ";" + cs_1_ftpB |makemv delim=";" cs1| table cs1

cheers

hketer
Path Finder

Thank you! I appreciate it!!

I want it to be separate values in the same field, and right now both of them is 1 value

cs1

FTP - using unsecured file transfer

FTP plaintext user & password

FTP - using unsecured file transfer
FTP plaintext user & password

0 Karma

koshyk
Super Champion

@hketer, how you want it to be separate values? Comma limited? you can just play around with

 | eval cs1 = cs_1_ftpA + ";" + cs_1_ftpB 

to fit your purpose

0 Karma

koshyk
Super Champion

any chance you can paste the sample data (rather than image)

0 Karma

hketer
Path Finder

04-29-2019 11:03:53 Local7.Debug CEF:0||6.5.7|A12755045326585428785|black policy|1|app=ftp cnt=2 cs1=['5050447193422847;;FTP - using unsecured file transfer;;;;;;', '2742510939747842;;FTP plaintext user & password;;;;;;'] cs1Label=reason cs2=['server_to_client', 'client_to_server'] cs2Label=direction deviceExternalId=Sensor dmac= dpt=3592 dst= end=1556525018000 proto=TCP smac= spt=21 src= start=1556525018000

0 Karma
Get Updates on the Splunk Community!

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...