topic Re: Replacing part of string when it's equal to a field value in Splunk Search

Replacing part of string when it's equal to a field value

mszopa — Wed, 24 May 2017 14:42:30 GMT

Hi!
I have fields myfield and name which contains text of an email going like this:

Example1:
myfield="From: Smith, John Sent: Thursday, April 13, 2017 9:38 AM To: xxxxxxxxx xxxxxx Subject: yyyyyyyy yyyy yyy yy yyyyy yyyy yyyyyy yy yyyyy yyy yyyyyyy yyyy y yyyyyyyy yy yyyy yyyyyyyyy yy yyyy yyyy yyyyyyyyyy yyyyy yyy yyyyyyy yyyyyyyyyyyyy yyyy yyy yyyy yy yyyyyyy yyyyyyyyyy yy yyyyyyy John xxxxxx xxxxxxx xxxxxxx xxxxxxxxx xxxxxxxxx xxxxxxx xxxxxx xxxxx xxxxxxxx xxxx xx"
name="John"

Example2:
"From: Sue, Peggy Sent: Tuesday, April 18, 2017 4:48 PM To: xxxx xxxxx Subject: yyyyyyyyy yyyyyyyyy yyyyyyyyyyyyy yyyyyy yyyyyyyyy yyyyyyyyyyy yyyyyyyyyy yyyyyyyyyy yyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyyyyyy yyy yyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyyyy yyyy yyyy y yyyy yyyy yyyyyyyy Peggy xxxx xxxxxxxx xxxx xxxxxx xx xxxxxx xxxxxxxxx xxxxxxx xxxxxxxx xxx"
name="Peggy"

so name is different in each email but always appears. I want to extract all the Ys. I tried using rex, but since name is not a static text but changes for each event, i didn't work. There are no specific characters that I could try using, the only constant thing is that name appears. Any ideas how I could do that?

Thanks a lot!

Re: Replacing part of string when it's equal to a field value

gcusello — Wed, 24 May 2017 15:33:55 GMT

Hi mszopa,
could you share an example of your logs?
Bye.
Giuseppe

Re: Replacing part of string when it's equal to a field value

niketn — Wed, 24 May 2017 15:52:41 GMT

@mszopa... it would be better if you can mock some sample events. However, in the existing sample that you have provided if sometext2 always exists after Subject: and before an email address you can use email address regular expression as your end pattern i.e.

Subject:\s(?<sometext2>.*)\s([\w-\.]+)@((?:[\w]+\.)+)([a-zA-Z]{2,4})

Please add sample events if you can, I have used following dummy data (Assuming single space between Subject: and sometext2 and sometext2 and Sender as per your Question):

 Subject: My Test Email to abc123@mymail.com

Re: Replacing part of string when it's equal to a field value

mszopa — Wed, 24 May 2017 16:02:56 GMT

Sorry, i could have been more specific from the start!

It's something like that:
""From: Smith, John Sent: Monday, May 08, 2017 3:11 PM To: Receiver's_email@address Subject: Something's not working Dear all, could you please help me with this issue? [problem description]
Thanks John Smith [workplace, etc]"

So the "Subject:" and "Name Last Name" always appear but the name changes. While getting regular expression starting from Subject is simple, I don't know how to remove everything after the name.
I also have a field called "sender" which here would be sender="John Smith" so I thought about using it.

All the best,
Magda

Re: Replacing part of string when it's equal to a field value

woodcock — Wed, 24 May 2017 16:13:24 GMT

Like this:

| rex "Subject:\s+(?<Subject>.*?)\s+Sender:?\s+[^@]+@"

Re: Replacing part of string when it's equal to a field value

akocak — Wed, 24 May 2017 20:40:54 GMT

@mszopa, it is best you provide 2 things, 2 raw events, and part of the text you want to extract on each. I red question and comments twice and still not clear what you are looking for.
Is this a multi line regex or single line?
what changes clearly in your event, does problem description always starts with "[" or in the next line.
if you are trying to be careful on information to share, replace messages with xxxxx and yyyy s to make clear. you can even replace names as micky mouse etc..
Also, which name, the one starts with lastname, firstname or the one at the end of the event.
If you explain it well, it appears me that your solution resides next to regex lookahead lookbehind, again need a clear info. to provide you something, There is a way you can use your known! sender name in Regex as well.

Re: Replacing part of string when it's equal to a field value

mszopa — Thu, 25 May 2017 08:42:19 GMT

Thank you all for quick replies! Ok, this is it (I have to replace letters with x)"

myfield="From: Smith, John xxxx Sent: Thursday, April 13, 2017 9:38 AM To: xxxxxxx, xxxxxxxxxxxxx xxxxxxxxxxx Subject: yyyyyyyyyyyyyyyyyyyy yyyyyyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyyyy yyyyyyyy yyyyyyyy yyyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyy yy yyy yyyyyyy yy yyyyyyyy yyy yyyyy yyy yyyyyyyy xxxx xx xxxx John xxxxxxxxxx xxxxxxxx xxxxxxxx xxxx"

name="John"

Value of name changes but is always included in the text. I need everything between words "Subject:" and name (here being "John").
I really hope I made it clear this time.

Re: Replacing part of string when it's equal to a field value

mszopa — Thu, 25 May 2017 08:44:52 GMT

Sadly, there is no "@" character, I've only got the name. I put an example in a comment above.

Re: Replacing part of string when it's equal to a field value

woodcock — Thu, 25 May 2017 20:19:42 GMT

Like this:

| makeresults 
| eval myfield="From: Smith, John Sent: Thursday, April 13, 2017 9:38 AM To: xxxxxxxxx xxxxxx Subject: yyyyyyyy yyyy yyy yy yyyyy yyyy yyyyyy yy yyyyy yyy yyyyyyy yyyy y yyyyyyyy yy yyyy yyyyyyyyy yy yyyy yyyy yyyyyyyyyy yyyyy yyy yyyyyyy yyyyyyyyyyyyy yyyy yyy yyyy yy yyyyyyy yyyyyyyyyy yy yyyyyyy John xxxxxx xxxxxxx xxxxxxx xxxxxxxxx xxxxxxxxx xxxxxxx xxxxxx xxxxx xxxxxxxx xxxx xx
From: Sue, Peggy Sent: Tuesday, April 18, 2017 4:48 PM To: xxxx xxxxx Subject: yyyyyyyyy yyyyyyyyy yyyyyyyyyyyyy yyyyyy yyyyyyyyy yyyyyyyyyyy yyyyyyyyyy yyyyyyyyyy yyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyyyyyy yyy yyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyyyy yyyyyyyyyyyy yyyyyyyyyyy yyyyyyyyyyyyy yyyy yyyy y yyyy yyyy yyyyyyyy Peggy xxxx xxxxxxxx xxxx xxxxxx xx xxxxxx xxxxxxxxx xxxxxxx xxxxxxxx xxx"
| makemv delim="
" myfield
| mvexpand myfield

| rename COMMENT AS "Everything above creates sample data; everything below is your solution"

| rex field=myfield "From:\s*(?:[^,]+,\s*)?(?<name>[^:]+)\s+Sent:\s+"
| eval myportion=split(myfield, name)
| eval myportion=mvindex(myportion, 1)
| rex field=myportion mode=sed "s/^.*? Subject: //"

Re: Replacing part of string when it's equal to a field value

mszopa — Fri, 26 May 2017 17:30:59 GMT

So simple, now that you wrote it!
Thank you very much for your help!