topic Re: How to edit my "rex" search in order to extract the User ID in my sample data? in Splunk Search

How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Thu, 12 Jan 2017 21:32:57 GMT

Another regular expression/rex field extraction question: How do I get USERID between timestamp and '@JavaClient' ?

Does USERID always have to begin with (?< and end with >.*) ?
What syntax represents all of these to use rex command ?
- number, integer, and digit
- date time (specific format? )
- letter
  - special characters

Currently, I have it like this but it's not returning XBATCH, and ABCSRV

| rex field=_raw "\.\d\d\d\s(?.*)\@JavaClient\s"

Raw data

PSAPPSRV.3204 (770) [2017-01-12T15:00:32.028 XBATCH@JavaClient IntegrationSvc](3) Returning context. ID=XBATCH, Lang=ENG, UStreamId=150032028_3204.770, Token=DU_CS/2017-01-12-09.00.31.708254/XBATCH/ENG/B82gf3Amj/f5Lx/k1J4f6t9RRtI=

PSAPPSRV.3204 (771) [2017-01-12T15:01:02.551 GetCertificate](3) Returning context. ID=SRBTTRAN, Lang=ENG, UStreamId=150102551_3204.771, Token=DU_CS/2017-01-12-09.01.02.708281/SRBTTRAN/ENG/o0fY1oj0zd5Es1vl3xMOnUaB5t0=

PSAPPSRV.3204 (785) [2017-01-12T15:01:05.037 ABCSRV@JavaClient IntegrationSvc](3) Processing WssecIsAuthenticatedUser command.  CtxUser=ABCSRV/ENG, SwUser=XBATCH/+

PSAPPSRV.3204 (785) [2017-01-12T15:01:05.037 ABCSRV@JavaClient IntegrationSvc](3) Processing SwitchUser command.  CtxUser=ABCSRV/ENG, SwUser=DPRBATCH/+, SwConnInfo=-, SwToken=-

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Thu, 12 Jan 2017 21:59:03 GMT

Your regex is correct in matching XBATCH and ABCSRV however for Splunk's rex command you will need to give your extraction a name. That is what the <? ... > in the capture group is for.

Try doing this

\.\d\d\d\s(?<USERID>.*)\@JavaClient\s

As for regex syntax here is a cheat sheet I regularly reference when writing regex. https://www.cheatography.com/davechild/cheat-sheets/regular-expressions/

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 15:54:52 GMT

Thanks, sorry, I missed that tag.

| rex field=_raw ".\d\d\d\s(?.*)\@JavaClient\s"

I get the result but some of USERIDs still show up in as "(785) [2017-01-12T15:01:05.037 ABCSRV"

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:37:39 GMT

Your regex seems to have changed. The one you posted originally seems like it should work.
Use this regex
\.\d\d\d\s(?<userid>.*)\@JavaClient\s
instead of
.\d\d\d\s(?<userid>.*)\@JavaClient\s

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 17:43:17 GMT

same result...not working either way 😞

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 17:44:52 GMT

was it @ sign that's the issue? maybe it needs special escape character that is not \

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:45:00 GMT

Are some events correctly pulling the user id out? Is it just some events that pull the numbers before?

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:46:14 GMT

The @ symbol is not a reserved character for regex so you shouldn't need to escape it.

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 17:47:16 GMT

Yes i got 2 results coming out correctly but a lot still in format "(785) [2017-01-12T15:01:05.037 ABCSRV"

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:48:49 GMT

Can you post some samples of events that didn't get it correctly pulled? That way I can put those into my regex tester and tweak the regex.

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:50:25 GMT

Also can you show me exactly the rex command that you are using?

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 17:50:37 GMT

| rex field=_raw ".\d\d\d\s(?.*)@JavaClient\s"

still not working

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 17:59:11 GMT

So it seems to work here for all the examples you provided. Now if you put exactly what I have here in the quotes of the rex command and it still isn't correctly pulling. Then pick out some events that aren't correctly pulling it and add them so I can see whats wrong with the regex.

This is the tool I use to test my regex out. https://regex101.com/

EDIT: Note the text highlighted in green is what regex is saying is getting captured by the capture group. And that group is given the name USERID.

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Tue, 29 Sep 2020 12:22:48 GMT

the link is very useful. Thank you!

These are data that don't work out. I can't upload screenshot because i don't have karma points 😞

PSAPPSRV.152 (132) 2017-01-13T10:28:59.614 DPRBATCH@JavaClient IntegrationSvc Returning context. ID=DPRBATCH, Lang=ENG, UStreamId=102859614_152.132, Token=DU_CS/2017-01-13-04.28.59.519666/DPRBATCH/ENG/jOYUAqpyw2rafErYxid/+2tm8gY=
PSAPPSRV.152 (132) 2017-01-13T10:28:59.614 DPRBATCH@JavaClient IntegrationSvc Returning SwitchUser variables. ID=m6c8RnSP6uzj1TlfV3Hmk7iOO7CrHSvcuI47sKsdK9y4jjuwqx0r3LiOO7CrHSvcuI47sKsdK9y4jjuwqx0r3A==, Pswd=+, ConnInfo=-, Lang=-, Token=-
PSAPPSRV.592 (9389) 2017-01-13T11:16:59.969 SAMSGSRV@JavaClient IntegrationSvc Processing SwitchUser command. CtxUser=SAMSGSRV/ENG, SwUser=DPRBATCH/+, SwConnInfo=-, SwToken=-
PSAPPSRV.592 (9389) 2017-01-13T11:16:59.969 SAMSGSRV@JavaClient IntegrationSvc Processing WssecIsAuthenticatedUser command. CtxUser=SAMSGSRV/ENG, SwUser=DPRBATCH/+

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 18:14:26 GMT

I see the problem there is a period followed by three digits earlier on those events... Here is a regex that matches the whole timestamp try this out.

\[\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}\s(?<userid>.*)\@JavaClient\s

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 18:18:06 GMT

yep! it works thank you!! But i also try with shorter one \d{2}.\d{3}\s(?.*)@JavaClient\s

that works too. which event has "a period followed by three digits" i only see a period before three digits.

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

kmccririe_splun — Fri, 13 Jan 2017 18:49:38 GMT

That is what I meant. The extra events you sent me had PSAPPSRV.XXX in the beginning that was matching and messing it up.

Re: How to edit my "rex" search in order to extract the User ID in my sample data?

pwongcha — Fri, 13 Jan 2017 19:46:40 GMT

I see. Thanks for pointing this out to me. I didn't know it would go all the way from the beginning of the string at the beginning of the line.