Selecting the field value in source using regex , than i have to compare the regex output with lookup table sso id ,

[siraj198204]

This is the Search ,
sourcetype=smtrace "supportcentral”

This is the search output ,

If you look at that search you will entry like this:
[10/14/2014][06:53:55.633][340724656][s418550/r4011][Supportcentral Internal][][][][][][206445568][][][][][][supportcentralalpcispweb535vprd][** Status: Authorized. ][]
206445568 , this is sso account coming at 11th field value ......
1. I want to select the the 11 field value , using regex command
2. After that i want to compare the regex output with |lookup identity_lookup this lookup table
the lookuptable is having the sso id ...
3.This regex output i want to compare with the lookup table .
4.The goal of this query to identify non sso accounts that are using the support central app

[aholzer · Accepted solution]

1) try the below. This should capture the value inside the 11th square brackets:

... | rex "(\[[^\]]*\]){10}\[(?P<sso_id>[^\]]*)\]"

2) Make sure the key in your lookup doesn't have spaces and you can use the below:

... | lookup identity_lookup <key in your file> AS sso_id

This will append any extra columns in your lookup to matching sso_ids from your search.

4) You can then look for any events that have nulls in the column from your lookup to get any sso_ids that don't have a match in your lookup table.

---- EDIT ----
My regex had an extra close parenthesis at the end ")". Sorry about the typo.

[siraj198204]

The above regex is working good , i am interested to learn regex ... do u have any tutorial or any material with example ... to learn regex ... completely ... can you help me on this .... ??

Thank u ...

Regards,
Siraj

[aholzer]

@siraj198204

You can try this tutorial to learn the basics: http://regexone.com/

When I'm confirming something or checking if something is possible I tend to refer to this one, but I think it's a little more advanced: http://www.regular-expressions.info/tutorial.html

And to practice I suggest you go to: https://regex.alf.nu/

Also if you found my answer helpful please mark it as the answer. If you found any comments helpful please upvote them too. Thanks.

[siraj198204]

Hi ,

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral | rex "([[^]]]){10}[(?P[^]])]" | lookup identity_lookup sso

When comparing this two , i need which ever id is not present in identity_lookup sso field ....

when comparing this lookup table and rex output .... i need the result ... which ever id is not present in lookup table when compare with regex ....

[ppablo]

Hi @siraj198204

For future reference, please be sure to click on "Add comment" under the actual posted answer or "Reply" on a comment if you're responding to another user. Right now, you've been posting a new answer (that aren't answers) each time through the "Your Answer" space at the bottom of the page. I've converted all 10+ of your answers to comments appropriately. This will help with other users seeing the proper flow of the conversation. Thanks!

[siraj198204]

Hi ,

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral | rex "([[^]]]){10}[(?P[^]])]" | lookup identity_lookup sso AS SSOID

I think , this is correct .... but something ... wrong syntax with lookup ....

| lookup identity_lookup sso AS SSOID ..... may be this syntax is wrong ...

Thank u ...

[siraj198204]

Hi,

›

10/16/14
10:25:06.241 AM
one example

[10/16/2014][07:25:06.241][915344304][eb940202-0344-543fab12-104c-01ab7c98][Supportcentral Internal][][][][][][206445199][][][][][][supportcentralalpcispweb536vprd][Authorizing user...][]
SSOID =206445199
host =useclpapl894.nbcuni.ge.com
index =casm_prod
source =/opt/siteminder/log/smtracedefault.log
sourcetype =smtrace

it is the SSO account ....

[siraj198204]

Hi ,

one example ,

[10/14/2014][17:42:11.066][329190320][][Supportcentral Internal][][][][][SC iNBCU Auth Scheme][604200523@nbcu][][][Local LDAP][][][supportcentralalpcispweb536vprd][** Status: Authentication Attempt Failed ...

This is user account , but it is the non - sso accounts ....

11th field is [604200523@nbcu] .... i want to search this type of account and can compare with lookup table ...

one example ,

[10/16/2014][07:24:42.532][915344304][s938551/r616][Supportcentral External][][][][][][][][][][][][supportcentralalpcispweb546vprd][Send response attribute 204, data size is 39][06-00019045-de22-1f5a-8ab9-e7a4039c3013]
SSOID =
host =useclpapl894.nbcuni.ge.com
index =casm_prod
source =/opt/siteminder/log/smtracedefault.log
sourcetype =smtrace

SSOID= ..... [] no value , but is a user id account ... it is using applications ... but there is some id is there for this account , i want to found this account id and compare with lookup table ...

Thank u ... 🙂

[siraj198204]

Hi ,

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral | rex "([[^]]]){10}[(?P[^]])]" | where len(sso_id)=0 , this command is finding ... 11th value is [] .... but inside this [] some non sso accounts id is there .... that i have to compare with lookup table ....

Thank u .. 🙂

[siraj198204]

Hi ,

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral | rex "([[^]]]){10}[(?P[^]])]" | where len(sso_id)=0 ,

This command is working good ,

›

10/16/14
9:22:29.850 AM

[10/16/2014][06:22:29.850][709823408][s932285/r16][Supportcentral Internal][][][][][][][][][][][][supportcentralalpcispweb538vprd][** Status: Protected. ][]
host =useclpapl894.nbcuni.ge.com
index =casm_prod
source =/opt/siteminder/log/smtracedefault.log
sourcetype =smtrace
sso_id =

[10/16/2014][06:22:29.850][709823408][s932285/r16][Supportcentral Internal][][][][][][][][][][][][supportcentralalpcispweb538vprd][** Status: Protected. ][] ---- in this the 11th value is [] , but inside this contains some vale .... may be it is 0001 or 000002 like this any value is there , some other non sso account user id is there .... that value i have to compare with lookup table ...

some user ID is there in this [] .... i have to match this value with lookup table ...

[siraj198204]

Hi ,

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral | rex "([[^]]]){10}[(?P[^]])]" | where len(sso_id)=0

This is working good ,

sso_id = [] ...

only finding this output ...

Thank u ... 🙂

[aholzer]

You could try instead:

index =casm_prod source =/opt/siteminder/log/smtracedefault.log sourcetype=smtrace supportcentral |rex ""(\[[^\]]*\]){10}\[(?P<sso_id>[^\]]*)\]" | where len(sso_id)=0

[siraj198204]

Hi,

sourcetype=smtrace supportcentral |rex "([[^]]]){10}[(?P[^]])]" | where isnull(sso_id)

I am not getting any output ,

[10/15/2014][08:06:46.606][548342704][][][SupportCentral allow access][][][][][][][][][][][][Check the Rule][]

index =casm_prod
source =/opt/siteminder/log/smtracedefault.log
sourcetype =smtrace
sso_id =

the 11 field is empty [] , this means non sso accounts , but it is the user account ...

0 of 2,345,208 events matched ... zero events are matching ... but if we give ,

sourcetype=smtrace supportcentral |rex "([[^]]]){10}[(?P[^]])]"

we are getting all events ...

Thank u ... 🙂

[aholzer]

Run the following to get those events with null sso_id:

sourcetype=smtrace supportcentral |rex ""(\[[^\]]*\]){10}\[(?P<sso_id>[^\]]*)\]" | where isnull(sso_id)

[siraj198204]

Hi ,

This regex is working good.

sourcetype=smtrace supportcentral |rex "([[^]]]){10}[(?P[^]])]" it is working good ,

it is giving all the sso_id,

i want to find only this below type of output ,

[10/15/2014][08:06:46.606][548342704][][][SupportCentral allow access][][][][][][][][][][][][Check the Rule][]

index =casm_prod
source =/opt/siteminder/log/smtracedefault.log
sourcetype =smtrace
sso_id =

in this 11 field is empty .... [] ... there is no sso id .... i want to find only this output....

these output are called non sso accounts ...

Thank u ... 🙂

[aholzer]

I added a comment to my original answer. Hope that helps.

[siraj198204]

Hi ,

sourcetype=smtrace supportcentral |rex "([[^]]]){10}[(?P[^]])]"

it is working good ...

now i am trying this with to match the lookup table ...

Thank u ... 🙂 ...

[siraj198204]

Hi ,

iam trying with the regex ? but getting the above mentioned error ...

Thank u .. 🙂

[aholzer]

I made another minor change, but if you are still getting the "unmatched parentheses" error then it won't help with that. Are you sure you copied the new version I posted?

[siraj198204]

Hi,

i tried this ,
sourcetype=smtrace "supportcentral”  |rex field=sourcetype "([[^]]+]){10}[(?P[^]]+)])" | lookup identity_lookup sso AS sso_id

But getting the below error ...

Error in 'rex' command: Encountered the following error while compiling the regex '([[^]]+]){10}[(?P[^]]+)])': Regex: unmatched parentheses

Thank u ...