Splunk Search

New Field REX HELP (API)

havatz
Explorer

Hi all

I want to create a new AWS monitoring alert.

For the first step I checked the AWS fields and I saw that I need to parse the "principalId" field from the "_raw". (I want to create the rule based on principalID"

Part of the raw:

{\"type\": \"Root\", \"principalId\": \"444444444444\", \"arn\"

havatz_2-1601280189138.png

 

Im running this query:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\W\W:\s\W\W(?P<principalId>\d*)"

and getting results but without the "prinicpialId" new field.

havatz_0-1601279883568.png

What am I missing in the query?

Thanks!

 

Labels (1)
0 Karma
1 Solution

ITWhisperer
Legend

Have you looked at the data in the splunk GUI? Are there really backslashes before the double quotes in the data?

Try this:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W+:\\s\\W+(?P<principalId>\\d*)"

View solution in original post

0 Karma

havatz
Explorer

this it the raw data:

 

"_raw": "{\"eventVersion\": \"1.05\", \"userIdentity\": {\"type\": \"Root\", \"principalId\": \"11111111111\", \"arn\": \"arn:aws:iam::11111111111:root\", \"accountId\": \"11111111111\", \"accessKeyId\": \"\"}, \"eventTime\": \"2020-09-25T17:49:45Z\", \"eventSource\": \"signin.amazonaws.com\", \"eventName\": \"ConsoleLogin\", \"awsRegion\": \"us-east-1\", \"sourceIPAddress\": \"xxxxxxxxx\", \"userAgent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)

0 Karma

ITWhisperer
Legend

Your second screenshot doesn't have any \ before the " although your test data (first screenshot) does. What is actually in your data? You could try

search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\W+:\s\W+(?P<principalId>\d*)
0 Karma

havatz
Explorer

Getting this error:

havatz_0-1601281598998.png

 

0 Karma

ITWhisperer
Legend

You are using curl and ReST API? Try escaping the backslashes

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W\\W:\\s\\W\\W(?P<principalId>\\d*)"

 

0 Karma

havatz
Explorer

The query should enter retro active on old events also? or should I wait for new ones?

0 Karma

havatz
Explorer

Yes.. its API .

This query  working great but im still getting the "principalId" empty

havatz_0-1601282525960.png

havatz_1-1601282651943.png

 

 

 

 

0 Karma

ITWhisperer
Legend

Have you looked at the data in the splunk GUI? Are there really backslashes before the double quotes in the data?

Try this:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W+:\\s\\W+(?P<principalId>\\d*)"

View solution in original post

0 Karma

havatz
Explorer

You're right! My mistake
It working!! Thank you!

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @havatz,

Are you sure that in your logs there's the back slash before quotes?

if not, please, try this regex:

| rex "\"principalId\":\s+\"(?<principalId>[^\"]+)\""

that you can test at https://regex101.com/r/iS9NFl/1

Otherwise, please try this regex:

| rex "principalId\\\":\s+\\\"(?<principalId>[^\"]+)"

that you can test at https://regex101.com/r/iS9NFl/2

Ciao.

Giuseppe

 

0 Karma

havatz
Explorer

curl -k -u "user pass" https://localhost:11111/services/search/jobs?output_mode=json -X POST -d search="search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-30d | rex field=_raw "principalId\\\":\s+\\\"(?<principalId>[^\"]+)"


{"messages":[{"type":"ERROR","text":"Error in 'SearchParser': Missing a search command before '^'. Error at position '129' of search query 'search index=aws userIdentity.type=Root eventName=...{snipped} {errorcontext = ncipalId>[^\"] )}'.","help":""}]}

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!