Splunk Search

New Field REX HELP (API)

havatz
Explorer

Hi all

I want to create a new AWS monitoring alert.

For the first step I checked the AWS fields and I saw that I need to parse the "principalId" field from the "_raw". (I want to create the rule based on principalID"

Part of the raw:

{\"type\": \"Root\", \"principalId\": \"444444444444\", \"arn\"

havatz_2-1601280189138.png

 

Im running this query:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\W\W:\s\W\W(?P<principalId>\d*)"

and getting results but without the "prinicpialId" new field.

havatz_0-1601279883568.png

What am I missing in the query?

Thanks!

 

Labels (1)
0 Karma
1 Solution

ITWhisperer
SplunkTrust
SplunkTrust

Have you looked at the data in the splunk GUI? Are there really backslashes before the double quotes in the data?

Try this:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W+:\\s\\W+(?P<principalId>\\d*)"

View solution in original post

0 Karma

havatz
Explorer

this it the raw data:

 

"_raw": "{\"eventVersion\": \"1.05\", \"userIdentity\": {\"type\": \"Root\", \"principalId\": \"11111111111\", \"arn\": \"arn:aws:iam::11111111111:root\", \"accountId\": \"11111111111\", \"accessKeyId\": \"\"}, \"eventTime\": \"2020-09-25T17:49:45Z\", \"eventSource\": \"signin.amazonaws.com\", \"eventName\": \"ConsoleLogin\", \"awsRegion\": \"us-east-1\", \"sourceIPAddress\": \"xxxxxxxxx\", \"userAgent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Your second screenshot doesn't have any \ before the " although your test data (first screenshot) does. What is actually in your data? You could try

search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\W+:\s\W+(?P<principalId>\d*)
0 Karma

havatz
Explorer

Getting this error:

havatz_0-1601281598998.png

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You are using curl and ReST API? Try escaping the backslashes

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W\\W:\\s\\W\\W(?P<principalId>\\d*)"

 

0 Karma

havatz
Explorer

The query should enter retro active on old events also? or should I wait for new ones?

0 Karma

havatz
Explorer

Yes.. its API .

This query  working great but im still getting the "principalId" empty

havatz_0-1601282525960.png

havatz_1-1601282651943.png

 

 

 

 

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Have you looked at the data in the splunk GUI? Are there really backslashes before the double quotes in the data?

Try this:

"search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-1d | rex field=_raw principalId\\W+:\\s\\W+(?P<principalId>\\d*)"
0 Karma

havatz
Explorer

You're right! My mistake
It working!! Thank you!

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @havatz,

Are you sure that in your logs there's the back slash before quotes?

if not, please, try this regex:

| rex "\"principalId\":\s+\"(?<principalId>[^\"]+)\""

that you can test at https://regex101.com/r/iS9NFl/1

Otherwise, please try this regex:

| rex "principalId\\\":\s+\\\"(?<principalId>[^\"]+)"

that you can test at https://regex101.com/r/iS9NFl/2

Ciao.

Giuseppe

 

0 Karma

havatz
Explorer

curl -k -u "user pass" https://localhost:11111/services/search/jobs?output_mode=json -X POST -d search="search index=aws userIdentity.type=Root eventName=ConsoleLogin earliest=-30d | rex field=_raw "principalId\\\":\s+\\\"(?<principalId>[^\"]+)"


{"messages":[{"type":"ERROR","text":"Error in 'SearchParser': Missing a search command before '^'. Error at position '129' of search query 'search index=aws userIdentity.type=Root eventName=...{snipped} {errorcontext = ncipalId>[^\"] )}'.","help":""}]}

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...