Splunk Search

Conditional regex help: How to capture two groups if they have an "exclusion type"?

sdee1013
Loves-to-Learn

hi everyone,

i'm trying to parse json inline.  i'm using kv mode= json already but i'm trying to achieve selective groups.

essentially i want to capture two groups if they have an "exclusion type"

sample json.

[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SizeRestrictions_BODY"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}]

so for this i wanted to capture only the ruleGroupId name if it has excludedRules not null, then capture the exclusionType

 

any help would be appreciated.

 

 

Labels (1)
0 Karma

sdee1013
Loves-to-Learn

that looks good but i'm trying to create an inline extraction (props.conf) for this. so it only returns that info

0 Karma

somesoni2
Revered Legend
0 Karma

sdee1013
Loves-to-Learn

works when its parsed but not in raw .  new lines aren't valid.    here it is in raw: tried to work around it but my regex is horrible...lol

 

{"timestamp":1646240486931,"formatVersion":1,"webaclId":"arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-prod-web-acl/24e4f178-f008-434a-80f4-cd16728b9ffd","terminatingRuleId":"Default_Action","terminatingRuleType":"REGULAR","action":"ALLOW","terminatingRuleMatchDetails":[],"httpSourceName":"ALB","httpSourceId":"111111-app/site-internal-alb-production/07fae64dff77a3b3","ruleGroupList":[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"CategorySocialMedia"},{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SignalNonBrowserUserAgent"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}],"rateBasedRuleList":[],"nonTerminatingMatchingRules":[],"requestHeadersInserted":null,"responseCodeSent":null,"httpRequest":{"clientIp":"10.10.1.127","country":"-","headers":[{"name":"Accept-Encoding","value":"gzip"},{"name":"User-Agent","value":"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"},{"name":"X-BufferBot","value":"Being Awesome! P.S. We're hiring! buffer.com/journey"},{"name":"cookie","value":"_bit=m21bv9-710b10c6d1aad7e0f7-00o"},{"name":"x-datadog-trace-id","value":"272920074770865622"},{"name":"x-datadog-parent-id","value":"827293848173300227"},{"name":"x-datadog-sampled","value":"1"},{"name":"x-datadog-sampling-priority","value":"0"},{"name":"host","value":"www.site.com"},{"name":"Connection","value":"close"}],"uri":"/ana/training-technical-assistance/using-grantsgov-workspace","args":"","httpVersion":"HTTP/1.1","httpMethod":"GET","requestId":"1-621fa2e6-49f36a5f01f555c90fe7e63e"},"labels":[{"name":"awswaf:managed:aws:bot-control:bot:category:social_media"},{"name":"awswaf:managed:aws:bot-control:bot:name:facebook"},{"name":"awswaf:managed:aws:bot-control:signal:non_browser_user_agent"}]}

 

 

0 Karma

sdee1013
Loves-to-Learn

let me add the whole json...this is actually nested.  

 

{
"timestamp": 1646229254523,
"formatVersion": 1,
"webaclId": "arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-stage-web-acl/26ac170c-03c4-4fd7-8fab-86e346789fef",
"terminatingRuleId": "Default_Action",
"terminatingRuleType": "REGULAR",
"action": "ALLOW",
"terminatingRuleMatchDetails": [],
"httpSourceName": "ALB",
"httpSourceId": "182116744736-app/ALB-Stage/fcc1f5f9483b035e",
"ruleGroupList": [
{
"ruleGroupId": "AWS#AWSManagedRulesAmazonIpReputationList",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesBotControlRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesCommonRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": [
{
"exclusionType": "EXCLUDED_AS_COUNT",
"ruleId": "SizeRestrictions_BODY"
}
]
},
{
"ruleGroupId": "AWS#AWSManagedRulesKnownBadInputsRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
}
],
"rateBasedRuleList": [],
"nonTerminatingMatchingRules": [],
"requestHeadersInserted": null,
"responseCodeSent": null,
"httpRequest": {
"clientIp": "67.218.14.10",
"country": "US",
"headers": [
{
"name": "host",
"value": "sample.com"
},
{
"name": "content-length",
"value": "50362"
},
{
"name": "cache-control",
"value": "max-age=0"
},
{
"name": "sec-ch-ua",
"value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Microsoft Edge\";v=\"98\""
},
{
"name": "sec-ch-ua-mobile",
"value": "?0"
},
{
"name": "sec-ch-ua-platform",
"value": "\"Windows\""
},
{
"name": "origin",
"value": "https://sample.com"
},
{
"name": "upgrade-insecure-requests",
"value": "1"
},
{
"name": "dnt",
"value": "1"
},
{
"name": "content-type",
"value": "multipart/form-data; boundary=----WebKitFormBoundaryuXOFvh7iQjJkEJHm"
},
{
"name": "user-agent",
"value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.62"
},
{
"name": "accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
},
{
"name": "sec-fetch-site",
"value": "same-origin"
},
{
"name": "sec-fetch-mode",
"value": "navigate"
},
{
"name": "sec-fetch-user",
"value": "?1"
},
{
"name": "sec-fetch-dest",
"value": "document"
},
{
"name": "referer",
"value": "https://sample.com/DischargeDetail.aspx"
},
{
"name": "accept-encoding",
"value": "gzip, deflate, br"
},
{
"name": "accept-language",
"value": "en-US,en;q=0.9"
},
{
"name": "cookie",
"value": "_ga=GA1.3.84334902.1642521795; __RequestVerificationToken=-8kinKddCjKCZTws-wPmXDZTFg39urggswPnYm5Y15UwfIjspHqTj1hOPAXIaRPHL2cupyt2vO4Gb5QUExZGd6e5djS0v81kxt2pH22Ow9XiJYr2NPWB_BdQb-VmCUHVXbiVZZ5NwTfGDrXd2O0uD_gba4fM3PhkQUO5f9zs5381; _gid=GA1.2.249665053.1645964709; _ga_33R15ZN4N1=GS1.1.1645965393.6.0.1645965397.56; _ga=GA1.2.84334902.1642521795; ASP.NET_SessionId=1fnikipv2poi14r3doy4kb2w"
}
],
"uri": "/ReleaseRequest.aspx",
"args": "",
"httpVersion": "HTTP/2.0",
"httpMethod": "POST",
"requestId": "1-621f7706-5e8f4ea33e2dc0cc66b98797"
}
}

0 Karma

venky1544
Builder

i just uploaded your data and assigned the sourcetype as _json

and ran the below query

index="newjson" sourcetype="_json" NOT excludedRules=null |table ruleGroupId ,"excludedRules{}.exclusionType"

venky1544_1-1646235752668.png

Note: you can rename the field as per your requirement 

is this what you wanted 

 

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...