topic Re: conditional regex help in Splunk Search

Conditional regex help: How to capture two groups if they have an "exclusion type"?

sdee1013 — Thu, 03 Mar 2022 00:16:50 GMT

hi everyone,

i'm trying to parse json inline. i'm using kv mode= json already but i'm trying to achieve selective groups.

essentially i want to capture two groups if they have an "exclusion type"

sample json.

[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SizeRestrictions_BODY"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}]

so for this i wanted to capture only the ruleGroupId name if it has excludedRules not null, then capture the exclusionType

any help would be appreciated.

Re: conditional regex help

venky1544 — Wed, 02 Mar 2022 15:43:48 GMT

i just uploaded your data and assigned the sourcetype as _json

and ran the below query

index="newjson" sourcetype="_json" NOT excludedRules=null |table ruleGroupId ,"excludedRules{}.exclusionType"

Note: you can rename the field as per your requirement

is this what you wanted

Re: conditional regex help

sdee1013 — Wed, 02 Mar 2022 16:30:37 GMT

that looks good but i'm trying to create an inline extraction (props.conf) for this. so it only returns that info

Re: conditional regex help

sdee1013 — Wed, 02 Mar 2022 16:39:45 GMT

let me add the whole json...this is actually nested.

{
"timestamp": 1646229254523,
"formatVersion": 1,
"webaclId": "arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-stage-web-acl/26ac170c-03c4-4fd7-8fab-86e346789fef",
"terminatingRuleId": "Default_Action",
"terminatingRuleType": "REGULAR",
"action": "ALLOW",
"terminatingRuleMatchDetails": [],
"httpSourceName": "ALB",
"httpSourceId": "182116744736-app/ALB-Stage/fcc1f5f9483b035e",
"ruleGroupList": [
{
"ruleGroupId": "AWS#AWSManagedRulesAmazonIpReputationList",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesBotControlRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesCommonRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": [
{
"exclusionType": "EXCLUDED_AS_COUNT",
"ruleId": "SizeRestrictions_BODY"
}
]
},
{
"ruleGroupId": "AWS#AWSManagedRulesKnownBadInputsRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
}
],
"rateBasedRuleList": [],
"nonTerminatingMatchingRules": [],
"requestHeadersInserted": null,
"responseCodeSent": null,
"httpRequest": {
"clientIp": "67.218.14.10",
"country": "US",
"headers": [
{
"name": "host",
"value": "sample.com"
},
{
"name": "content-length",
"value": "50362"
},
{
"name": "cache-control",
"value": "max-age=0"
},
{
"name": "sec-ch-ua",
"value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Microsoft Edge\";v=\"98\""
},
{
"name": "sec-ch-ua-mobile",
"value": "?0"
},
{
"name": "sec-ch-ua-platform",
"value": "\"Windows\""
},
{
"name": "origin",
"value": "https://sample.com"
},
{
"name": "upgrade-insecure-requests",
"value": "1"
},
{
"name": "dnt",
"value": "1"
},
{
"name": "content-type",
"value": "multipart/form-data; boundary=----WebKitFormBoundaryuXOFvh7iQjJkEJHm"
},
{
"name": "user-agent",
"value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.62"
},
{
"name": "accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
},
{
"name": "sec-fetch-site",
"value": "same-origin"
},
{
"name": "sec-fetch-mode",
"value": "navigate"
},
{
"name": "sec-fetch-user",
"value": "?1"
},
{
"name": "sec-fetch-dest",
"value": "document"
},
{
"name": "referer",
"value": "https://sample.com/DischargeDetail.aspx"
},
{
"name": "accept-encoding",
"value": "gzip, deflate, br"
},
{
"name": "accept-language",
"value": "en-US,en;q=0.9"
},
{
"name": "cookie",
"value": "_ga=GA1.3.84334902.1642521795; __RequestVerificationToken=-8kinKddCjKCZTws-wPmXDZTFg39urggswPnYm5Y15UwfIjspHqTj1hOPAXIaRPHL2cupyt2vO4Gb5QUExZGd6e5djS0v81kxt2pH22Ow9XiJYr2NPWB_BdQb-VmCUHVXbiVZZ5NwTfGDrXd2O0uD_gba4fM3PhkQUO5f9zs5381; _gid=GA1.2.249665053.1645964709; _ga_33R15ZN4N1=GS1.1.1645965393.6.0.1645965397.56; _ga=GA1.2.84334902.1642521795; ASP.NET_SessionId=1fnikipv2poi14r3doy4kb2w"
}
],
"uri": "/ReleaseRequest.aspx",
"args": "",
"httpVersion": "HTTP/2.0",
"httpMethod": "POST",
"requestId": "1-621f7706-5e8f4ea33e2dc0cc66b98797"
}
}

Re: conditional regex help

somesoni2 — Wed, 02 Mar 2022 19:32:46 GMT

See if this helps:

https://regex101.com/r/xVgyru/1

Re: conditional regex help

sdee1013 — Wed, 02 Mar 2022 20:45:22 GMT

works when its parsed but not in raw . new lines aren't valid. here it is in raw: tried to work around it but my regex is horrible...lol

{"timestamp":1646240486931,"formatVersion":1,"webaclId":"arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-prod-web-acl/24e4f178-f008-434a-80f4-cd16728b9ffd","terminatingRuleId":"Default_Action","terminatingRuleType":"REGULAR","action":"ALLOW","terminatingRuleMatchDetails":[],"httpSourceName":"ALB","httpSourceId":"111111-app/site-internal-alb-production/07fae64dff77a3b3","ruleGroupList":[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"CategorySocialMedia"},{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SignalNonBrowserUserAgent"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}],"rateBasedRuleList":[],"nonTerminatingMatchingRules":[],"requestHeadersInserted":null,"responseCodeSent":null,"httpRequest":{"clientIp":"10.10.1.127","country":"-","headers":[{"name":"Accept-Encoding","value":"gzip"},{"name":"User-Agent","value":"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"},{"name":"X-BufferBot","value":"Being Awesome! P.S. We're hiring! buffer.com/journey"},{"name":"cookie","value":"_bit=m21bv9-710b10c6d1aad7e0f7-00o"},{"name":"x-datadog-trace-id","value":"272920074770865622"},{"name":"x-datadog-parent-id","value":"827293848173300227"},{"name":"x-datadog-sampled","value":"1"},{"name":"x-datadog-sampling-priority","value":"0"},{"name":"host","value":"www.site.com"},{"name":"Connection","value":"close"}],"uri":"/ana/training-technical-assistance/using-grantsgov-workspace","args":"","httpVersion":"HTTP/1.1","httpMethod":"GET","requestId":"1-621fa2e6-49f36a5f01f555c90fe7e63e"},"labels":[{"name":"awswaf:managed:aws:bot-control:bot:category:social_media"},{"name":"awswaf:managed:aws:bot-control:bot:name:facebook"},{"name":"awswaf:managed:aws:bot-control:signal:non_browser_user_agent"}]}