hi everyone,
i'm trying to parse json inline. i'm using kv mode= json already but i'm trying to achieve selective groups.
essentially i want to capture two groups if they have an "exclusion type"
sample json.
[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SizeRestrictions_BODY"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}]
so for this i wanted to capture only the ruleGroupId name if it has excludedRules not null, then capture the exclusionType
any help would be appreciated.
that looks good but i'm trying to create an inline extraction (props.conf) for this. so it only returns that info
See if this helps:
works when its parsed but not in raw . new lines aren't valid. here it is in raw: tried to work around it but my regex is horrible...lol
{"timestamp":1646240486931,"formatVersion":1,"webaclId":"arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-prod-web-acl/24e4f178-f008-434a-80f4-cd16728b9ffd","terminatingRuleId":"Default_Action","terminatingRuleType":"REGULAR","action":"ALLOW","terminatingRuleMatchDetails":[],"httpSourceName":"ALB","httpSourceId":"111111-app/site-internal-alb-production/07fae64dff77a3b3","ruleGroupList":[{"ruleGroupId":"AWS#AWSManagedRulesAmazonIpReputationList","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesBotControlRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":[{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"CategorySocialMedia"},{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"SignalNonBrowserUserAgent"}]},{"ruleGroupId":"AWS#AWSManagedRulesKnownBadInputsRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null},{"ruleGroupId":"AWS#AWSManagedRulesCommonRuleSet","terminatingRule":null,"nonTerminatingMatchingRules":[],"excludedRules":null}],"rateBasedRuleList":[],"nonTerminatingMatchingRules":[],"requestHeadersInserted":null,"responseCodeSent":null,"httpRequest":{"clientIp":"10.10.1.127","country":"-","headers":[{"name":"Accept-Encoding","value":"gzip"},{"name":"User-Agent","value":"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"},{"name":"X-BufferBot","value":"Being Awesome! P.S. We're hiring! buffer.com/journey"},{"name":"cookie","value":"_bit=m21bv9-710b10c6d1aad7e0f7-00o"},{"name":"x-datadog-trace-id","value":"272920074770865622"},{"name":"x-datadog-parent-id","value":"827293848173300227"},{"name":"x-datadog-sampled","value":"1"},{"name":"x-datadog-sampling-priority","value":"0"},{"name":"host","value":"www.site.com"},{"name":"Connection","value":"close"}],"uri":"/ana/training-technical-assistance/using-grantsgov-workspace","args":"","httpVersion":"HTTP/1.1","httpMethod":"GET","requestId":"1-621fa2e6-49f36a5f01f555c90fe7e63e"},"labels":[{"name":"awswaf:managed:aws:bot-control:bot:category:social_media"},{"name":"awswaf:managed:aws:bot-control:bot:name:facebook"},{"name":"awswaf:managed:aws:bot-control:signal:non_browser_user_agent"}]}
let me add the whole json...this is actually nested.
{
"timestamp": 1646229254523,
"formatVersion": 1,
"webaclId": "arn:aws:wafv2:us-east-1:111111:regional/webacl/alb-stage-web-acl/26ac170c-03c4-4fd7-8fab-86e346789fef",
"terminatingRuleId": "Default_Action",
"terminatingRuleType": "REGULAR",
"action": "ALLOW",
"terminatingRuleMatchDetails": [],
"httpSourceName": "ALB",
"httpSourceId": "182116744736-app/ALB-Stage/fcc1f5f9483b035e",
"ruleGroupList": [
{
"ruleGroupId": "AWS#AWSManagedRulesAmazonIpReputationList",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesBotControlRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
},
{
"ruleGroupId": "AWS#AWSManagedRulesCommonRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": [
{
"exclusionType": "EXCLUDED_AS_COUNT",
"ruleId": "SizeRestrictions_BODY"
}
]
},
{
"ruleGroupId": "AWS#AWSManagedRulesKnownBadInputsRuleSet",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null
}
],
"rateBasedRuleList": [],
"nonTerminatingMatchingRules": [],
"requestHeadersInserted": null,
"responseCodeSent": null,
"httpRequest": {
"clientIp": "67.218.14.10",
"country": "US",
"headers": [
{
"name": "host",
"value": "sample.com"
},
{
"name": "content-length",
"value": "50362"
},
{
"name": "cache-control",
"value": "max-age=0"
},
{
"name": "sec-ch-ua",
"value": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Microsoft Edge\";v=\"98\""
},
{
"name": "sec-ch-ua-mobile",
"value": "?0"
},
{
"name": "sec-ch-ua-platform",
"value": "\"Windows\""
},
{
"name": "origin",
"value": "https://sample.com"
},
{
"name": "upgrade-insecure-requests",
"value": "1"
},
{
"name": "dnt",
"value": "1"
},
{
"name": "content-type",
"value": "multipart/form-data; boundary=----WebKitFormBoundaryuXOFvh7iQjJkEJHm"
},
{
"name": "user-agent",
"value": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36 Edg/98.0.1108.62"
},
{
"name": "accept",
"value": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"
},
{
"name": "sec-fetch-site",
"value": "same-origin"
},
{
"name": "sec-fetch-mode",
"value": "navigate"
},
{
"name": "sec-fetch-user",
"value": "?1"
},
{
"name": "sec-fetch-dest",
"value": "document"
},
{
"name": "referer",
"value": "https://sample.com/DischargeDetail.aspx"
},
{
"name": "accept-encoding",
"value": "gzip, deflate, br"
},
{
"name": "accept-language",
"value": "en-US,en;q=0.9"
},
{
"name": "cookie",
"value": "_ga=GA1.3.84334902.1642521795; __RequestVerificationToken=-8kinKddCjKCZTws-wPmXDZTFg39urggswPnYm5Y15UwfIjspHqTj1hOPAXIaRPHL2cupyt2vO4Gb5QUExZGd6e5djS0v81kxt2pH22Ow9XiJYr2NPWB_BdQb-VmCUHVXbiVZZ5NwTfGDrXd2O0uD_gba4fM3PhkQUO5f9zs5381; _gid=GA1.2.249665053.1645964709; _ga_33R15ZN4N1=GS1.1.1645965393.6.0.1645965397.56; _ga=GA1.2.84334902.1642521795; ASP.NET_SessionId=1fnikipv2poi14r3doy4kb2w"
}
],
"uri": "/ReleaseRequest.aspx",
"args": "",
"httpVersion": "HTTP/2.0",
"httpMethod": "POST",
"requestId": "1-621f7706-5e8f4ea33e2dc0cc66b98797"
}
}
i just uploaded your data and assigned the sourcetype as _json
and ran the below query
index="newjson" sourcetype="_json" NOT excludedRules=null |table ruleGroupId ,"excludedRules{}.exclusionType"
Note: you can rename the field as per your requirement
is this what you wanted