cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question
khenson
Engager
Member since: ‎10-01-2021
‎11-02-2021
Community Statistics
Posts 4
Solutions 0
Karma Given 0
Karma Received 0
Member Since ‎10-01-2021
View All

mvexpand multi-value fields when not null

by in Splunk Search
‎10-27-2021 01:59 PM
‎10-27-2021 01:59 PM
Hi all.  I'm trying to create a table from AWS WAF logs.  There is a section of the log that is called ruleGroupList{} and it is a list containing multiple dictionaries.  Sometimes there is field called "excludedRules" that is null.  When it is not null, it is a list containing a dictionary with a field called ruleId.     ruleGroupList :   [   [-]      { [-]         excludedRules :  null         nonTerminatingMatchingRules : [ [+]        ]         ruleGroupId :  AWS#AWSManagedRulesBotControlRuleSet         terminatingRule :  null      }      { [-]         excludedRules : [ [-]          { [-]             exclusionType :  EXCLUDED_AS_COUNT             ruleId :  SizeRestrictions_BODY          }        ]         nonTerminatingMatchingRules : [ [+]        ]         ruleGroupId :  AWS#AWSManagedRulesCommonRuleSet         terminatingRule :  null      } In this case, I want to: list the ruleGroupList{}.ruleGroupId and the ruleGroupList{}.excludedRules{}.ruleId in a table, when ruleGroupList{}.excludedRules is not NULL.  If it is NULL, then I don't want to display the values for that dictionary.  There are 7 dictionaries in this ruleGroupList{} (as long as I don't change my WAF settings in AWS). This is my search: <search> | | spath input=ruleGroupList{} path=excludedRules | rename ruleGroupList{}.ruleGroupId as ruleGroup, ruleGroupList{}.excludedRules{}.ruleId as ruleGroupId, ruleGroupList{}.excludedRules as testNullExcludedRules | eval x=case(!isnull(testNullExcludedRules),mvzip(ruleGroup,ruleGroupId),isnull(testNullExcludedRules),x) | mvexpand x | eval x = split(x,",") | eval ruleGroupId=case(!isnull(testNullExcludedRules),mvindex(x,1)) | eval ruleGroup=case(!isnull(testNullExcludedRules),mvindex(x,0)) | table _time,ruleGroup,ruleGroupId This gives me the ruleGroupId correctly, but it always lists the first instance of the ruleGroup: I can't figure out how to ignore the ruleGroup when it's corresponding excludedRules is NULL. thanks for any help! Kevin ... View more
Labels

Re: How to parse logs with a mix of JSON and non-J...

by in Getting Data In
‎10-04-2021 09:44 AM
‎10-04-2021 09:44 AM
Thank you very much for your time and sharing this.  I will start looking at how to incorporate this in my sourcetype in SplunkCloud. ... View more

Re: How to parse logs with a mix of JSON and non-J...

by in Getting Data In
‎10-04-2021 09:19 AM
‎10-04-2021 09:19 AM
I see an error in my sanitized log, the last </Field43> should have been </Auth>.  This seems pretty close, but the JSON didn't appear to show up as separate fields.  I was thinking that the JSON could be parsed into: response.Auth.Field1 respoonse.Auth.Field2 Does that make sense?   thanks! ... View more

How to parse logs with a mix of JSON and non-JSON

by in Getting Data In
‎10-04-2021 08:23 AM
‎10-04-2021 08:23 AM
Hi.  I have log source that has a mix of various field types and then a larger nested JSON payload.  I can't quite wrap my head around how to parse this out in our SplunkCloud environment. High level, the log contains this: date field server name field (separated by four dashes most of the time, but some env have three) process name[PID]  source code function variable field ending with a colon char source code function variable's value, which may or may not have special chars like ()  JSON  ends with [] Sanitized example: Oct 1 20:04:22 my-web01-aa-env my-web[14597]: app.NOTICE: Gateway Transaction Response (BP) {"response":"<Auth><Field1>ch-abcd1234-ab12-ab12-ab12-abcdef123456</Field1><Field2>0123</Field2><Field3>0123</Field3><Field4>Successful Request</Field4><responseMessage><Field5>0123</Field5><Field6>0123</Field6><Field7>0123</Field7><Field8>0123</Field8><Field9>0123</Field9><Field10>0123</Field10><Field11>0123</Field11><Field12>0123</Field12><Field13>012 - Approved (APPROVAL 001077)</Field13><Field14>Approved: 012345 (approval code)</Field14><Field15>00000</Field15><Field16>Address not available (Address not verified)</Field16><Field17>40</Field17><Field18></Field18><Field19>012345</Field19><Field20>20211001</Field20><Field21>0</Field21><Field22>840</Field22><Field23>USD</Field23><Field24>2021-10-01 16:04:21.493</Field24><Field25>2021-10-01 16:04:21.493</Field25><Field26>0123456</Field26><Field27>0123</Field27><Field28>0123456</Field28><Field29>0006</Field29><Field30>ABCDEF</Field30><Field31>Abc</Field31><Field32>ABC ABC ABC</Field32><Field33>Abcd</Field33><Field34>00</Field34><Field35>ABCDEF 012345 </Field35><Field36>ABCDEF0123</Field36><Field37>4F:A0000000041010;95:0000008000;9F10:0110A040002A0000000000000000000000FF;9B:E800;91:6325A37CFBC5CEDD0012;8A:</Field37><Field38>012345012345</Field38><Field39>abcd</Field38><Field39>False</Field39><Field40 /><Field40 /><Field41 /><Field42 /><Field42 /></Field43></Field43>"} []   My grok-fu is not great.  Would appreciate any suggestions.  Thanks! ... View more
Labels
Contact Me
Online Status
Offline
Date Last Visited
‎11-02-2021 08:51 PM