Extracting fields from logs where a particular field sometimes does not exist

neilsmith2 — Thu, 23 Nov 2023 10:16:40 GMT

Hi all,

looking for help with how I can extract all available fields in a set of logs where a particular field sometimes does not exist.
In Log A, the 'inline result' field exists, but in Log B it does not and hence my current regex then fails for that log entry. I know I could probably use an Splunk app to auto manage this but I want to understand how I could do this myself.
Any suggestions please?

Log A
%FTD-1-4xxxxx: DeviceUUID: X, InstanceID: 13, FirstPacketSecond: 2023-11-23, ConnectionID: y, SrcIP: 10.10.10.10, DstIP: 11.11.11.11, SrcPort: 666, DstPort: 999, Protocol: tcp, IngressInterface: z, EgressInterface: inta, IngressZone: intb, EgressZone: intc, Priority: 1, GID: 1, SID: 58724, Revision: 6, Message: SERVER-OTHER Apache Log4j logging remote code execution attempt, Classification: Attempted User Privilege Gain, Client: Web browser, ApplicationProtocol: HTTP, IntrusionPolicy: IntPolicy-000001, ACPolicy: ACpolicy_00001, AccessControlRuleName: ACrule-000001, NAPPolicy: Balanced Security and Connectivity, InlineResult: Would have blocked, IngressVRF: Global, EgressVRF: Global

Log B
%FTD-1-yyyyyy: DeviceUUID: Y, InstanceID: 15, FirstPacketSecond: 2023-11-23, ConnectionID: Z, SrcIP: 12.12.12.12, DstIP: 13.13.13.13, SrcPort: 111, DstPort: 222, Protocol: tcp, IngressInterface: Port-channel6, EgressInterface: INT1, IngressZone: INT2, EgressZone:INT3, Priority: 2, GID: 133, SID: 59, Revision: 1, Message: DCE2_EVENT__SMB_BAD_NEXT_COMMAND_OFFSET, Classification: Potentially Bad Traffic, WebApplication: SMBv3-unencrypted, Client: NetBIOS-ssn (SMB) client, ApplicationProtocol: NetBIOS-ssn (SMB), IntrusionPolicy: INTIDS, ACPolicy: ACBpolicy, AccessControlRuleName: ACBrule, NAPPolicy: Balanced Security and Connectivity, IngressVRF: Global, EgressVRF: Global

Re: Extracting fields from logs where a particular field sometimes does not exist

PickleRick — Thu, 23 Nov 2023 10:33:44 GMT

A "?" operator in regex can be applied to a whole group. So this regex:

(?<prefix>\w+)(\s(?<middle>\w+))?\s(?<postfix>\w+)

Should match and parse two worded strings to prefix-postfix pair but if you happen to have the optional middle word, it will be parsed out as a "middle" field.

As always - use https://regex101.com/ and test your regexes. (it also provides a good explanation of what each regex component does).

Re: Extracting fields from logs where a particular field sometimes does not exist

neilsmith2 — Thu, 23 Nov 2023 10:57:06 GMT

Ah , yep that works perfectly! thank you!

topic Re: Extracting fields from logs where a particular field sometimes does not exist in Splunk Search

Extracting fields from logs where a particular field sometimes does not exist

Re: Extracting fields from logs where a particular field sometimes does not exist

Re: Extracting fields from logs where a particular field sometimes does not exist