topic Re: Dynamic field extraction with regex syslog's in Splunk Enterprise

Dynamic field extraction with regex syslog's?

snix — Mon, 06 Mar 2023 14:40:58 GMT

We are ingesting Firepower logs via syslog using the cisco:asa TA. Many of the events I am interested in are Threat Defense events that are tied to an ID like this FTD-6-430002. When I narrow down my search to events with just that ID I find the rest of the event has plenty of info in key:value pairs but no fields have been extracted from the pairs.

Sanitized example event:

Mar 3 16:01:21 172.16.51.72 Mar 03 2023 22:01:21 firepower : %FTD-6-430002: EventPriority: Low, DeviceUUID: 00000-0000-0000-000000000000, InstanceID: 1, FirstPacketSecond: 2023-03-03T22:01:21Z, ConnectionID: 5000, AccessControlRuleAction: Allow, SrcIP: 100.100.100.100, DstIP: 200.200.200.200, SrcPort: 60000, DstPort: 10, Protocol: tcp

Is there a regex command that can dynamically extract all the field names from something like "DstPort: 10" to Field Name of DstPort with a value of 10?

I know Cisco provides a eStreamer TA that may extract these fields but it looks very involved to setup and I already have the syslog configured.

Re: Dynamic field extraction with regex syslog's

richgalloway — Sat, 04 Mar 2023 01:33:10 GMT

The extract command is made for that kind of parsing, but it will be tripped up by the extra ":" separators in the timestamp.

Regex will do the job, however, but there's no special command for it. Assuming the fields are always in the same order then this will do it.

| rex "firepower : (?<firepower>[^:]+): EventPriority: (?<EventPriority>\w+), DeviceUUID: (?<DeviceUUID>[^,]+), InstanceID: (?<InstanceID>[^,]+), FirstPacketSecond: (?<FirstPacketSecond>[^,]+), ConnectionID: (?<ConnectionID>[^,]+), AccessControlRuleAction: (?<AccessControlRuleAction>[^,]+), SrcIP: (?<SrcIP>[^,]+), DstIP: (?<DstIP>[^,]+), SrcPort: (?<SrcPort>\d+), DstPort: (?<DstPort>\d+), Protocol: \w+"

If the fields might appear in any order then a separate rex command is needed for each one.

Re: Dynamic field extraction with regex syslog's

snix — Mon, 06 Mar 2023 20:28:34 GMT

Thank you! Yeah I was hoping there was a way to just pull the filed name from the event automatically but for what I need it for right now I just need a few fields.

Yeah the events seem to vary a bit on what fields are included and not included so I opted to take your second suggestion and spit them up and that worked like a charm:

| rex "AccessControlRuleAction: (?<AccessControlRuleAction>[^,]+)" | rex "SrcIP: (?<SrcIP>[^,]+)" | rex "DstIP: (?<DstIP>[^,]+)" | rex "DstPort: (?<DstPort>[^,]+)"