topic Re: Field Extraction with square brackets in Splunk Dev

Field Extraction with square brackets

shreya1770 — Fri, 07 May 2021 11:28:51 GMT

I'm trying to parse the below sample using Delimiters, could anyone help with the extraction. Delimiters doesn't work for this. Can someone help with Regex commands.

[2021-05-07T20:54:50.6222+10:00] [BDF] [ERROR:32] [BD99999] [security2] [client_id: 10.10.18.236] [host_id: google.com ] [host_addr: 10.10.05.11] [pid: 5397] [tid: 139783720359680] [user: apaapp] [ecid: 005kRh1ly^x8dpK_yTk3yW0001K80002jb] [rid: 0] [VirtualHost: google:4445] [client 0.10.18.236] ModSecurity: Warning. Pattern match "^[\\\\d.:]+$" at REQUEST_HEADERS:Host. [file "/apps/vbgrt/bdf/Google/Middleware/user_projects/domains/bdf_domain/config/fmwconfig/components/BDF/instances/bcp/crs-rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "735"] [id "920350"] [msg "Host header is a numeric IP address"] [data"10.10.05.11:4445"] [severity "WARNING"] [ver "OWASP_PQR/3.3.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_PQR"] [tag "capec/1000/210/272"] [tag "PCI/6.5.10"] [hostname "google"] [uri "/"] [unique_id "HTjues090uwmX0Cz1kLVwAAAIw"]

Re: Field Extraction with square brackets

richgalloway — Fri, 07 May 2021 12:29:04 GMT

What exactly do you want to extract from that event?

Re: Field Extraction with square brackets

ITWhisperer — Fri, 07 May 2021 12:53:32 GMT

rex max_match=0 "\[(?<field>[^\]]+)\]"

However, you have [ and ] inside a quoted string which may not be what you want, but would be trickier to avoid and perhaps might be just easier to accept and deal with separately.

Re: Field Extraction with square brackets

erika_horton — Fri, 07 May 2021 17:19:35 GMT

^ that is roughly how I would write the extractions as well. I don't like delimiters, I would punch the sample data into regex101 and write an extraction out where each capture group is everything inside the bracket
example:

\[BDF\]\s\[(?<error_code>[^\]]+)\]\s\[(?<field2>[^\]]+)\]\s\[(?<field3>[^\]]+)\]

Just repeat the pattern until finished.

Re: Field Extraction with square brackets

shreya1770 — Sat, 08 May 2021 13:41:28 GMT

Need to extract the data like uri, unique id and many more. Mostly those are enclosed with [ ]