All Apps and Add-ons

Splunk App for AWS: How to adjust the regular expression in props.conf to extract fields in ELB Access Logs?

klaxdal
Contributor

Hi All .

Really need some help with this one .

I had to adjust the regex within props.conf to extract the field for all ELB's correctly ( see below )

^(?P<timestamp>[^ ]+)[^ \n]* (?P<elb>[^ ]+)\s+(?P<client_ip>[^:]+):(?P<client_port>\d+)\s+(?P<backend>[^ ]+)\s+(?P<request_processing_time>[^ ]+)\s+(?P<backend_processing_time>[^ ]+)\s+(?P<response_processing_time>\d+\.\d+)\s+(?P<elb_status_code>\d+)\s+(?P<backend_status_code>\d+)\s+(?P<received_bytes>\d+)\s+(?P<sent_bytes>\d+)\s+"(?P<request>[^"]+)"\s+"(?P<user_agent>\-)"\s+(?P<ssl_cipher>[^ ]+)\s+(?P<ssl_protocol>.+)

Everything populates correctly in the dashboard with exception of elb_status_code and backend_status_code - any insight into what I have missed or done wrong would be greatly appreciated.

Thanks !

0 Karma
1 Solution

adayton20
Contributor

If the value of elb_status_code is always followed by the key "eventtype" in each of those events, you could try something like this:

elb_status_code\s\=\s(?P<elb_status_code>.[^eventtype]*)

Worked for me:
alt text

Also, as knielsen mentioned, the value of elb_status_code is "-"in the log sample you provided us.

View solution in original post

0 Karma

adayton20
Contributor

If the value of elb_status_code is always followed by the key "eventtype" in each of those events, you could try something like this:

elb_status_code\s\=\s(?P<elb_status_code>.[^eventtype]*)

Worked for me:
alt text

Also, as knielsen mentioned, the value of elb_status_code is "-"in the log sample you provided us.

0 Karma

adayton20
Contributor

Glad I could help 🙂

0 Karma

klaxdal
Contributor

That did the trick ! Your the "Regex Whisperer"

Thanks adayton20 !

I can now see all 16 ELBs and the dashboard is populating with the required information .

0 Karma

krisrmal
Engager

Hi @klaxdal 

Would you be able to share the props.conf config block you have added to ingest ELB logs. Currently I'm experiencing similar issue, where I cannot parse the logs properly. Thanks!

0 Karma

adayton20
Contributor

Could you post a sample of the event data?

0 Karma
Get Updates on the Splunk Community!

Fun with Regular Expression - multiples of nine

Fun with Regular Expression - multiples of nineThis challenge was first posted on Slack #regex channel ...

[Live Demo] Watch SOC transformation in action with the reimagined Splunk Enterprise ...

Overwhelmed SOC? Splunk ES Has Your Back Tool sprawl, alert fatigue, and endless context switching are making ...

What’s New & Next in Splunk SOAR

Security teams today are dealing with more alerts, more tools, and more pressure than ever.  Join us on ...