All Apps and Add-ons

Splunk App for AWS: How to adjust the regular expression in props.conf to extract fields in ELB Access Logs?

Contributor

Hi All .

Really need some help with this one .

I had to adjust the regex within props.conf to extract the field for all ELB's correctly ( see below )

^(?P<timestamp>[^ ]+)[^ \n]* (?P<elb>[^ ]+)\s+(?P<client_ip>[^:]+):(?P<client_port>\d+)\s+(?P<backend>[^ ]+)\s+(?P<request_processing_time>[^ ]+)\s+(?P<backend_processing_time>[^ ]+)\s+(?P<response_processing_time>\d+\.\d+)\s+(?P<elb_status_code>\d+)\s+(?P<backend_status_code>\d+)\s+(?P<received_bytes>\d+)\s+(?P<sent_bytes>\d+)\s+"(?P<request>[^"]+)"\s+"(?P<user_agent>\-)"\s+(?P<ssl_cipher>[^ ]+)\s+(?P<ssl_protocol>.+)

Everything populates correctly in the dashboard with exception of elb_status_code and backend_status_code - any insight into what I have missed or done wrong would be greatly appreciated.

Thanks !

0 Karma
1 Solution

Contributor

If the value of elb_status_code is always followed by the key "eventtype" in each of those events, you could try something like this:

elb_status_code\s\=\s(?P<elb_status_code>.[^eventtype]*)

Worked for me:
alt text

Also, as knielsen mentioned, the value of elb_status_code is "-"in the log sample you provided us.

View solution in original post

0 Karma

Contributor

If the value of elb_status_code is always followed by the key "eventtype" in each of those events, you could try something like this:

elb_status_code\s\=\s(?P<elb_status_code>.[^eventtype]*)

Worked for me:
alt text

Also, as knielsen mentioned, the value of elb_status_code is "-"in the log sample you provided us.

View solution in original post

0 Karma

Contributor

Glad I could help 🙂

0 Karma

Contributor

That did the trick ! Your the "Regex Whisperer"

Thanks adayton20 !

I can now see all 16 ELBs and the dashboard is populating with the required information .

0 Karma

Engager

Hi @klaxdal 

Would you be able to share the props.conf config block you have added to ingest ELB logs. Currently I'm experiencing similar issue, where I cannot parse the logs properly. Thanks!

0 Karma

Contributor

Could you post a sample of the event data?

0 Karma