Apache Logs Data Parsing

[anandhalagaras1]

Hi Team,

We have recently started ingesting Apache access and request logs from an application, but the data parsing isn't working as expected.

Could you please let me know the field names for these events so I can try to extract them manually? Alternatively, do we have any format or add-on available that would enable automatic field extraction? If so, that would also be fine with me.

For your information, our Splunk Search Head is hosted in the cloud and managed by Splunk Support.

I have provided the log structure for both log sources for reference. Please help to check and update.

Request Logs:
[09/Aug/2024:07:50:37 +0000] xx.yyy.zzz.aa TLSv1.2 ABCDE-FGH-IJK256-LMN-SHA123 "GET /share/page/ HTTP/1.1" xxxxx
[09/Aug/2024:07:50:37 +0000] xx.yyy.zzz.aa TLSv1.2 xxxxx-xxx-xxx256-xxx-xxx123 "GET /share/page/ HTTP/1.1" -

Access Logs:
xx.yyy.zzz.aa - - [09/Aug/2024:07:57:00 +0000] "GET /share/page/ HTTP/1.1" 200 xxxxx
aaa.bbb.ccc.dd - - [09/Aug/2024:07:56:53 +0000] "GET /share/page/ HTTP/1.1" 200 -

Thank you.

[JohnEGones]

These might be useful:

https://community.splunk.com/t5/All-Apps-and-Add-ons/parsing-log-text-to-get-a-specific-info/m-p/484...

https://community.splunk.com/t5/Getting-Data-In/How-to-configure-Splunk-to-properly-parse-logs-that-...

Also see if an app helps, the extractions and such like are useful to inspect and use as needed: 

https://splunkbase.splunk.com/app/3186#/overview

(from https://community.splunk.com/t5/Getting-Data-In/How-to-parse-Apache-access-logs-in-Splunk/m-p/266983)

[anandhalagaras1]

Can Anyone help on my request please.

[PickleRick]

1. This is not a Professional Support service. People do have their lives and respond when they have some spare time.

2. For typical apache httpd logs there are two built-in sourcetypes - access_combined and apache_error.

[isoutamo]

Splunk have default definitions for access_combined, access_combined_wcookie, access_common and apache_error. You can look those (.../system/default/props.conf and transforms.conf). You could use this as base for your own sourcetype definitions. 

[anandhalagaras1]

I cloned the "access_combined" sourcetype for the access logs, and now the fields are being extracted as desired.

However, I'm unable to parse the request logs as expected. If anyone has some time, I would appreciate assistance with parsing the request logs. It would be really helpful.

Request Logs Format:
[09/Aug/2024:07:50:37 +0000] xx.yyy.zzz.aa TLSv1.2 ABCDE-FGH-IJK256-LMN-SHA123 "GET /share/page/ HTTP/1.1" xxxxx
[09/Aug/2024:07:50:37 +0000] xx.yyy.zzz.aa TLSv1.2 xxxxx-xxx-xxx256-xxx-xxx123 "GET /share/page/ HTTP/1.1" -