Adobe CQ5 Request.log parsing?

[wobes17]

Generally web and app server record the response times along with URL, http status code, etc in the access.log. CQ5 however, records response times in their request.log which is a multiline file. Is is possible for splunk to parce the multiline file?

here is some example output of the file:

24/May/2013:10:09:50 -0400 [1638] -> GET /crx/server/crx.default/jcr%3aroot/var.1.json?_dc=1369404590843&node=xnode-264 HTTP/1.1
24/May/2013:10:09:50 -0400 [1638] <- 200 text/plain;charset=utf-8 8ms
24/May/2013:10:10:00 -0400 [1639] -> GET /crx/server/crx.default/jcr%3aroot/libs.1.json?_dc=1369404600880&node=xnode-265 HTTP/1.1
24/May/2013:10:10:00 -0400 [1639] <- 200 text/plain;charset=utf-8 6ms

The 1st line of each is the Request and the second is the response. at the end of the response line it shows you the time in milliseconds it took to process (the metric i'm looking for). For each pair there is a transaction ID that ties the two together. in this example the requests and the responses are next to each other, but on a busy system they could be lines apart in the log file.

How can I leverage splunk with this use case?

[deepakmb]

Hi @wobes17,
May i know, In splunk how you are adding response time?
I have a requirement to add response time in splunk.

Below is the example log storing in splunk from AEM,

time=[29/May/2019:05:59:59 -0400], dest_host=-, src=-, http_method=GET, user=-, request="GET / HTTP/1.0", status=404, bytes_out=198, http_referrer="-", vendor_product="Apache", http_user_agent="HTTP-Monitor/1.1", x-forwarded-for="-"

I want to add response time also for this request. please help on this.

Thanks,
Deepak manibabu.

[sabarishogili]

Hi Deepak,

An random question, how did you get the json formatted log in AEM?

[mpetrovsky]

Yes it's possible. Use the "transaction" command.

source="/opt/cq5/instance_1/logs/access.log" | transaction maxpause=4m keepevicted=true ThreadId host maxevents=2 

The above statement

1. looks for a maximum of 2 events (maxevent=2) to form one row
2. it looks at a 4 minute window between the two events (maxpause=4m)
3. matches the two events by comparing the host and "ThreadId"

The "ThreadId" is a custom field extraction that picks out the unique number in the square brackets 'eg [1638]' in your example:

EXTRACT-ThreadId Inline  (?i)^[^\+]*\+\d+\s+\[(?P<ThreadId>[^\]]+)]\s+[<-][>-]  

Hope that helps