transforms.conf

[shangshin]

[my_fields]
REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[qstring:method_url_protocol]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]

[method_url_protocol]
DELIMS = " "
FIELDS = method, url, protocol

Hi,
I define these 2 stanzas above in transforms.conf and expect to extract some info from web access log. As you can see the sample quoted string below, it contains 3 fields. However, these 3 fields are not extracted our successsfully. Can you shed some light on it?

"POST /amazon.com/view.do HTTP/1.1"

[jpmasseria]

Try using [[access-request]] to extract method,uri and version.

Below is what I used and it worked for me.

REGEX =
^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]\s++[[qstring:someurl]]\s++[[nspaces:response_time]]

[richprescott]

I think Ayn is referring to adding the regex as follows in props.conf.


[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields,method_url_protocol


[Ayn]

This is why I'm asking if you're actually referring to that transform from props.conf. If you just setup the transform but don't refer to it anywhere, it won't ever be applied.

[shangshin]

here is the complete sample roq in web access log
10.39.208.2 - clinet_user_id [29/May/2012:14:04:10 -0400] "POST /amazon.com/view.do HTTP/1.1" 200 1214 "google.com" "Java/1.5.0_06"

As you can, the field of method, url and protocol can be extracted out as a single value using the first stanza (my_fields). However, the second stanza (method_url_protocol) is unable to parse the value. I guess I didn't set it up properly...

[Ayn]

What about the method_url_protocol transform? That's the one that, if configured properly, would do the work.

[shangshin]

yes, that's right. In props.conf,

[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields

[Ayn]

Are you referring to these transforms in props.conf?