Getting Data In

transforms.conf

shangshin
Builder
[my_fields]
REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[qstring:method_url_protocol]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]

[method_url_protocol]
DELIMS = " "
FIELDS = method, url, protocol

Hi,
I define these 2 stanzas above in transforms.conf and expect to extract some info from web access log. As you can see the sample quoted string below, it contains 3 fields. However, these 3 fields are not extracted our successsfully. Can you shed some light on it?

"POST /amazon.com/view.do HTTP/1.1"

Tags (1)

jpmasseria
Engager

Try using [[access-request]] to extract method,uri and version.

Below is what I used and it worked for me.

REGEX =
^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user_id]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]]\s++[[qstring:referer_url]]\s++[[qstring:useragent]]\s++[[qstring:someurl]]\s++[[nspaces:response_time]]

richprescott
Path Finder

I think Ayn is referring to adding the regex as follows in props.conf.


[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields,method_url_protocol

0 Karma

Ayn
Legend

This is why I'm asking if you're actually referring to that transform from props.conf. If you just setup the transform but don't refer to it anywhere, it won't ever be applied.

0 Karma

shangshin
Builder

here is the complete sample roq in web access log
10.39.208.2 - clinet_user_id [29/May/2012:14:04:10 -0400] "POST /amazon.com/view.do HTTP/1.1" 200 1214 "google.com" "Java/1.5.0_06"

As you can, the field of method, url and protocol can be extracted out as a single value using the first stanza (my_fields). However, the second stanza (method_url_protocol) is unable to parse the value. I guess I didn't set it up properly...

0 Karma

Ayn
Legend

What about the method_url_protocol transform? That's the one that, if configured properly, would do the work.

0 Karma

shangshin
Builder

yes, that's right. In props.conf,

[access_log_reg]
NO_BINARY_CHECK = 1
pulldown_type = 1
REPORT-myfields = my_fields

0 Karma

Ayn
Legend

Are you referring to these transforms in props.conf?

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...