How to use rex to create a field and using lookup to inject the data in the same field?

[mikeyty07]

I am using rex field to extract the field name and then inject the data so I can get only the desired fields but not able to do so.

My Access logs: 
server - - [date& time] "GET /google/page1/page1a/633243463476/googlep1 HTTP/1.1" 200 350 85

My search query:
<query>
| rex field_=(?<SRC>\d+\.\d+\.\d+\.\d+).+\]\s\"(?<http_method>\w+)\s(?<serviceName>/[^/]+)(?<uri_path>[^?\s]+)\s(?<uri_query>\S+)\"\s(?<statusCode>\d+)\s(?<body_size>\d+)\s\s(?<response_time>\d+)

My Search query with lookup
<query>
| rex field=_raw "(?<SRC>\d+\.\d+\.\d+\.\d+).+\]\s\"(?<http_method>\w+)\s(?<serviceName>/[^/]+)(?<uri_path>[^?\s]+)\s(?<uri_query>\S+)\"\s(?<statusCode>\d+)\s(?<body_size>\d+)\s\s(?<response_time>\d+)"

|search serviceName="/google"
| lookup abc.csv uri_path OUTPUT serviceName apiName | search searviceName=* operationName=*

I am using above query to lookup from csv file but I get all the api has same count and not able to get the stats or logs for only particular.

Is there a way to match this and produce result with both uri_path and api_name? can anyone please help me on this?
Eg: csv file looks like this and i am trying to match apiName and uri_path so the logs are getting properly.

serviceName	uri_path	http_method	apiName
/google	/page1/page1a/*/googlep1	post	postusingRRR
/google	/page1/page1a/sada/*/googlep1	get	getusingep2
/google	/pag5/ggg/*/ooopp/ggplr	delete	deleteusing

[richgalloway]

The regex does not match the sample log entry.  The regex is looking for an IP address, but none exist in the log.

Additionally, the slashes in the regex must be escaped.  Plus Splunk SPL requires triple-escapes with embedded quotation marks.

This regex works with the sample log entry, except it doesn't extract the SRC field.

\]\s\"(?<http_method>\w+)\s(?<serviceName>\/[^\/]+)(?<uri_path>\S+)\s(?<uri_query>\S+)\\\"\s(?<statusCode>\d+)\s(?<body_size>\d+)\s(?<response_time>\d+)

Are the asterisks in the uri_path field of the lookup wildcards or literal characters?  Splunk will treat them as literal '*' characters and will fail to find a match.  To support wildcards in a lookup, you must create a lookup definition ("abc", for instance) that points to abc.csv.  The definition must use the advanced setting "WILDCARD(uri_path)" in the Match type field.  Then use 

| lookup abc uri_path OUTPUT serviceName apiName 

in the query.

There is no 'operationName' field in the sample data so search operationName=* will cause the query to return no results.

[mikeyty07]

The rex search query is good which I can get the fields. Its the same field i am trying to match so it can produce all the results for one api.

this is an example from my csv file.

serviceName	uri_path	http_method	apiName
/google	/page1/page1a/*/googlep1	post	postusingRRR
/google	/page1/page1a/sada/*/googlep1	get	getusingep2
/google	/pag5/ggg/*/ooopp/ggplr	delete	deleteusing

the /*/ value changes in logs so I'm using the apiName to connect. In real the api looks like this
/google/page1/page1a/{pageNumber}/googlep1


Only issue is with the lookup. How do i use the wildcard on this?

[richgalloway · Accepted solution]

To support wildcards in a lookup, you must create a lookup definition ("abc", for instance) that points to abc.csv. The definition must use the advanced setting "WILDCARD(uri_path)" in the Match type field.

Go to Settings->Lookups and click "Add new" next to "Lookup definitions".  Fill in the form, selecting 'abc.csv' as the lookup file.  Check the "Advanced options" box then fill in the "Match type" box as shown.  Click Save.