topic Re: How to extract these fields from my sample data? in Splunk Search

How to extract these fields from my sample data?

vrmandadi — Wed, 04 Jan 2017 20:56:19 GMT

Hello Experts,

Below is the sample event

event_type:  LogMessage 
    ip:  xx.x.xx.xx 
    job:  router_z1 
    job_index:  1 
    level:  info 
    message_type:  OUT 
    msg:  url.com - [04/01/2017:19:37:03.701 +0000] "GET /api/v1/vha/supported-vehicles HTTP/1.1" 200 0 1565 "-" "fordpass-app-na/17 CFNetwork/808.2.16 Darwin/16.3.0" xx.x.xx.xx:45810 x_forwarded_for:"xxx.xx.xxx.xx, xx.xx.xx.xxx, xx.x.xx.xx" x_forwarded_proto:"https" vcap_request_id:8d612e09-f611-415f-6172-a688700be726 response_time:0.004987 app_id:19065d0e-f6da-4cc3-8a71-581fd768dafe

origin:  gorouter 
source_instance:  1 
source_type:  RTR 
time:  2017-01-04T19:37:03Z 
timestamp:  1483558623706728200

I am trying to extract the "GET /api/v1/vha/supported-vehicles " and other similar kinds like "POST /oauth/token", "GET /eureka/apps/delta" ,GET /api/v1/badges,"GET /api/v1","POST /api/v1/humanify-session" etc in the message field.

I tried using the erex command

erex clean examples="GET /api/v1/badges,GET /api/v1,POST /api/v1/record"

but it is not showing all of them and I also tried using IFX, but I can only select one field...

Is there any other way to extract all these fields from the msg app other than regex...

Thanks for the help

Re: How to extract these fields from my sample data?

somesoni2 — Wed, 04 Jan 2017 21:06:01 GMT

Is this a json data?

Try something like for now

your base search | rex "\]\s+\"(?<message>\w+\s+\S+)"

Re: How to extract these fields from my sample data?

gokadroid — Thu, 05 Jan 2017 01:46:18 GMT

If GET and POST is all you are interested in then you can try this as well:

your base search
| rex "\"(?<actionUri>(GET|POST)\s*?[\S]+)"
| table actionUri

If additional actions are needed feel free to add in (GET|POST|...) part of the above rex

Re: How to extract these fields from my sample data?

vrmandadi — Thu, 05 Jan 2017 13:46:28 GMT

Hi somesh, so using rex command is the only way to extract them?....is there a way to use IFX or erex

Re: How to extract these fields from my sample data?

somesoni2 — Thu, 05 Jan 2017 16:10:58 GMT

If your logs and the string that you're trying to extract is not formatted in a standard way (they don't follow same patter/position for all events), the erex and IFX may fail to extract all values. Using your custom regular expression which will explicitly take care of all variation is the way to go in cases like that.

Re: How to extract these fields from my sample data?

vrmandadi — Fri, 06 Jan 2017 01:01:52 GMT

ok thank you somesh

Re: How to extract these fields from my sample data?

vrmandadi — Fri, 06 Jan 2017 01:02:24 GMT

Thank You!

Re: How to extract these fields from my sample data?

somesoni2 — Fri, 06 Jan 2017 04:49:54 GMT

You'll be able to save the regular expression that you used in search as field extraction, so that its automatically available (similar to what IFX would have done).