topic Re: How can i extract keywords from my log as field values for field name API's in Splunk Search

How can i extract keywords from my log as field values for field name API's

Sujithkumarkb — Wed, 30 Sep 2020 03:36:44 GMT

How can i extract the below block letter keywords (OrderUpdateWithAccountInfoRequest ,VinValidationRequest,GetEntitledRequest ..)from my log as field values for field name API's?

2020-01-09 03:58:08,280 INFO com.hti.gw.interceptor.ServiceInterceptor (Hughes_Tre13342)
<OrderUpdateWithAccountInfoRequest ** xmlns:ns5=".......
2020-01-08 06:25:25,836 INFO com.vzt.pg.AbstractMiddlewareDelegate (AMP_RptDqckdAsT5ldcFG8eh_tdzbmtxux44z850) <VinValidationRequest** xmlns:ns2="http://www.hughestelematics.com.....
2020-01-08 06:25:25,546 INFO com.vzt.pg.AbstractMiddlewareDelegate (AMP_RptDqckdAsT5ldcFG8eh_tdzbmtxux44z850) <GetEntitledRequest xmlns:ns2="ht......
2020-01-08 06:20:13,637 INFO com.vzt.pg.AbstractMiddlewareDelegate
(AMP_RptDqckdAsT5ldcFG8eh_9wiiwnvakzcdc66) <VinValidationRequest xmlns:ns2="http:/......

Re: How can i extract keywords from my log as field values for field name API's

richgalloway — Thu, 09 Jan 2020 13:20:01 GMT

The rex command will do that at search time.

... | rex "\)\s+\<(?<API>\S+)" | ...

Re: How can i extract keywords from my log as field values for field name API's

TISKAR — Thu, 09 Jan 2020 14:53:18 GMT

Hello :),

I simulate the extractation of API field, This is example to extract field API from log, you can past this request in your empty search and run it:

| makeresults
| eval raw="2020-01-09 03:58:08,280 INFO \[com.hti.gw.interceptor.ServiceInterceptor\] (default task-24) (Hughes_Tre13342)
<OrderUpdateWithAccountInfoRequest xmlns:ns5=\"....\" API=\"Smile\"> <second ligne>" 
| rex field=raw "OrderUpdateWithAccountInfoRequest[^API]+API=\"(?<API>[^\"]+)"

Re: How can i extract keywords from my log as field values for field name API's

Sujithkumarkb — Fri, 10 Jan 2020 06:21:58 GMT

@richgalloway Thanks for the response , but this rex is extracting only the value "?xml" and not OrderUpdateWithAccountInfoRequest ,VinValidationRequest,GetEntitledRequest ...

Re: How can i extract keywords from my log as field values for field name API's

Sujithkumarkb — Fri, 10 Jan 2020 06:22:48 GMT

@TISKAR , thanks for your response but this rex is extracting only the value "Smile" and not OrderUpdateWithAccountInfoRequest ,VinValidationRequest,GetEntitledRequest ...

Re: How can i extract keywords from my log as field values for field name API's

TISKAR — Fri, 10 Jan 2020 08:29:33 GMT

Try this:

| makeresults
 | eval raw="2020-01-09 03:58:08,280 INFO \[com.hti.gw.interceptor.ServiceInterceptor\] (default task-24) (Hughes_Tre13342)
 <OrderUpdateWithAccountInfoRequest xmlns:ns5=\"....\" API=\"Smile\"> <second ligne>" 
 | rex field=raw "<(?<API>\S+)"

Re: How can i extract keywords from my log as field values for field name API's

Sujithkumarkb — Fri, 10 Jan 2020 09:26:45 GMT

@TISKAR , that works . This way i can extract one API at a time , But there are more than 400 APIs like OrderUpdateWithAccountInfoRequest ..
How can I generically run Rex and extract all APIs under the field API and save them as extracted field values

Re: How can i extract keywords from my log as field values for field name API's

TISKAR — Fri, 10 Jan 2020 10:16:28 GMT

@Sujithkumarkb try to use max_match option of rex:

  | rex max_match=50 field=raw ".*<(?<API>\S+)"

Example:

| makeresults
  | eval raw="2020-01-09 03:58:08,280 INFO \[com.hti.gw.interceptor.ServiceInterceptor\] (default task-24) (Hughes_Tre13342)
  <OrderUpdateWithAccountInfoRequest xmlns:ns5=\"....\" API=\"Smile\"> 2020-01-09 03:58:08,280 INFO \[com.hti.gw.interceptor.ServiceInterceptor\] (default task-24) (Hughes_Tre13342)
  <TEST xmlns:ns5=\"....\" API=\"Smile\">" 
  | rex max_match=50 field=raw ".*<(?<API>\S+)"

Re: How can i extract keywords from my log as field values for field name API's

richgalloway — Fri, 10 Jan 2020 11:52:22 GMT

Interesting. There is no "?xml" in your sample data so, of course, that was not tested. Is there anything else about the data we should know?

Re: How can i extract keywords from my log as field values for field name API's

Sujithkumarkb — Fri, 10 Jan 2020 12:42:39 GMT

There is ?xml in the data , but that being extracted as the field value is not expected .
The expected extraction is OrderUpdateWithAccountInfoRequest ,VinValidationRequest,GetEntitledRequest etc shown on the raw data .

The data is api requests that are made , where i am trying to capture the API names and once filtered for it on dashboard , i will be populating its respective responses .
for example OrderUpdateWithAccountInfoRequest is a API request which i would want to add as a value in multselect and once it is selected i would populate the OrderUpdateWithAccountInfoResponse xml from raw data as output.