Splunk Search

How do I search for correlating events from Source log?

roderickjones
Engager

So we log API calls and response errors. However I'm having issues searching for the corrilating event from a log. 

 

Example:

Im searching for event related to and endpoint https://api.lyft.com/v1/paydisputes/oi/orders? and I'm about to get the log back of what was sent. 

Nov 24 16:20:17 ip-11-222-111-100 order-service: [2020-11-24 16:20:17.399] [INFO ] [417863580]: [reactor-http-server-epoll-8] c.v.o.o.m.s.HostedApiRestClient [] - Sending Request - Method: 'GET', Path: 'https://api.test.com/v1/paydisputes/oi/orders?arn=1363473846864745645&authorizationCode=039319&settlementAmount=12.17&authorizationAmount=12.17&settlementDate=2020-11-16T00:00:00Z&authorizationDate=2020-11-16T00:00:00Z&settlementCurrency=USD&authorizationCurrency=USD&creditCardBin=473702&creditCard4=1788&orderId=14738858272569688&transactionId=234235423523543&sellerId=42201', Headers: '{x-test-api-version=[1.4], Accept=[application/json], Authorization=[Bearer ******], Content-Type=[application/json]}' 

* host = ip-11-222-111-100
* index = syslog
* source = /var/log/syslog
* sourcetype = syslog

When I go to "Show Source" there is a associated event :

Nov 24 16:20:17 ip-11-222-111-100 order-service: [2020-11-24 16:20:17.512] [ERROR] [417863693]: [reactor-http-client-epoll-11] c.v.o.o.OrderGlobalWebExceptionHandler [] - Error occurred while processing web requestorg.springframework.web.server.ResponseStatusException: 404 NOT_FOUND "Failed to receive order from merchant hosted api with statusCode=404, Not Found"#012#011at com.verifi.orderservice.order.merchanthosted.service.HostedApiRestClient.handleResponse(HostedApiRestClient.java:68)#012#011at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:153)#012#011at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67)#012#011at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)#012#011at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:192)#012#011at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:192)#012#011at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:76)#012#011at .....

Nov 24 16:20:17 ip-11-222-111-100 #011org.springframework.web.server.ResponseStatusException: 404 NOT_FOUND "Failed to receive order from merchant hosted api with statusCode=404, Not Found"

 

I can't seem to make a search that would allow me to pull any errors associated with the GET call based on the next event that shows an error code. I think it's because they are not correlated in any way other than time possibly?

to4kawa
Ultra Champion

No example is not enough.

If reactor-http-server-epoll adds up, it is possible, I guess.

index=_internal | head 1 | fields _raw | eval _raw="Nov 24 16:20:17 ip-11-222-111-100 order-service: [2020-11-24 16:20:17.399] [INFO] [417863580]: [reactor-http-server-epoll-8] c.v.o.o.m.s.HostedApiRestClient [] - Sending Request - Method: 'GET', Path: 'https://api.test.com/v1/paydisputes/oi/orders?arn=1363473846864745645&authorizationCode=039319&settlementAmount=12.17&authorizationAmount=12.17&settlementDate=2020-11-16T00:00:00Z&authorizationDate=2020-11-16T00:00:00Z&settlementCurrency=USD&authorizationCurrency=USD&creditCardBin=473702&creditCard4=1788&orderId=14738858272569688&transactionId=234235423523543&sellerId=42201', Headers: '{x-test-api-version=[1.4], Accept=[application/json], Authorization=[Bearer ******], Content-Type=[application/json]}' "
| appendpipe [ eval _raw="Nov 24 16:20:17 ip-11-222-111-100 order-service: [2020-11-24 16:20:17.512] [ERROR] [417863693]: [reactor-http-client-epoll-11] c.v.o.o.OrderGlobalWebExceptionHandler [] - Error occurred while processing web requestorg.springframework.web.server.ResponseStatusException: 404 NOT_FOUND \"Failed to receive order from merchant hosted api with statusCode=404, Not Found\"#012#011at com.verifi.orderservice.order.merchanthosted.service.HostedApiRestClient.handleResponse(HostedApiRestClient.java:68)#012#011at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onNext(MonoFlatMapMany.java:153)#012#011at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67)#012#011at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:114)#012#011at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:192)#012#011at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:192)#012#011at reactor.core.publisher.MonoNext$NextSubscriber.onNext(MonoNext.java:76)#012#011a" 
| appendpipe [ eval _raw="Nov 24 16:20:17 ip-11-222-111-100 #011org.springframework.web.server.ResponseStatusException: 404 NOT_FOUND \"Failed to receive order from merchant hosted api with statusCode=404, Not Found\""]]
| rex "(?<time>\w{3} \d\d \S+) (?<ip>\S+)"
| rex "\[(?<status>ERROR|INFO)\]"
| rex "Method: (?<method>\S+),"
| rex "statusCode=(?<status_code>\d+)"
| stats values(method) as method values(status_code) as status_code values(status) as status by ip time

rush job



Get Updates on the Splunk Community!

Industry Solutions for Supply Chain and OT, Amazon Use Cases, Plus More New Articles ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...