All Apps and Add-ons

Splunk Search does not return all event data on a field

mgarciar
Path Finder

I'm facing a very strange issue in my Splunk search. I have a data input coming from a REST API that returns a multi-level (nested) JSON response:

REST API Response

The entity node has several nodes, each node represents one access point. Each access point contains a field called ipAddress. This API is being called every 5 min and response stored in Splunk. When I do a search to get the list of IP Addresses from one event I don't get all of them. For some reason, is like Splunk is reading only the first seven nodes inside entity, because when I do:

source="rest://AccessPointDetailsAPI" | head 1

Splunk shows only the following values on the field (7 values although there are around 27😞

Splunk Field data sample

I'm using demo license if that matters. Why I cannot see all values ? If I change my search to look for a specific iPAddress on the API response but that is not on the Splunk list of field values I get no records.

It's like the search does not get all the values on the event for some fields.

Thanks and regards,

0 Karma
1 Solution

mgarciar
Path Finder

I think I understand the problem now. So the event is a big json and Splunk is not properly parsing all fields on the big json.

We need to tell splunk to parse the specific field we need with spath and specifying the field:

yoursearch | spath output=myIpAddress path=queryResponse.entity{}.accessPointDetailsDTO.ipAddress | table myIpAddress

http://docs.splunk.com/Documentation/Splunk/5.0.4/SearchReference/Spath

But I think also is important to analyze if maybe the data input needs to be divided in multiple events rather than a single huge event.

View solution in original post

mgarciar
Path Finder

I think I understand the problem now. So the event is a big json and Splunk is not properly parsing all fields on the big json.

We need to tell splunk to parse the specific field we need with spath and specifying the field:

yoursearch | spath output=myIpAddress path=queryResponse.entity{}.accessPointDetailsDTO.ipAddress | table myIpAddress

http://docs.splunk.com/Documentation/Splunk/5.0.4/SearchReference/Spath

But I think also is important to analyze if maybe the data input needs to be divided in multiple events rather than a single huge event.

swapnil_wadkute
Engager

It works like a charm. Thank you 🙂

0 Karma

swapnil_wadkute
Engager

I am also using REST API to get data in Splunk and facing same problem. Did you get any solution on this?

0 Karma

mgarciar
Path Finder

I think so ... check my answer

0 Karma

richgalloway
SplunkTrust
SplunkTrust

It sounds like the data is not getting indexed properly. How are you getting the data into Splunk?

---
If this reply helps you, Karma would be appreciated.
0 Karma

mgarciar
Path Finder

I'm using the REST API app (https://splunkbase.splunk.com/app/1546/)

The event is showing all the data in splunk. Actually, if I do a search like this:

source="rest://PRIME_AccessPointDetailsAPI" | search "11.111.122.33"

Splunk will find that data, but if I do this

source="rest://PRIME_AccessPointDetailsAPI" | search queryResponse.entity{}.accessPointDetailsDTO.ipAddress="11.111.122.33"

I won't get any data back, unless the ipAddress value I use is on the list Splunk gives me on the field (second image)

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...