Subsearches (and custom fields?)

[rickschultz]

I could be doing something wrong, but I can't seem to get subsearches to behave like I expect. I can get something like the documentation (HowSubsearchesWork) example to work, but anything more complicated seems to fail.

This query:

index="main" *CONNECTION | top host limit=1 | fields host

shows the host with the most CONNECTION log entries.

As expected, this query:

* [search index="main" *CONNECTION | top host limit=1 | fields host]

shows all log messages from the host that has the most connection logs. When I try using a different fields, however, the behavior changes.

For example, this query shows the most frequent UUIDs (a custom field):

index="main" *CONNECTION | top UUID limit=1 | fields UUID

The following all return "No matching events found.":

* [search index="main" *CONNECTION | top UUID limit=1 | fields UUID]
* [search index="main" *CONNECTION | top UUID limit=1 | fields UUID | rename UUID as query]
* [search index="main" *CONNECTION | top UUID limit=1 | fields UUID | rename UUID as search]

Pasting the output from either of

index="main" *CONNECTION | top UUID limit=1 | fields UUID | format
index="main" *CONNECTION | top UUID limit=1 | fields UUID | rename UUID as search | format

into a new splunk search produces the expected results.

Could this be a syntax or configuration issue, or do I not understand how subsearches work? We're on 4.1.3; could this be related to SPL-32669 ?

thanks in advance,

rick

[Lowell]

What happens when you put a "format" in the subsearch? Like does this work:

index=main [search index="main" *CONNECTION | top UUID limit=1 | fields UUID | format ] 

I've found times where my subsearch will not work without tacking on a | format on the end, I'm not sure why, and it doesn't seem like you should have to. Perhaps someone more familiar with subsearches help explain when you need format and when you do not.

---

Another thing to look into is using the "Job Inspector" and looking at the "remoteSearch" value. You should see "litsearch" followed by the expanded form of your search. You may find something interesting going on here that could explain why your subsearch isn't working properly.

[rickschultz]

Search Job Inspector shows the following, though I'm not sure how to interpret it:

remoteSearch | fields keepcolorder=t * "*" "host" "index" "source" "sourcetype" "splunk_server"

[rickschultz]

index=main [search index="main" *CONNECTION | top UUID limit=1 | fields UUID | format ]

also yields "No matching events found."