topic "search base=X" not working with append in Splunk Search

"search base=X" not working with append

ChrisCLewis — Wed, 09 Jan 2019 12:29:55 GMT

I am using the "search base=X" approach to generate stats.

When I try to run two searches using append (or join etc) I am only getting stats from the first listed query, even if I change the order of their order. I can run the queries in separate panels and get results so am confident that the issue isn't with the searches themselves.
Multiple cannot be used in the same panel (I have tried loadjob without success).

Many thanks for any pointers / tricks I have missed in getting "search base=X" and append to work together

Re: "search base=X" not working with append

renjith_nair — Wed, 09 Jan 2019 12:38:03 GMT

@ChrisCLewis,
- Do you have a common field in both search which is used in the stats grouping? If not , rename one of them.
- If there are null values in the group by field, fillnull them with a value.

Please provide the search you are using if possible after masking any confidential data.

Re: "search base=X" not working with append

ChrisCLewis — Wed, 09 Jan 2019 12:48:24 GMT

Many thanks for the speedy reply.
All field names are "in common" and have just updated with fillnull - unfortunately still only getting the results from the first query.

Re: "search base=X" not working with append

renjith_nair — Wed, 09 Jan 2019 13:02:53 GMT

Do you mind sharing the search you are currently using with append?

Re: "search base=X" not working with append

ChrisCLewis — Tue, 29 Sep 2020 22:43:49 GMT

This is the append query, I originally tried it with |table instead of |fields but makes no difference.

Re: "search base=X" not working with append

renjith_nair — Wed, 09 Jan 2019 13:23:13 GMT

Alright. So there is no grouping at the end of the search by combining both results. so initial questions are invalidated and fillnull will not help.
By looking at the search only - created, code, description, month, number are the fields available in the output.
You had already mentioned that the searches are executing fine when run separately. Are these searches using same time range? Is that from time token or explicitly mentioned in the search using earliest & latest?

Just for testing, would you mind running the below search and check if you are able to see the dummy values ?

|search Q IN ("AB", "CD") | stats count as number by month
| eval code = "Q1"
| eval created = strftime(time(), "%Y-%m-%d %H:%M")
| eval description = "All about Q"
| fields created code description month uniques comment number volume
|append 
[|makeresults|eval created="dummy",code="Dummy",description="Dummy",month="Dummy",number="Dummy"|fields - _time]

Re: "search base=X" not working with append

ChrisCLewis — Wed, 09 Jan 2019 13:32:14 GMT

The time frame is defined in the base search, currently a token but have also used explicit range too.

The dummy values do appear with the |makeresults.

Re: "search base=X" not working with append

renjith_nair — Wed, 09 Jan 2019 13:45:11 GMT

Thanks Chris. Can we just run this as well and see the dummy value under field second appears in some of the events

|search Q IN ("AB", "CD") | stats count as number by month 
| eval code = "Q1"
| eval created = strftime(time(), "%Y-%m-%d %H:%M")
| eval description = "All about Q"
| fields created code description month number
|append 
[|search Q IN ("EF", "GH") 
| makemv delim=";" mvfield | mvexpand mvfield 
| eval check = mvfield 
| lookup some.csv check OUTPUT check as check_csv 
| where check like check_csv
| stats count as number by month 
| eval code = "R1"
| eval created = strftime(time(), "%Y-%m-%d %H:%M")
| eval description = "all about the R" 
| fields created code description month number|eval second="Dummy" ]
|table created code description month number,second

Re: "search base=X" not working with append

ChrisCLewis — Wed, 09 Jan 2019 14:09:41 GMT

Just run now and only results for Q are shown, nothing for R (including "second")

Re: "search base=X" not working with append

renjith_nair — Thu, 10 Jan 2019 05:50:49 GMT

Chris, its bit strange.
Are you really getting results for exactly same search if they are running separately? Also did the appended search run in search window and not in a dashboard?

Re: "search base=X" not working with append

ChrisCLewis — Thu, 10 Jan 2019 09:42:53 GMT

Good morning,
yes - getting exactly the same results when running separately - all I did was to copy the queries to a new dashboard and add in the append element.

The original searches worked whilst appended in search - the base search does a lot of work which means the queries are now greatly shortened and use different field names etc so they only work with the base search.

It is very odd, I've used append a lot but first time using base search

Re: "search base=X" not working with append

renjith_nair — Thu, 10 Jan 2019 10:00:51 GMT

Ok, so you are running it in a dashboard. What if you run the entire search with append in a search window?