topic Re: Why is search not returning result when using map? in Splunk Search

Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 16:55:42 GMT

Hello,
My following search results records for Account:

index="X" AND (sourcetype="A:Proxy" OR sourcetype="A:orderpusher")          
| where NOT isnull(Account)
| table _raw, _time, Account, User

But if i use map (which is required for my xml dashboard form ), then I do not get any result:

| makeresults
| map search="search  index=neonconnect_dev AND (sourcetype=NeonConnect:Proxy OR sourcetype=neonconnect:orderpusher)"
| where NOT isnull(Account) 
| table _raw, _time, Account, User

My source data contains required data(file), still I can't see result. Kindly help.
Thanks.

Re: Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 17:38:58 GMT

Correction: In both above queries I use same Index and SourceType (though it seems different in my question)

Re: Why is search not returning result when using map?

to4kawa — Fri, 06 Dec 2019 19:50:17 GMT

| makeresults
| map search="search index=_internal splunkd"

Hi, @sheikhazad
this query produces results.
I think | where NOT isnull(Account) is evil.

| makeresults
| map search="search index=neonconnect_dev  (sourcetype=NeonConnect:Proxy OR sourcetype=neonconnect:orderpusher)"
| table _raw, _time, Account, User
| search Account!=""

How about this?

Re: Why is search not returning result when using map?

woodcock — Fri, 06 Dec 2019 20:16:23 GMT

Tell us more about why map is required for my xml dashboard form. I do not believe that is true and the real solution is to NOT use map.

Re: Why is search not returning result when using map?

woodcock — Fri, 06 Dec 2019 20:18:08 GMT

It is almost certainly because you are not using the double-quotes for your sourcetype value. Try this:

| makeresults
| map search="search index=\"X\" AND (sourcetype=\"A:Proxy\" OR sourcetype=\"A:orderpusher\")
| where NOT isnull(Account)
| table _raw, _time, Account, User"

Be aware that this limits both the run-time of your search and the size of your results set.

Re: Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 20:25:31 GMT

Sadly it doesnt work. Pls see my main comment and I got result when I changed query and still need answers why changing my query works

Re: Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 20:46:03 GMT

Problem solved but still want answer:

My input source (which is log file) contains a lot of unnecessary data like:

2019-12-06 20:26:54,753 UTC : INFO PC=I, PM=0, PI=0, PR=3
2019-12-06 20:27:24,770 UTC : INFO PC=I, PM=0, PI=0, PR=3

And my log files are kept on updating with these data which DO NOT contain "Account" info which I am looking for.

I changed the query to limit the data in index by adding "AND OrderStatus AND User AND Account" which makes sure that the data contains Account info like:

2019-12-04 17:05:59,026 UTC : INFO User=ASHAH, Account=AShah, AccountId=2, OrderStatus=Unknown, Status=Pending

Old failed query:
| map search="search earliest=$$beginTimeFromMinTime$$ index=$$orders_index$$ AND (sourcetype="NeonConnect:Proxy" OR sourcetype="neonconnect:orderpusher") "

New successful query:
| map search="search earliest=$$beginTimeFromMinTime$$ index=$$orders_index$$ AND (sourcetype="NeonConnect:Proxy" OR sourcetype="neonconnect:orderpusher") AND OrderStatus AND User AND Account"

My question is, how limiting data in Splunk Index solved the problem? Is a lot of data in Splunk Index cause unreliable result?

Re: Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 20:52:42 GMT

Sadly it doesnt work. Pls see my main comment and I got result when I changed query and still need answers why changing my query works

Re: Why is search not returning result when using map?

sheikhazad — Fri, 06 Dec 2019 21:01:59 GMT

I want that all accounts are shown in my dasboard's drop down menu. Without map it doesnt work. Not sure why. I am 1 day old kid in splunk 😞

Re: Why is search not returning result when using map?

woodcock — Fri, 06 Dec 2019 22:02:59 GMT

Then let's solve that problem the right way. Trust me: map is not the answer. Show us your dashboard XML.

Re: Why is search not returning result when using map?

arjunpkishore5 — Sun, 08 Dec 2019 04:43:27 GMT

What exactly are you trying to achieve with map? Could you provide more details on what you're trying to achieve in the dashboard? There might be an alternative solution to it.

In the meanwhile, for your query, I suspect the metadata is getting altered when results are passed from map. Try enclosing the whole query in map

| makeresults
| map [ search index=neonconnect_dev  sourcetype IN ("NeonConnect:Proxy", "neonconnect:orderpusher")
  | where NOT isnull(Account)
  | table _raw, _time, Account, User]

Re: Why is search not returning result when using map?

sheikhazad — Wed, 11 Dec 2019 11:01:24 GMT

Here is my dashboard and I am showing relevant queries only. With this queries with map, my problem is solved:

    <query>
        | makeresults
        | addinfo
        | eval orders_index = "$Trading$"
        | eval beginTimeFromMinTime=relative_time(info_min_time, "@d")
        | map search="search earliest=$$beginTimeFromMinTime$$ index=$$orders_index$$ AND (sourcetype=\"NeonConnect:Proxy\" OR sourcetype=\"neonconnect:orderpusher\") AND OrderStatus AND User AND Account"
    </query>
    <earliest>$orderTime.earliest$</earliest>
    <latest>$orderTime.latest$</latest>
    <sampleRatio>1</sampleRatio>
</search>

........................................................................................
Below query is to show Users in drop down
.......................................................................................

        <input type="dropdown" token="tokFilterUser" searchWhenChanged="true">
            <label>Filter on User</label>
            <search base="baseProxySearch">
                <query>
                    | where NOT isnull(User)
                    | dedup User
                    | eval User=lower(User)
                    | dedup User
                    | table User
                    | sort User
                </query>
            </search>
            <fieldForLabel>User</fieldForLabel>
            <fieldForValue>User</fieldForValue>
            <choice value="*">All Users</choice>
            <default>*</default>
        </input>

........................................................................................
Below query is to show Accounts in drop down
.......................................................................................

Filter on Account

Re: Why is search not returning result when using map?

sheikhazad — Wed, 11 Dec 2019 11:02:34 GMT

Pls see my dashboard xml in main comment

Re: Why is search not returning result when using map?

sheikhazad — Wed, 11 Dec 2019 11:04:14 GMT

Unfortunately, when I enclose map with [ ] as you suggested, it fails to work. Pls see my dashboard xml in main comment