Searches not working across apps

[lclayton95]

I have two of the exact same searches and one works within the search app but not this custom internal app that package the savedsearch.

The search works for both apps until the where command is introduced. 

| eval delta_time = delete_time - create_time, hours=round(delta_time/3600,2)\
| where delta_time < (48 * 3600)\

This returns results in the search app but not in the app that houses this alert. The app is shared globally and all the objects within it. I also have the admin role with no restricted indexes or data.

[PickleRick]

There are no miracles.

I understand that when you add the "where" command you stop getting any results.

That would mean that either delta_time is calculated differently and for some reason its values are never in the desired range (which is very very unlikely) or that delta_time field is not getting properly evaluated in the preceeding step (which is much more likely).

The easiest way to check it would be to run it up to the where command (but without it) and check the contents of delta_time field.

If it is not defined in the app-embeded search, check the values of the fields which the delta_time field is supposed to be based on. They might either be not/wrongly extracted or - a counterintuitively - they might be extracted "too good" and ending up being multivalued fields (or string fields - sometimes Splunk doesn't recognize numbers properly and you have to explicitly call tonumber() on them but that would be surprising if it happened in one case and not the other).

[lclayton95]

Seems like this issue is due to a transaction command that is not combining the events as intended. This then breaks the search when the other lines are added, however it does not app in another app which leads me to believe the field extraction is not happening properly.

[sainag_splunk]

I believe the issue might be related to field extractions. There's likely a field called delta_time or delete/create in the Search app that isn't set to global for all apps. To troubleshoot:

1. Inspect the search.log file.
2. Look for entries containing "lispy".
3. Examine the search TERMS in these entries.
4. See if you can find anything related to the fields mentioned above.

This approach might help you identify why the search isn't working as expected for users without direct index access. If you find that certain fields aren't available globally, you may need to adjust their extraction settings.

[lclayton95]

I don’t have any field extraction called delta_time it was created with the eval command. I tried searching all configurations and all permissions seem to be set correctly 

[sainag_splunk]

The only reason I can think of for this issue is a permission conflict problem. Did you look at the search.log as mentioned? Try comparing the search.log files for the working and non-working instances. Without knowing the full search details, it's hard to validate exactly what's going on. There must be settings defined for this sourcetype. Try running the btool command and see if you can find anything relevant there.

splunk btool props list sourcetype --debug

Hope this helps

[PickleRick]

Sufficiently modern btool supports --app and --user switches letting you compare effective configs in different search-time contexts (caveat - doesn't consider permissions)