Filtering fields via REST API not working with parentheses

[tomapatan]

Hi Everyone,

I`m running a query via the Splunk REST API (using  Python), and need to filter events based on the following requirements:

- Always include events where TITLE is one of: A, B, C, D, E

- Only include events where TITLE=F and FROM=1 OR TITLE=G and FROM=2

This works fine in Splunk Web, but when sent via the REST API the conditional clause for TITLEs F and G don`t get applied correctly

Works via Splunk WEB and REST (without filtering based on FROM)

index=my_index System="MySystem*" Title=A OR Title=B OR Title=C OR Title=D OR Title=E OR Title=F OR Title=G

Works on WEB, not via REST (filtering based on FROM)

index=my_index System="MySystem*" Title=A OR Title=B OR Title=C OR Title=D OR Title=E OR (Title=F and FROM=1) OR (Title=G AND FROM=2)

I`ve tried to apply the filtering downstream, but the issue persists.

I’m unable to query a saved search because some fields are extracted at search time and aren’t available when accessed via the REST API. As a result, I need to extract those fields directly within the query itself when using the REST API. (Note: the TITLE field is being extracted correctly.)

Many thanks.

[tomapatan]

It turns out the issue wasn’t related to parentheses or evaluation order. The real problem was that the FROM field is only available at search time-so it worked in Splunk Web, but not through the REST API. I had to use an inline field extraction to get it working properly.

[PickleRick]

While your solution might work your understanding of the problem is wrong. However you're spawning your search (REST, WebUI, scheduler...) it's still a search-time operation (as the name says). So search-time operations are performed (inline extraction is still a search-time extraction BTW). Unless the search is run in wrong context making some knowledge objects unavailable.

That would suggest that your REST-spawned search was run in a different context (app/user) than the webui-spawned one. If the FROM field was extracted using a specific app to which the REST-spawned search had no access or was extracted privately for a specific user and the REST was spawned as another user - that resulted in FROM field not being properly extracted.

Use the btool, Luke.

[tomapatan]

Interesting-and you're right. The knowledge objects are actually defined in a different app than the one I'm using to run the API calls.

[PickleRick]

1. I'm assuming you are aware of the field names case sensitivity and your field isn't by any chance named From, from or FrOm.

2. Is your search initiated by API running in the same user/app context as the search spawned from web? It smells like some context mismatch resulting in wrongly/not extracted fields.

[PrewinThomas]

@tomapatan 

Can you try with below,

search_query = '''
search index=my_index System="MySystem*" (Title=A OR Title=B OR Title=C OR Title=D OR Title=E OR Title=F OR Title=G)
| eval include=if((Title="F" AND FROM="1") OR (Title="G" AND FROM="2") OR match(Title, "^[ABCDE]$"), 1, 0)
| where include=1 '''

Note:
since you are using python, hope you are using url encoding. Without encoding, the API may misinterpret or strip them.

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

[tomapatan]

Thanks, tried to filter downstream without success, unfortunately. I am using URL encoding.

[livehybrid]

Hi @tomapatan 

Is your first "and" lowercase in both examples? This should be uppercase, if its made to uppercase does it behave as expected or do you still get the issue? Im just wondering if the UI does some correction before running the litsearch.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[tomapatan]

Thanks, "AND" is uppercase in both examples, but the issue persists.

I followed your suggestion and checked the search job properties and the eventSearch changes to:

index=my_index System="MySystem*" (Title=A OR Title=B OR Title=C OR Title=D OR Title=E OR (Title=F FROM=1) OR (Title=G FROM=2))

Still not working via REST, unfortunately.