On our forwarders we have a [default]
_meta
value that specify a few key::value
pairs, e.g. a key to tell us what site this server forwarding the data belongs to (e.g. site::staging
).
When I search for site=staging
I get fewer results than with site::staging
, and I can't seem to find relevant documentation that explains why and it isn't easy to determine what is actually being filtered in the first compared to the latter when I look at the results (no easy/logical at-a-glance answer).
If I take a specific host and a simple file we monitor, like host=someserver source=/var/log/messages
- the result set is vastly different, with site::staging
containing what seems to be the entire log and site=staging
giving just a few lines from the log (and those lines don't contain text which mention staging
either). I can't think of any logic that would dictate why just a few lines seem to match site=staging
- I'd expect that field to either be searchable or not, not some odd subset of data. Also the UI highlights the source
field in the events as matching staging
when I do site::staging
which points to it thinking that it knows what I want. This makes it even more annoying when using the UI to add a field to the search, because when you click the site
field and "add to search" it adds it as site=staging
which doesn't yield all the results.
This is with SplunkCloud if that makes any difference.
In terms of syntax, the :: is obviously to use only indexed fields and the = should use indexed or non-indexed as per the documentation
Write better searches : Use_indexed_and_default_fields
Effectively if you are using = you are looking for something extracted at search time, if you use :: you are looking for an indexed field.
Use fields to retrieve events from this documentation:
When searching for default field values and custom indexed field values you can use the standard <field>=<value> syntax. This syntax matches default fields, custom indexed fields, and search-time fields.
If you are not seeing all the results with = but you see it with :: I'd log a support case, have you tested in smart mode and fast mode just in case to see if there is a difference?
Fast Mode vs Smart Mode doesn't make any difference - we'll lodge a support case and I'll update with any answer.
I'm running into this same issue and have not found a solution. Was Splunk Support able to help you resolve it?
https://answers.splunk.com/answers/411019/whats-the-difference-between-hostabc-and-hostabc.html
have you seen this answer?
It helps from a background of where ::
came from, but doesn't really explain the issue.