Why are some tokens not expanding in incident review even though the fields are present in the notable event?

[dflodstrom]

Splunk Version 7.3.2, ES Version 5.3.1

Post-upgrade a couple of our notables are displaying tokens in the notable title rather than expanding to the values of those fields from the notable event; I can see these fields and their values when I search index=notable. I've re-created these rules manually and the issue exists in the new rules too. I've also tried running "notable | expandtoken field1 field2" and the tokens do not expand, I see $tokens$ in the Title and Description. I've seen this issue in the past when a token is misspelled and therefore the field doesn't exist, like $desst$.

Any suggestions are appreciated.

[dflodstrom]

bump. no answers from the community or support.

[DavidHourani]

wow you still have the issue ? I thought you had it fixed 😞

What have you tried doing so far ?

[DavidHourani]

Hi @dflodstrom,

Have a look here at the limitations :

https://docs.splunk.com/Documentation/ES/6.0.0/Admin/Expandtoken

Could it be that you have an underscore or a delimiter in your token?

Cheers,
David

[dflodstrom]

The limitations listed in that doc do not apply to my case:
"The search command does not support token delimiters in the middle of a field name." and "If you have tokens dependent on the expansion of other tokens, those tokens might not be reliably expanded because you cannot specify the order in which tokens are expanded."

[DavidHourani]

What happens if you rename the fields that are failing in the newly re-created rule ? Could be a long-shot, but could it be that the fields have special characters in them that somehow aren't supported anymore ?

[dflodstrom]

The field names themselves have no spaces or special characters; $signature$, $dest$ The values do sometimes have special characters, especially signature, but not always but the token expansion fails regardless of the values.