topic Re: Are hidden characters breaking my joins? in Splunk Search

Are hidden characters breaking my joins?

ttovarzoll — Wed, 22 Mar 2023 21:01:31 GMT

I am trying to build an Alert for login failures in AWS CloudTrail. In general I have it working -- but my joins are missing some of the desired events. Specifically, I am building an 'index' value consisting of the username+IP, e.g.

| eval user_IP = username + src_ip

but I now see that some seemingly-identical values are being evaluated as separate. For instance, when you click on the Selected Values view (left-side in the results) there will be 2 separate entries which -- at least on-screen -- appear to be identical.

WHAT THE POPUP SHOWS

user_IP
2 Values, 100% of events

Values                Count
firstuser172.31.1.1   2
firstuser172.31.1.1   1

I suspect there is a hidden character in the second Value. Or, maybe a trailing space (though there is none when I try adding each to the search).

----

How can I modify my 'eval' to generate values without hidden characters?

(I already tried adding a lower() function but without success)

Re: Are hidden characters breaking my joins?

yuanliu — Thu, 23 Mar 2023 05:03:41 GMT

One way to check is to bracket the string.

| eval user_IP = ">" . username . src_ip . "<"

Re: Are hidden characters breaking my joins?

PickleRick — Thu, 23 Mar 2023 05:36:06 GMT

If I remember correctly there were some issues with leading/trailing spaces when using the interesting fields @yuanliu 's idea is a way to verify it.

Re: Are hidden characters breaking my joins?

ttovarzoll — Thu, 23 Mar 2023 16:13:47 GMT

Aha! yes, that's a good trick. At first I didn't understand how that would fix the problem but then I realized that it was the perfect troubleshooting step -- and it demonstrated that there was a trailing space.

So, I have now modified my eval statement and confirmed that I am now receiving the missing events

| eval user_IP = username . trim(src_ip)

I also had to add matching trim() statements to several other references to 'username' throughout the SPL query.