Transaction endswith field breaks when I remove field from search, why?

[ra01]

When I run this search, Splunk returns one item for the "transaction"

eventtype=pageactions tag=external_traffic id=***** ip=****** 
EmailAddress=******@yahoo.com
| transaction id ip endswith=(EmailAddress=******@yahoo.com) maxspan=3m maxevents=3

But if I remove the EmailAddress value from the search it returns "no results found." Why?

eventtype=pageactions tag=external_traffic id=***** ip=****** 
    | transaction id ip endswith=(EmailAddress=******@yahoo.com) maxspan=3m maxevents=3

My goal is to find the logs preceding the log with the users' email address, and I get why I'm not getting any results back.

I add their email address (field, value pair) and it works, I remove it so I can see all the logs, not just the last log where their email address was passed through and it returns nothing. This doesn't make sense.

[jplumsdaine22]

This might be a job for stats rather than transaction. Have you tried the following?

eventtype=pageactions tag=external_traffic id=***** ip=******  EmailAddress=******@yahoo.com | stats values(_raw)  by id,ip,_time

This is a handy chart on when to use each aggregation command: http://docs.splunk.com/Documentation/Splunk/6.3.2/Search/Abouteventcorrelation

[ra01]

the key here is that only one event in the set of events I'm looking for has a EmailAddress field. I have records of people browsing a web site, and then at some point they submit a form with an email.

I want to look at what this user did prior to submitting that form. I'm using the EmailAdress in the endswith clause to work backwords from that event to see other events.

I've used the values() function for other things, but here I specifically need to see what events a user caused prior to submitting their email.

[jplumsdaine22]

ah gotcha - I didn't read the whole question!

[jkat54 · Accepted solution]

what happens if you add 'emailaddress=*' to the root search?

[ra01]

that will work, but my problem is I want to see the logs before the user passed their email address through this form.

So not all the logs will have that field. They'll all have an "id" and "ip" field I've defined for the transaction.

I can't understand why broadening the search by taking out the email field in the root search returns no results, but adding it in provides a result.

[jkat54]

If the ip and id are unique, you can remove the endswith condition. Does that work?

Then if you only want transactions where they did provide an email, you can add a '|search EmailAddress=*' after the transaction maybe. I think what's happening is related to lipsy.

The @ sign is a minor segmenter in sermenters.conf and this may be causing issues in lipsy world related to how the email address field is being stored in the index.

http://docs.splunk.com/Documentation/Splunk/6.0.8/admin/Segmentersconf

[ra01]

I'm specifically looking for the events prior to them submitting their email. If I don't provide an endswith clause then I don't see how I can return just the logs around that action point. For instance, someone might be browsing our web site, submit an email and keep browsing for another hour. Without an endwith clause that transaction will go on for an hour.

I only care about the few events just before they submitted that email.

Thanks for the info on the "@." I can't reach my splunk account on the weekend, but if that's an issue I might be able to change the field I'm using as an endpoint (maybe create a new field that's just the letters/numbers before the "@" in an email)

[jkat54]

So does that work like you mentioned, like the example below, etc.?

 eventtype=pageactions tag=external_traffic id=***** ip=****** 
 | transaction id ip endswith=yahoo.com maxspan=3m maxevents=3

[ra01]

yes!

and since there's a few different ways email can get into the logs, if I do endswith=("EmailAddress" AND "*****@yahoo.com") I get the logs I want.

Thank you.

[jkat54]

Awesome, then it is the segmenter '@' that is causing your pain AND you found out what I was going to say next! Good way to work around the info given!!!

So care to mark my answer as the solution? Pretty please!!! 😉 Upvotes help others find it quickly too. Cheers & pleasure working with you ra01!

[ra01]

I don't think a comment can be marked as a solution, I looked. You might need to paste the same in the "your answer" section at the bottom of the page.

and I think i updated all your comments.

[jkat54]

I noticed that right after asking you to... so I converted it to an answer now. Should be fine to mark as answer.