topic Search reads first event instead of desired event in Getting Data In

Search reads first event instead of desired event

crobicha — Fri, 04 Nov 2011 21:46:42 GMT

I am using the Google Analytics Data Export API to pull some data down into a log file so it can be indexed by splunk. Data is printed out in the following format:

ga:eventCategory=ViewChange | ga:eventAction=Grid ga:totalEvents=6 ga:uniqueEvents=2

ga:eventCategory=SessionEvent | ga:eventAction=UserLogin ga:totalEvents=13 ga:uniqueEvents=9

My search for counting logins is:

source="googleanalytics.txt" "ga:eventCategory=SessionEvent | ga:eventAction=UserLogin" | extract kvdelim="=" | timechart span=1d sum(totalEvents) as "Total Logins", sum(uniqueEvents) as "Unique Logins"

The problem is that the search is taking the first occurrence of ga:totalEvents, regardless of if it is a UserLogin event or not.

Edit: To be more clear, for the above example the timechart displays 6 total, 2 unique logins instead of the expected 13 total, 9 unique. The pipe inside the quotes is read as a search character, but I have removed it just to make sure, am seeing the same result when just searching for "ga:eventAction=UserLogin"

Re: Search reads first event instead of desired event

Ayn — Sat, 05 Nov 2011 08:47:33 GMT

You must have mispasted your search - the "|ga:eventAction" would be a syntax error as Splunk would try to interpret that as a search command. Please check.

Re: Search reads first event instead of desired event

tgow — Sun, 06 Nov 2011 21:17:19 GMT

Why don't you try this instead:

source="googleanalytics.txt" (ga:eventCategory="SessionEvent" OR ga:eventAction="UserLogin") | extract kvdelim="=" | timechart span=1d sum(totalEvents) as "Total Logins", sum(uniqueEvents) as "Unique Logins"

Re: Search reads first event instead of desired event

crobicha — Mon, 07 Nov 2011 15:39:59 GMT

I've run this in the search window and it does work, because it is in quotes splunk must recognize that it is a literal string and not a pipe

Re: Search reads first event instead of desired event

crobicha — Mon, 07 Nov 2011 15:49:40 GMT

I may switch to this syntax since it is more clear and doesn't use the pipe, but this doesn't fix my issue.

Re: Search reads first event instead of desired event

crobicha — Fri, 18 Nov 2011 17:10:36 GMT

I ended up using a regex after spending way too much time messing with sourcetypes and props.conf, this is my final search:

source="googleanalytics.txt" "ga:eventCategory=SessionEvent | ga:eventAction=UserLogin" | rex field=_raw "ga:eventAction=UserLogin[\s]ga:totalEvents=(?.)[\s]ga:uniqueEvents=(?.)" | eval _time = _time - 172800 | timechart span=1d sum(totalEvents) as "Total Logins", sum(uniqueEvents) as "Unique Logins"

The eval _time statement is because I haven't gotten splunk to pick up the timestamp in the log file properly, instead it timestamps the date when the script is run. GA data isnt guaranteed accurate until 48 hours later so the script pulls from the 24 period starting 3 days ago.