Ironport WSA "Top Google Search Terms" query

[imarks001]

I am trying to come up with a search that would parse Google search queries made though my Ironport web proxy. I would like to add this as a real time report on a dashboard eventually. Does anyone have any tips for a regex that could parse the a Google search request URL and display on the words in the search? Bing results would be great too.

Thanks

[pschulz]

If someone's stumbling upon this, I've made a simple field extration regex which does the job by filtering out all text between ampersands (&) where "q=" is showing up.

Just go to "Manager" - "Field extractions" - Click "New" and enter the line below in the field "Extraction/Transform" after choosing your app & source/sourcetype/host:
(?i)&q=(?P<_google_query>\w+[^&]*)&

Regards,
Phil

PS: Maybe someone will cure the bug of not beeing able to post "<thisisnotag!!!>" as it's regarded as html-tag 😉

[tidavids]

Assuming you have the WSA app installed that extrapolates the cs_url field, try the following:

"www.google.com/search" "q=" | rex field=cs_url "q=(?P<search>[^&]*)\&" | rex field=search mode=sed "s/\+/ /g" | top search

[nkpiquette]

Digging up an old threat, but this just came in handy. The sed I had to use in 6.2 was as follows:
rex field=search mode=sed "s/+/ /g"

[imarks001]

Thanks, I tried this but I receive the following error. 1. Error in 'rex' command: Encountered the following error while compiling the regex 'q=(?P[^&]*)&': Regex: unrecognized character after (?P

[imarks001]

The URL is getting truncated when I post it.

Here is everything after google.com

search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&gs_rfai=Cqt3eklUiTPiZO4XyzASIzfSXBgAAAKoEBU_Q0UPQ&fp=d2eda4910a7445ea

[imarks001]

Sorry for the confusion. I would be looking to parse the actual search query out of our WSA logs. For instance, I would want to pull "world cup winners" out of the following log entry and be able to report on top searches.

1277318549.935 295 192.168.150.6 TCP_MISS/200 9957 GET http://www.google.com/search?hl=en&source=hp&q=world+cup+winners&aq=3s&aqi=g-s10&aql=&oq=word+cup+&g... - DIRECT/www.google.com text/html DEFAULT_CASE-NoAuthPolicy-NoAuthGlobal-NONE-NONE-DefaultRouting

[Simeon]

Your question is confusing as there is no sample data to reference. Also, your use of parse is confusing. My guess is that you want to produce an extracted field for query terms within google and bing URLs. Please update your question with more details.