Browscap TA not working in 6.1

[bruceclarke]

All,

We just updated our instance to Splunk 6.1 and our browscap add-on started returning an error:

Script for lookup table 'browscap_lookup' returned error code 1

I'm not the one who installed the add-on, but it looks like the add-on is only installed on the search head. Does this add-on also need to be included on the indexers? Is this something new with the way Splunk handles searches across search peers in 6.1?

Thanks!

[darrend]

Philip,

That's awsome, will check out 6.1.3 myself.

Thanks
Darren

[philip_wong]

Sorry, I maintained too different versions. I checked again today, it's negative. Seems 6.1.3 is not fixing this.

[philip_wong]

However, I have another problem.

E.g. "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"

Test result is telling me that the following fields would be able to retrieve ua_name,ua_category,ua_platform,ua_versionua_name,ua_category,ua_platform,ua_version...

But I can see only these 2 fields mostly.
ua_platform = Win7
ua_version = 10.0

$SPLUNK_HOME/bin/splunk cmd python ./browscap_lookup.py < ./test3.stdin
http_user_agent,ua_name,ua_category,ua_platform,ua_version,ua_version_major,ua_version_minor,ua_aol_version,ua_is_crawler,ua_is_mobile,ua_is_syndication_reader,ua_supports_tables,ua_supports_frames,ua_supports_iframes,ua_supports_java,ua_supports_javascript,ua_supports_vbscript,ua_supports_activex,ua_supports_cookies,ua_supports_css,ua_is_alpha,ua_is_beta
Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0),IE 10.0,IE,Win7,10.0,10,0,0.0,False,False,False,True,True,True,True,True,True,True,True,True,False,True

[philip_wong]

I just upgrade to 6.1.3 and don't see the error again.
So I assumed that Splunk has fixed the run path of external Python script. Though I couldn't find it mentioned in release notes

[darrend]

Dave,

I have used it extensively in v6 with a distributed environment to great success, but in 6.1 using your suggested work arounds, i cannot. I have never had to push the TA-browscap app to my cluster as a deployed app previously, but in 6.1 i get:

[indexer1] Script for lookup table 'browscap_lookup' returned error code 1. Results may be incorrect.

I thought i would try and push the app to all of my indexers incase that resolved the issue, but still received the same error using both of your workarounds.

However if i leave the setup as it was working in 6.0 and before, and implement your work arounds, it functions when i have all the result set on the search head prior to performing the lookup, but not if i try to perform the lookup at the indexing layer(as my dashboards had been configured to do previously).

I am using it as a normal lookup and not an automatic lookup.

Thanks
Darren.

[darrend]

Dave,

These work arounds do not appear to work when you are using a distributed infrastructure. I have got them working on a search head, that is part of a search head pool as long as i mangle my search to bring the data to my search head prior to performing the lookup, this is of course highly inefficient.

I have tried both of your work arounds on a clustered index environment, both by pushing into the slave-apps folder from the cluster master's master-apps, and updating the python script to point to the full path of the browscap.csv and also to push the app to each indexer manually. Neither approaches worked.

How's your bug report going? Have you filed it with Splunk support?

Thanks
Darren.

[philip_wong]

Darren,

Did you try to run "| lookup local=true browscap_lookup"? I guess it can limit the lookup only run on search heads.

[laithmurad]

I was having the problem Darren mentioned with clustered environment, doing this did fix the problem for me. | lookup local=true browscap_lookup

[dshpritz]

Hi darrend,
I have used TA-browscap in a distributed infrastructure in the past, with no issue. Are you using it as an automatic lookup?

I do have an open case with Splunk Support.

Thanks,

Dave

[dshpritz · Accepted solution]

It appears there may be a problem with TA-browscap in Splunk 6.1.x. I hope to look into it this week.

Thanks,

Dave

[areis]

Is there any update on this? 6.3.0 is out and it still doesn't work quite right. Had to use the aforementioned workaround to get it to process, but even then, I'm having issues.

[dominiquevocat]

uh, would not something like this
open(os.path.join(scriptpath, 'browscap.csv'))
work?

[philip_wong]

As I know geoip (Google Maps) app has the same problem

http://answers.splunk.com/answers/135685/geoip-command-stops-working-after-upgrade-to-611-geoip-data...

[bruceclarke]

Seems like it would be a bug to me. Why should every app need to update the $SPLUNK_HOME/bin/ folder?

Thanks for the response. The first workaround worked perfectly for me!

[dshpritz]

I wanted to follow up on this, and at least provide a workaround. It appears that as of 6.1, Splunk does not set the working directory of dynamic lookups to the /bin directory of the app, but instead sets it to $SPLUNK_HOME/bin. I am not sure if this is a bug or a feature, and have opened a ticket.
In the meantime, there are two workarounds:
1. Move the browscap.csv to $SPLUNK_HOME/bin/
2. Edit the TA-browscap/bin/browscap_lookup.py and look for the line that starts with "browscap = load_file", and change the path that says "browscap.csv" to the full path to the browscap.csv.

HTH,
Dave

[bruceclarke]

I guess it's also possible that the upgrade to 6.1 somehow deleted our instances of TA-browscap from the indexers, but this seems less likely. Just wanted to see if this is a known change.