ta_browscap_express: "Script for lookup table 'browscap_lookup_express' returned error code 1. Results may be incorrect." after upgrade to Splunk 6.4

[edwinmae]

TA-Browscap_express was working just fine (Linux / Splunk Server 6.3) but after the upgrade to Splunk Server 6.4 (also the Forwarders were updated to 6.4), it stopped working with the error:

Script for lookup table 'browscap_lookup_express' returned error code 1. Results may be incorrect. 

I re-installed the app many times (and also deleted the browscap_lite.csv from the var/run/splunk directory) without success

--

File (Fields) + Browser example that was retrieved

"GJK_Browscap_Version","GJK_Browscap_Version"
"6001","Tue, 07 Apr 2015 07:57:27 +0000"
"PropertyName","MasterParent","LiteMode","Parent","Comment","Browser","Browser_Type","Browser_Bits","Browser_M
aker","Browser_Modus","Version","MajorVer","MinorVer","Platform","Platform_Version","Platform_Description","Pl
atform_Bits","Platform_Maker","Alpha","Beta","Win16","Win32","Win64","Frames","IFrames","Tables","Cookies","Ba
ckgroundSounds","JavaScript","VBScript","JavaApplets","ActiveXControls","isMobileDevice","isTablet","isSyndica
tionReader","Crawler","CssVersion","AolVersion","Device_Name","Device_Maker","Device_Type","Device_Pointing_Me
thod","Device_Code_Name","Device_Brand_Name","RenderingEngine_Name","RenderingEngine_Version","RenderingEngine
_Description","RenderingEngine_Maker"
"DefaultProperties","true","true","","DefaultProperties","DefaultProperties","","0","","","0.0","0","0","","",
"","0","","false","false","false","false","false","false","false","false","false","false","false","false","fal
se","false","false","false","false","false","0","0","","","","","","","","","",""

"Mozilla/4.0 (compatible; MSIE 7.0; *Windows NT 6.1*WOW64*Trident/7.0*","false","false","IE 11.0 for Desktop",
"IE 11.0 in IE 7.0 Compatibility Mode","IE","Browser","32","Microsoft Corporation","IE 7.0 Compatibility Mode"
,"11.0","11","0","Win7","6.1","Windows 7","64","Microsoft Corporation","false","false","false","false","true",
"true","true","true","true","true","true","true","true","true","false","false","false","false","false","false"
,"false","3","0","Windows Desktop","Various","Desktop","mouse","Windows Desktop","","Trident","7.0","For Inter
net Explorer since version 4.0 and embedded WebBrowser controls (such as Internet Explorer shells, Maxthon and
 some media players).","Microsoft Corporation"

--

Any ideas what went and/or could be wrong?

[jschwanz · Accepted solution]

The TA-browscap_express app ships with browscap file format 6001. Somewhere between v6001 and v6014 they added three columns, and that breaks the way TA-browscap_express deals with the cache. I've already submitted a pull request to the Github repo for this change. To update your existing environment follow these steps:

1. Update your lookup definition with these new values: a. Command = browscap_lookup.py http_user_agent ua_propertyname ua_masterparent ua_litemode ua_parent ua_comment ua_browser ua_browser_type ua_browser_bits ua_browser_maker ua_browser_modus ua_version ua_majorver ua_minorver ua_platform ua_platform_version ua_platform_description ua_platform_bits ua_platform_maker ua_alpha ua_beta ua_win16 ua_win32 ua_win64 ua_frames ua_iframes ua_tables ua_cookies ua_backgroundsounds ua_javascript ua_vbscript ua_javaapplets ua_activexcontrols ua_ismobiledevice ua_istablet ua_issyndicationreader ua_crawler ua_isfake ua_isanonymized ua_ismodified ua_cssversion ua_aolversion ua_device_name ua_device_maker ua_device_type ua_device_pointing_method ua_device_code_name ua_device_brand_name ua_renderingengine_name ua_renderingengine_version ua_renderingengine_description ua_renderingengine_maker,ua_fromcache b. Supported fields = http_user_agent,ua_propertyname,ua_masterparent,ua_litemode,ua_parent,ua_comment,ua_browser,ua_browser_type,ua_browser_bits,ua_browser_maker,ua_browser_modus,ua_version,ua_majorver,ua_minorver,ua_platform,ua_platform_version,ua_platform_description,ua_platform_bits,ua_platform_maker,ua_alpha,ua_beta,ua_win16,ua_win32,ua_win64,ua_frames,ua_iframes,ua_tables,ua_cookies,ua_backgroundsounds,ua_javascript,ua_vbscript,ua_javaapplets,ua_activexcontrols,ua_ismobiledevice,ua_istablet,ua_issyndicationreader,ua_crawler,ua_isfake,ua_isanonymized,ua_ismodified,ua_cssversion,ua_aolversion,ua_device_name,ua_device_maker,ua_device_type,ua_device_pointing_method,ua_device_code_name,ua_device_brand_name,ua_renderingengine_name,ua_renderingengine_version,ua_renderingengine_description,ua_renderingengine_maker,ua_fromcache
2. Delete your browscap_lite.csv
3. Update browscap_list.csv.example for lines 2-4: "6014","Thu, 21 Apr 2016 09:16:00 +0000" "PropertyName","MasterParent","LiteMode","Parent","Comment","Browser","Browser_Type","Browser_Bits","Browser_Maker","Browser_Modus","Version","MajorVer","MinorVer","Platform","Platform_Version","Platform_Description","Platform_Bits","Platform_Maker","Alpha","Beta","Win16","Win32","Win64","Frames","IFrames","Tables","Cookies","BackgroundSounds","JavaScript","VBScript","JavaApplets","ActiveXControls","isMobileDevice","isTablet","isSyndicationReader","Crawler","isFake","isAnonymized","isModified","CssVersion","AolVersion","Device_Name","Device_Maker","Device_Type","Device_Pointing_Method","Device_Code_Name","Device_Brand_Name","RenderingEngine_Name","RenderingEngine_Version","RenderingEngine_Description","RenderingEngine_Maker" "DefaultProperties","true","true","","DefaultProperties","DefaultProperties","","0","","","0.0","0","0","","","","0","","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","false","0","0","","","","","","","","","",""
4. (optional) Update default/transform.conf with the new values from step 1

[ontkanin]

Thank you @jschwanz, this is exactly what has fixed my problem as well.

[edwinmae]

Awesome

Works like a charm.
Thanks!

Great App

[jschwanz]

I started getting this when I updated the browscap.csv file today. Even after reverting to a previous file I'm still getting this error.

Edit: There are three new fields in the browscap.csv. I'm verifying some changes and will update with my results and any necessary changes.

[edwinmae]

It seems when I remove the 'cache file' from /opt/splunk/var/run/splunk, the first search is OK

ua_activexcontrols
false

ua_alpha
false

ua_aolversion
0

ua_backgroundsounds
false

ua_beta
false

ua_browser
Chrome

ua_browser_bits
32

ua_browser_maker
Google Inc

ua_browser_type
Browser

ua_comment
Chrome 50.0

ua_cookies
true

ua_crawler
false

ua_cssversion
3

ua_device_code_name
Windows Desktop

ua_device_maker
Various

ua_device_name
Windows Desktop

ua_device_pointing_method
mouse

ua_device_type
Desktop

ua_frames
true

ua_fromcache
false

ua_iframes
true

ua_ismobiledevice
false

ua_issyndicationreader
false

ua_istablet
false

ua_javaapplets
false

ua_javascript
true

ua_litemode
false

ua_majorver
50

ua_masterparent
false

ua_minorver
0

ua_parent
Chrome 50.0

ua_platform
Win10

ua_platform_bits
64

ua_platform_description
Windows 10

ua_platform_maker
Microsoft Corporation

ua_platform_version
10.0

ua_propertyname
Mozilla/5.0 (Windows NT 10.0*WOW64) AppleWebKit/* (KHTML* like Gecko) Chrome/50.Safari/

ua_renderingengine_description
a WebKit Fork by Google

ua_renderingengine_maker
Google Inc

ua_renderingengine_name
Blink

ua_tables
true

ua_vbscript
false

ua_version
50.0

ua_win16
false

ua_win32
false

ua_win64
true

---

second search states "Script for lookup table 'browscap_lookup_express' returned error code 1. Results may be incorrect" - Game Over

[edwinmae]

More info : I have zero ua_fields