All Apps and Add-ons

Website input add-on: How do you change the user agent string?

New Member

Hi,

How do you change the user agent string of website input?

Can this be included in the data inputs interface?

Thanks.

0 Karma

Champion

It doesn't yet, but I can certainly add the ability to set this.

Could you provide some context regarding why you need this? I just want to know more details because that could influence how I design this feature.

Let me know and I'll get stated on this; it shouldn't be too hard.

Update:
This has been added in version 1.1. You can specify the user-agent string on the input manager page or select from a dialog which includes a list of user-agents.

New Member

Hi Luke, Thanks for developing this app and supporting it.

I am attempting to scrape a website and receive a HTTP 403 forbidden. I encountered a similar issue when I was using liburl2 in a python script. The website in question is blocking the user agent string so I was looking for a work around. I thought you might be using something similar so a quick edit of one of your scripts may be an option if possible. If you decided to add this into the tool, you may have a drop down for common user agent strings, that can also allow people to modify them.

0 Karma

Champion

I opened a ticket to cover this work, see: http://lukemurphey.net/issues/1034