Your question seems a bit ambiguous, saying you are out of luck? What is the problem? Do you get a different answer completely than when you view a page in a browser? If so, then do what @cam says and parse the headers, you can do it in Firefox through the extension, or you can do it in Chrome natively. In any case, try to imitate the headers you see in any browser you make in your user agent mechanism. Here is the script that I used to map iTunes request headers when I was processing data in the app store:
def mimic_itunes( mech_agent ) mech_agent.pre_connect_hooks << lambda {|headers| headers[:request]['X-Apple-Store-Front'] = X_APPLE_STOREFRONT; headers[:request]['X-Apple-Tz'] = X_APPLE_TZ; headers[:request]['X-Apple-Validation'] = X_APPLE_VALIDATION; } mech_agent.user_agent = 'iTunes/9.1.1 (Windows; Microsoft Windows 7 x64 Business Edition (Build 7600)) AppleWebKit/531.22.7' mech_agent end
Note: the constants in the example are just strings ... actually this is not so important if you know you can add any string there
Using this approach, you should be able to modify / add the headers that the web application may need.
If this is not the problem you are facing, send more details about what exactly is happening.
user131441
source share