How to get an AJAX request request to wait for a page to be displayed before returning a response?

Question

I am writing Greasemonkey Script for pages in this site (Site1). Site1 has offers and offers of various kinds, and my GM Script aims to do the following:

When you visit the offer on Site1, Script asks Site2 to find out if this hotel is listed on Site2. If so, display the search results from Site2 on Site 1.

The problem is that Site2 displays a progress bar ("Download Results") and then displays the results. So my Ajax request always returns empty results and looks like this (see Reddish part):
(Click to enlarge)

However, in fact, it should have the full content of the search results from site2, for example:
(Click to enlarge)

I tried synchronous Ajax request as well as GM_xmlhttpRequest no avail.

This is a problematic progress indicator on site 2:
(Click to enlarge)

How can I make an AJAX request wait for a search on Site2 to be fully displayed before returning a response to Site1?

For reference, my full working Script code is on pastebin.com .

This is the corresponding snippet:

 $(document).ready(function(){ var rewardsSiteResults = $('<div class="panel deal-panel rc-lr"></div>').attr('id', "rewardsSiteResults") .html("<p>" + progressMessageText + "</p> ").append(spinnerGif); $(insertSelector).after(rewardsSiteResults); var addressMap = getAddressOfHotel(); var pinCode = addressMap[pinCodePlaceHolder]; var hotelName = addressMap[hotelNamePlaceHolder]; var queryURL = constructQueryURL(pinCode, hotelName); $.ajaxSetup({async:true, timeout: 5000}); $.get(queryURL,null, function(response) { if(!displayed){ displayed=true; //rewardsSiteResults.html("adfaasddsf"); var text = $(response).find("#col2"); $(text).find("script").remove(); //console.log(text.html()) // $('<iframe id="someId"/>').appendTo('#rewardsSiteResults') // .contents().find('body').append(response); rewardsSiteResults.html("<div class='panel deal-panel rc-lr'>" + text.html() +"</div>"); //console.log(response); } },'html'); }); 

Answer 1

In order for AJAX to “wait for the page to display”, you will actually need to completely process the page, load and run all the included CSS and javascript files. This is not easy and is not recommended. Fortunately, you still don't need to do this.

Here are three best ways to solve this problem:

• The resource page (mpdining.rewardsnetwork.com, for this question) may have an API. If so, find it and use it. This is your best choice, if available.

• Analyze the resource's JavaScript page and / or AJAX requests. Use GM_xmlhttpRequest() to directly retrieve only payload data, rather than trying to parse a resource page.

  Sometimes this process is quite simple, but some sites require complex interaction and / or authentication.

• Load the resource page in a hidden iframe; install the Greasemonkey script to run both on the resource page and on the main page and to transfer the desired data using postMessage() .

  This approach will almost always work, although you may have to prevent some pages from trying to “throw out” the iframe.

---

Using a hidden iframe to retrieve data from a cross-domain resource page:

Greasemonkey scripts will work on a regular page as well as on pages inside an iframe. In fact, you can install the same script to run both on both domains and on multiple domains.

If the main page and iframed resource page work with GM script (s), the script instances can interact with each other, domain, using postMessage() .

For example, suppose we have a site fiddle.jshell.net/9ttvF/show that contains travel data, and we want to mash this site with the corresponding data from the resource site, jsbin.com/ahacab , which uses AJAX to get the data payload.

The target (main) site is as follows:

The resource site looks like this: first

Then ends like this:

Next script:

• Loads a resource page in a hidden iframe.
• Launches the second instance, which runs on the iframed page.
• Waits for the completion of the iframed page, processing the results as desired.
• Sends the required payload data to the GM script on the target (main) page.
• The landing page script then inserts the payload data to complete the markup.

 // ==UserScript== // @name _Cross-site, AJAX scrape demo // @include http://fiddle.jshell.net/9ttvF/show/ // @include http://jsbin.com/ahacab* // @require http://ajax.googleapis.com/ajax/libs/jquery/1.8.3/jquery.min.js // @require https://gist.github.com/raw/2625891/waitForKeyElements.js // @grant GM_addStyle // ==/UserScript== if (/fiddle\.jshell\.net/i.test (location.host) ) { console.log ("***Master-page start..."); /*--- Inform the user. */ $("#plainResults").before ( '<div id="gmFetchRez">Greasemonkey is fetching results from jsbin.com...</div>' ); /*--- Setup to process messages from the GM instance running on the iFrame: */ window.addEventListener ("message", receiveMessage, false); /*--- Load the resource site in a hidden iframe. */ $("body").append ('<iframe src="http://jsbin.com/ahacab" id="gmIframe"></iframe>'); } else { console.log ("***Framed start..."); /*--- Wait for the AJAXed-in content... */ waitForKeyElements ("#results table.rezTable", sendResourcePageData); } function sendResourcePageData (jNode) { console.log ("Results found! Sending them to the main window..."); window.top.postMessage (jNode.html(), "*"); } function receiveMessage (event) { if (event.origin != "http://jsbin.com") return; $("#gmFetchRez").html (event.data); } //--- Use CSS to control appearances. GM_addStyle ( " \ #gmIframe { \ display: none; \ } \ #gmFetchRez { \ background: lightYellow; \ border: 3px double red; \ padding: 1em; \ } \ " ); 

---

The end result is as follows: when running the script: