[Tickets #10749] POST full text of the web page being bookmarked with the bookmarklet

bugs at horde.org bugs at horde.org
Sat Nov 12 23:04:56 UTC 2011


Ticket URL: http://bugs.horde.org/ticket/10749
  Ticket             | 10749
  Created By         | Chuck Hagenbuch <chuck at horde.org>
  Summary            | POST full text of the web page being bookmarked with
                     | the bookmarklet
  Queue              | Trean
  Type               | Enhancement
  State              | New
  Priority           | 1. Low
  Milestone          |
  Patch              |
  Owners             |

Chuck Hagenbuch <chuck at horde.org> (2011-11-12 23:04) wrote:

To avoid not being able to save full content for the web page being  
bookmarked if it's password protected, grab it with javascript and  
send it along to trean. From  
What part of Instapaper's infrastructure are you most proud of?

The bookmarklet has a mechanism to save pages from sites that require  
logins for full content, such as the Wall Street Journal and Harper's,  
by sending a copy of the page's HTML from the customer's browser to  
the server. It's like automating the "Save as..." menu item: if you  
have your own account for these sites and can see the page in your  
browser, you can save it to Instapaper.

The way it does this is ridiculous: instead of calling a simple GET  
request to save the page, since an entire page's contents would  
quickly overrun any URL-length limits in the stack, it injects a FORM  
with a POST action and populates a hidden value with the page contents.

But form-data requests from browsers aren't Gzip-compressed, so the  
resulting data is huge and needs to be sent over people's (often slow,  
often mobile) upstream connections. So I found an open-source DEFLATE  
implementation in Javascript - really - and the bookmarklet compresses  
the page data right there in the browser before sending it.

The whole procedure is hideously complex, but works incredibly well.

More information about the bugs mailing list