[Tickets #10749] POST full text of the web page being bookmarked with the bookmarklet

[bugs at horde.org]

DO NOT REPLY TO THIS MESSAGE. THIS EMAIL ADDRESS IS NOT MONITORED.

Ticket URL: http://bugs.horde.org/ticket/10749
------------------------------------------------------------------------------
  Ticket             | 10749
  Created By         | Chuck Hagenbuch <chuck at horde.org>
  Summary            | POST full text of the web page being bookmarked with
                     | the bookmarklet
  Queue              | Trean
  Type               | Enhancement
  State              | New
  Priority           | 1. Low
  Milestone          |
  Patch              |
  Owners             |
------------------------------------------------------------------------------


Chuck Hagenbuch <chuck at horde.org> (2011-11-12 23:04) wrote:

To avoid not being able to save full content for the web page being  
bookmarked if it's password protected, grab it with javascript and  
send it along to trean. From  
http://www.randsinrepose.com/archives/2011/01/25/interview_marco_arment.html:
What part of Instapaper's infrastructure are you most proud of?

The bookmarklet has a mechanism to save pages from sites that require  
logins for full content, such as the Wall Street Journal and Harper's,  
by sending a copy of the page's HTML from the customer's browser to  
the server. It's like automating the "Save as..." menu item: if you  
have your own account for these sites and can see the page in your  
browser, you can save it to Instapaper.

The way it does this is ridiculous: instead of calling a simple GET  
request to save the page, since an entire page's contents would  
quickly overrun any URL-length limits in the stack, it injects a FORM  
with a POST action and populates a hidden value with the page contents.

But form-data requests from browsers aren't Gzip-compressed, so the  
resulting data is huge and needs to be sent over people's (often slow,  
often mobile) upstream connections. So I found an open-source DEFLATE  
implementation in Javascript - really - and the bookmarklet compresses  
the page data right there in the browser before sending it.

The whole procedure is hideously complex, but works incredibly well.