Learning Search

Paywalls Everywhere You Go? Get to the Goodies With These Two Paywall Ladder Bookmarklets

On the one hand, I hate paywalls. I spend much of my day reading the news and paywalls are the bane of my existence. I want the news, not a login page. On the other hand, I love paywalls. Why? Because newsrooms have to keep the lights on and journalists need to eat. It’s good to subscribe to newspapers (I subscribe to three, much to the chagrin of my wallet) and support local journalism. That means respecting a paywall.

But no one said you couldn’t be clever.

The thing about a lot of the news behind paywalls is that it doesn’t stay behind paywalls. It gets syndicated, sometimes to paywall-free sources. Most stories, even those paywalled, have a paragraph or so of content. To find these articles elsewhere, you could easily copy a phrase and then look for it in Google News. Or you could make a couple of bookmarklets and have a one-click, instant search for different case scenarios.

In this article we’re going for the latter option: two bookmarklets that will help you get to articles you can’t access otherwise. They won’t work 100% of the time, but I think you’ll be surprised at how short some of those paywalls are.

First, A Quick Refresher

Have you heard of bookmarklets? They’re very old technology. In fact, the last time I wrote about them in a substantive article was over three years ago. They’re basically bookmarks souped-up with javascript, which lets you do things like add highlighted words to a query or paste in a phrase to an otherwise-constructed search.

The code I’m using in these bookmarklets is very much like what I was using in the 2018 article. I remain eternally grateful to Jesse Ruderman for his Squarefree bookmarklets page which allowed me to avoid digging through my two-decade-old notes for usable javascript.

Bookmarklet 1: Search Two Large News Aggregators With Paywall Ladder 1

The first bookmarklet searches two large news collections: Yahoo News and MSN News. You don’t hear much about news search engines any more because many people have moved to social media for their news consumption. Still, both these sites have a lot of content syndicated from a wide variety of sources. I’ve seen Bloomberg, Los Angeles Times, and Reuters, among others.

Here’s the bookmarklet code. You will have to copy and paste it into a bookmark, as I cannot figure out how to make WordPress happy with a bookmarklet link.

Here’s how for Firefox: Press Ctrl+Shift+B to show the bookmarks toolbar. Copy the code in the block below. Right click on the bookmarks toolbar. Choose Paste. A new bookmark will appear on the toolbar but the name will look weird because Firefox is using the javascript for a name. Right click on it and choose Edit Bookmark, then change the Name field to something more readable. 

 Here’s how for Chrome: Exactly the same, only you right click and choose Edit instead of Edit Bookmark.

javascript:q = "" + (window.getSelection ? window.getSelection() : document.getSelection ? document.getSelection() : document.selection.createRange().text); if (!q) q = prompt("You didn%27t select any text. Enter a search phrase:", ""); if (q!=null) location="http://www.google.com/search?q=%28site%3Amsn.com %7C site%3Anews.yahoo.com%29%20" + escape(q).replace(/ /g, "+"); void 0

This bookmarklet will take your search input and search only those two sites. Let’s do an example. Say you come across an article on Reuters about France and booster shots. But they want you to register to read it.

No worries. Just highlight some text from the article and click on the Paywall Ladder 1 bookmarklet. I think I’ll search for France said on Thursday it would make COVID-19 booster shots.

(Don’t want to copy and paste text? Just click on the bookmarklet. It’ll prompt you for some text, which you can add manually.) 

Here’s the result:

Four very precise results, and at least one (the second) an exact match. And all you had to do is highlight some text and click!

(TIP: When you’re picking out a string of text to search for, stick with the body of the article and not the headline. In my experience headlines are rewritten too often to make them a good search source.)

Sometimes, though, you might try a search for Reuters article text — for example, Portugal, which has one of the world’s highest rates of vaccination against COVID-19, and get a result like this:

You can see here a lot of articles relevant to your search terms, but not the exact article. Why not?

When you search for a paywalled article that’s been published fairly recently — in the last couple of hours, though sometimes it can be longer — you’ll sometimes find that the article is too new to have gotten to Yahoo News or MSN News; it does take some time for content to get distributed. That’s what Paywall Ladder 2 is for; to find recently published content in a much larger area.

Bookmarklet 2: Search 24 Hours’ Worth of Google With Paywall Ladder 2

The Paywall Ladder 2 bookmarklet will search Google, but only the last 24 hours’ worth of indexed content. 24 hours of indexed content on Google is still a huge amount though, so pick a good excerpt.

Here’s the bookmarklet:

javascript:q = "" + (window.getSelection ? window.getSelection() : document.getSelection ? document.getSelection() : document.selection.createRange().text); if (!q) q = prompt("You didn%27t select any text. Enter a search phrase:", ""); if (q!=null) location="http://www.google.com/search?tbs=qdr:d&q=" + escape(q).replace(/ /g, "+"); void 0

Let’s run that Portugal search again and use this bookmarklet instead of the first one:

Clearly it’s not perfect, ha! But it did find a similar story at the bottom of the page (that’s an Associated Press story). You can tweak this search a little more if it’s not finding exactly the article you’re seeking. See the Past 24 Hours setting under the search bar?

Use the pulldown menu to change that to Past hour.

Let’s take a look at the search result now:

Still no Reuters articles, but there are other articles with relevant information that you might be able to use instead. And you might surprise yourself: I tried to use the Paywall Ladder 2 bookmarklet to find the Bloomberg article mentioned in the above screenshot, and while I didn’t I found a copy of the Reuters article I was originally looking for at Gulf News. Don’t be afraid to do some experimental searching if you find clusters of articles.

Sometimes a news outlet needs a paywall to survive. There’s nothing wrong with that. If you read or use a news outlet on a regular basis, please consider a subscription or a donation — the creators out here need all the help they can get. But if you’re looking for an article as the result of a search or some other one-off, try these quick bookmarklets before you pull out your wallet. You may be able to avoid the paywall toll.

Leave a Reply