Do you miss Yahoo and DMoz and all the other link directories that used to roam free across the vast plains of the Internet? I would put the serious decline as starting in 2011, when Google Directory shut down.

That was a while ago so if you were busy then, a bit of backstory: link directories indexed Web resources instead of spidering them. In other words, an entry for a website on a directory like Yahoo would be the name of the site and a description of a few sentences. If it was a really big/important/popular site, subsites might also be indexed and described as well. A directory’s listings were searchable by keyword as well as browsable by category and generally they were browsable by other parameters as well (most recent, most popular, editor’s picks, etc.)

A collection of Websites aggregated with such basic information is a very different search experience from that of a full-text engine like Google, which spiders and makes searchable the entire text content of sites. The limited amount of searchable text in a directory meant that general searches were more productive, while the category structure meant that you could gain context about your topic as you explored to learn more about it. I firmly believe that the Internet ecosystem would benefit from a good Web directory, though there isn’t one available.

Well, if they’re so great, why don’t major link directories exist today? I would guess a couple of reasons. First, Yahoo started charging a $299 submission fee to add links to its directory. I think it might have been intended to act as a filter for spam and so forth but it ended up filtering out all the hobbyists and labor-of-love types, the people who make really great content, because they couldn’t afford $299. Then, if I recall correctly, Yahoo decided not only did the directory require $299 as a submission fee but also as a yearly fee of $299 – and that was the beginning of the end for Yahoo. (I apparently ranted about this in late December 2014, so feel free to read that if you want more of my opinions about what lead to Yahoo’s decline.)

(If you care to draw parallels between that and the current mania of social media platforms for charging their users, with implications about how high those fees will go once users are locked in, I will sit here quietly and not interrupt.)

The second reason, I think, has to do with attention flow. Before the rise of social media, there was a tremendous amount of content curation energy focused on the Web itself. As Facebook got larger, there was still a certain amount of flow between the greater Web and Facebook. Facebook made content available by RSS and didn’t throttle the reach of external links like it does now. But as Facebook went more and more walled-garden, that curation energy became focused on Facebook to Facebook’s benefit and the Web’s deficit. And that was the beginning of the end for the Open Directory Project.

But the thing is, there’s still a lot of curation energy on the Web. Huge numbers of people contribute to Wikipedia. And more importantly, Wikidata has as one of its properties “official website.” So why not turn Wikipedia into a searchable link directory?

Sounded good to me, so I made WOLPE: Wikipedia Official Link Property Explorer – https://searchgizmos.com/wolpe/ . I really wanted to call it WikiHoo but I can’t afford a lawyer.

How to Use WOLPE

WOLPE is easy to use. You’ve got two search type options: Title Only, or Summary (which searches a brief description of the page.) Both pages have limited text, but the summary has enough that it will find closely-related keywords. Make your search general, two or three words.  WOLPE will find you Wikipedia pages with official website properties; links, in other words. Here’s what the default search, Eurythmics, looks like with the Title Only option:

Just one result. On the other hand, if you search for the summary, you’ll find the pages for the duo’s two members, as well as affiliated people and topics.

I wanted to add a third option for a full-text search of Wikipedia, but I couldn’t figure out a good way to do it without running afoul of Wikipedia’s API rate limits. It worked only if I rate-limited the API calls to an extent that it was just too slow.

I also want to figure out how to add a category structure, because I liked it as a mechanism for passive learning as you were exploring your chosen topic. But it’s fun to use as-is!