John Gruber at Daring Fireball has a few ideas why Yahoo has gone into decline. “I would argue that Yahoo lost its way early. Yahoo was an amazing, awesome resource when it first appeared, as a directory to cool websites. Arguably, the directory to cool websites. It was hard to find the good stuff on the early web, and Yahoo created a map. Their whole reason for being was to serve as a starting point that sent you elsewhere. Then came portals. The portal strategy was the opposite of the directory strategy — it was about keeping people on Yahoo’s site, instead of sending them elsewhere. It was lucrative for a while, but ran its course. And it turned out that the web quickly became too large, far too large, for a human-curated directory to map more than a fraction of it. The only way to index the web was algorithmically, as a search engine. And one search engine stood head and shoulders above all others: Google.”

I agree with this up to a point. I do agree that the idea of portals distracted and kind of knocked everything off-kilter for a while. Excite got a lot of attention, but there are only so many ways that you can display sports scores and weather, and unfortunately many of them involved pushy, annoying ads. Yahoo did get caught up in this madness for a while and it certainly didn’t help.

What I think really cooked Yahoo’s goose, though, is the fact that their directory became very money-oriented. In 2001, as near as I can remember, Yahoo started requiring payments for expedited consideration. Check out the Yahoo Directory Submit terms of service. “Yahoo Directory Submit is a fee-based service that allows you to pay for expedited consideration of your web site for possible inclusion in Yahoo’s directory of web sites (the “Directory”). Inclusion in the Directory is not guaranteed. Subject to the terms below, and in Yahoo’s sole discretion, if your web site is accepted for inclusion in the Directory through Yahoo on or after December 28, 2001, then your web site’s continued inclusion in the Directory will be subject to an additional review each year and to the applicable, then-current NON-REFUNDABLE recurring annual fee (“Recurring Annual Fee”). A little further down in the terms of service the fee is denoted as $299 a year for non-adult sites, $600 a year for adult sites.

So you spend $299 to get the site considered, but a listing was not guaranteed. And even if it was accepted, you a) had to pay annually and b) had no guarantee of what your listing would look like at Yahoo: “Yahoo also reserves the right, in its sole discretion, to remove your site from the Directory, move the listing to a different category or subcategory, and change or remove any keywords, comments, or annotations at any time, for any reason, including but not limited to, a change in the nature, business, goods, or services or content available on the site.”

Yay.

This blatant and in my opinion over-aggressive attempt to monetize the Yahoo Directory is what started its demise, because $299 (annually!) is a huge demand for a promise of basically nothing. Small, struggling startups are not going to pony up $300 in this case. People finding cool stuff (or making cool stuff) aren’t going to bother dropping their URLs in the black hole of Yahoo’s non-paid, non-expedited directory submissions.

Admittedly Yahoo did not just jump in with something this drastic. The effort to commercialize the Yahoo Directory started on a much smaller and more palatable scale in 2001. But by the end of the year, the money grab had gotten a lot more egregious. With this shift to paid listings and a focus on monetizing above everything else, Yahoo killed and ate its own soul. For the last seven or eight years the directory was just a carcass.

But the idea that there’s no place on the Web for a good directory? If the Web is just getting larger and larger and larger, isn’t that more of an argument for a good map? Google had the Google Directory (which drew mostly off DMoz, but that’s a whole other story) from 2000 to 2011. Zeef is raising money for a crowdsourced link directory. There’s got to be a way to either use the power of a busy social network to organically grow a link directory (could I trade a minute or two of categorizing the links I post on Facebook for the promise of further reach to my friends? Oh I think I could) or to use a busy, vibrant, not-necessarily-social-network community to crowdsource links into the proper categories (Reddit, Metafilter, etc. This might also be possible for something like LinkedIn if one wanted to develop a link directory.)

The last two really big directories on the Web were Yahoo and DMoz. Yahoo failed because it made an unwise attempt to monetize and imploded. DMoz had its own issues which would require an entire article. I have yet to see a thorough attempt, in the age of both crowdsourcing and social networks, to harness the interactivity that goes on in real-time now and turn that into an organically-growing directory of links and information. I do believe it would be useful, I do believe it could be viable commercially – and I do believe we’re still waiting for it.