The Death of DMOZ?

We’ve all been present at some party or event at which someone trots out the funniest part from a Monty Python sketch, or even worse attempts to do the entire 200-line routine in two or more poorly punctuated British accents. If this hasn’t yet happened to you, you obviously haven’t been a techie long enough. There are, you see, 10 types of techies in any organization. There are those who repeat Monty Python jokes and those fated to listen to them over and over again.

Generally speaking there is no median, no in between. It is either A or B, on or off. Nothing but dead air separates those who trot out the old skits and listeners politely repressing their inner psychopath. One should consider that frightening fact before leaping into the frivolity of the dead parrot sketch but since we’re talking about the closing of the Open Directory Project, or DMOZ, today I think it’s worth taking the risk.

So this guy walks into a pet store, or more realistically, blogged a complaint about DMOZ in 2006…

Customer: … I wish to complain about this parrot what I purchased not half an hour ago from this very boutique.

Owner: Oh yes, the, uh, the Norwegian Blue…What’s,uh…What’s wrong with it?

C: I’ll tell you what’s wrong with it, my lad. ‘E’s dead, that’s what’s wrong with it!

O: No, no, ‘e’s uh,…he’s resting.

C: Look, matey, I know a dead parrot when I see one, and I’m looking at one right now.

O: No no he’s not dead, he’s, he’s restin’! Remarkable bird, the Norwegian Blue, idn’it, ay? Beautiful plumage!

C: The plumage don’t enter into it. It’s stone dead.

O: Nononono, no, no! ‘E’s resting!

C: All right then, if he’s restin’, I’ll wake him up!

(shouting at the cage)

‘Ello, Mister Polly Parrot! I’ve got a lovely fresh cuttle fish for you if you show…(owner hits the cage)

O: There, he moved!

C: No, he didn’t, that was you hitting the cage!

O: I never!!

C: Yes, you did!

O: I never, never did anything…

C: (yelling and hitting the cage repeatedly) ‘ELLO POLLY!!!!!

Testing! Testing! Testing! Testing! This is your nine o’clock alarm call!

(Takes parrot out of the cage and thumps its head on the counter. Throws it up in the air and watches it plummet to the floor.)

C: Now that’s what I call a dead parrot.

O: No, no…..No, ‘e’s stunned!

C: STUNNED?!?

O: Yeah! You stunned him, just as he was wakin’ up! Norwegian Blues stun easily, major.

And on it goes. Let’s cut to the chase…

Though it is still functioning as a website, DMOZ is dead. It is not resting as some wish to believe and some stunned (and now) ex-editors suggest. DMOZ is deader than a doornail with even less substance to it than the specter of Marley’s Ghost.

Now, don’t get me wrong. I am not happy that DMOZ has died though I am of the chorus saying it’s about time. It is good to see it is no longer suffering and I am happy that search engine users are no longer subject to its misery.

The Open Directory is the oldest and largest human edited directory on the web. Since 1998, it has provided a massive depository of human edited listings for free; one used and trusted by most of the major search engines over the years. At one time, a listing at DMOZ was a long-term foundation piece of a winning SEO strategy.

Today, the editorial and submission interfaces are disabled. To make a long story short a server appears to have crashed at AOL and the remaining AOL staff doesn’t seem to care. According to an Open Directory founder, Rich Skrenta,

“Apparently the machine holding dmoz in AOL ops crashed. Standard backups had been discontinued for some reason; during unsuccessful attempts to restore some of the lost data, ops blew away the rest of the existing data on the system.”

Ouch. Skrenta was one of the originals that launched the NewHoo (first known as GnuHoo) directory in early 1998 as a Yahoo-like directory that would be edited by an unlimited number of volunteer editors. The Internet, at that time was exploding with new websites and Yahoo’s staff of professional editors couldn’t keep up with the growth. NewHoo and its rapidly expanding community of editors could.

Arguably the first of what is now called Web2.0 initiatives, NewHoo actually grew faster than Yahoo did in the last half of 1998. Less than a year after it began, the all-volunteer project had acquired 8000 editors and over 430,000 websites. It had changed its name again, this time to the Open Directory Project, to differentiate itself from Yahoo. In April 1999, the ODP directory was picked up as a reference source by its first major search engine, Lycos. Aside from a sudden number of business dealings that put the ODP in the hands of Netscape and eventually in to the AOL/Time Warner fiasco, the rest is data-feed history.

Google populated itself in part through regular DMOZ RDF data-dumps and as Google’s popularity and reach grew from 2000 to today, the importance of a listing in the Open Directory grew. Because ODP listings were human-reviewed, Google and other search engines tended to trust them and give stronger placements, faster.

Between 2002 and into 2005, a listing at DMOZ could mean the difference between weeks and months for initial organic search placements. Towards the beginning of 2005 and throughout 2006 however, the editorial energy level at DMOZ lagged with submissions waiting 6 months or more to be reviewed. Some categories had gone years without proper editing. By 2005 the submission backlog had built itself into a systemic issue in the SEO community. Editorial issues and charges of corruption began to plague the editorial collective.

Flashing ahead to today, the server seems to be broken, the core data lost and apparently, nobody made a backup, and that’s all we wrote for the ODP.

Before we raise our heads from a collective bow, let’s take time to remember the early success of the ODP was the forerunner of what we would now call a Web2.0 environment. Those who built the ODP went on to become parts of the Mozilla Founation and have become leaders of the Open Source movement. Both ideologically and technologically, the ODP made tremendous contributions to the online community and the search marketing community. All things pass but it is still sad to see the days when the dragon appeared to die.