Flutterby™! : link rot 2006-10-20 15:53:03.220527+00

link rot

2006-10-20 15:53:03.220527+00 by Dan Lyke 4 comments

As we approach 9 years of Flutterby, one of the problems I see is that link rot in the archives is going to be influencing search engines in bad ways. I'm running across publications and weblogs I used to link to that have become spam sites.

I don't necessarily want to just blow away all the old content here (although I do think we need some better ways to re-use the old stuff, I know there have been some great comment threads that I'd love to re-read if I could find 'em).

Anyone have suggestions? I guess an enhancement to the CMS which lets me de-link any sites I find which have expired would be a good thing, maybe I need to re-map them to a page which lists what the URL used to be (because sometimes there's useful information in that URL), but if you know anyone who's dealing with this stuff successfully, or have any ideas (they don't even have to be good ones), let me know.

Just sorta fingerstorming, but perhaps the CMS should (can?.... could be coded to?) automatically generate "archived link" pages for ALL links after a certain period of time which lists the URL and interested parties could cut/paste to see where that link goes or doesn't go..... or there may be some kinda "no follow" or whatever that could stop search engines from going past those archived link pages.

I FULLY agree about the recycling old content, as I've found myself searching for old threads here with spotty success. Kinda serious/kinda giggly.... a 10th anniversary Flutterby archives DVD.... The Flutterby Cocoon, if you will.... with some kinda mechanism to search it would be something I'd enjoy and donate toward.

The Flutterby Reunion Special, followed by the expose on E!ntertainment channel..... and eventually the feature on History Channel narrated by Peter Coyote, with historical reinactments.....

One of the things I've pondered is moving all the links on my site to a db (a big step in the first place), and then periodically grinding through them and automagically updating redirects (though I'm not sure what to do on the domain-parking ad sites) and also flagging dead sites as dead. The latter would get turned into text, rather than a link, so the original URL would still be available, but it wouldn't actually be a link.

Been thinking about it for a few years though, and haven't gotten any farther on implementing it, since when I do have time for system administration like that, I tend to use most of it dealing with active annoyances, which generally means updating my spam-filters or adding a password to my submission page so I don't have to wade through a gajillion spam links.