Tim Berners-Lee's heart's in the right place: An InfoWorld article quotes him as saying "Your data needs to be understood not by people, but by machines". Unfortunately that's not going to happen any time soon.
Take a look through http://www.pricewatch.com/ sometime. Click through on one of those nice shiny low prices and then look at the prices of other products on the site that low-balled your first product. Not such a good deal anymore, huh?
I overheard my office-mate calling up the 800 number of a vendor with a web site which had incredible comparison information between the various brands he was interested in. He'd gone so far as to pick up the phone, and they still wanted to fax or e-mail him pricing, because they needed to be able to pester him later.
If price is the only consideration, then there's no incentive to share pricing information. So vendors aren't going to share freely.
Similarly, I've been looking at ways to automate my web logging activities. There are several long lists of stuff which I'm not interested in for the most part, but I can do keyword searching and some adaptive algorithms to pull out the entries I do want to know about, saving me time. When I pull out this information, it's easier for me to throw away the advertising on which the publisher depends for income.
You think they're going to voluntarily toss away the advertising? Hell no, they're going to obfuscate that HTML as much as they can to make it as hard as possible to machine read it.
Nobody, except perhaps a few glory hounds and academics, have any incentive to share information. If they could get away with it, everyone would publish in PDF or Flash. If you can't get a human to invest their time reading it you can't extract or impart the information necessary to make money with it.
In Literary Machines Ted Nelson said that payment was an essential part of any hypertext publishing network. The web is a perfect example of why that's true, and until we get such an economic structure in place we're doomed to more flash and less substance.
Thursday, March 25th, 1999 danlyke@flutterby.com