“Tell all the Truth, but tell it slant,” advised the poet Emily Dickinson. There’s a reason that the Defense Advanced Research Projects Agency does its funding quietly. Can you imagine how, at the height of the Vietnam War, the Senate would have reacted to a Pentagon scheme to render the US telephone system invulnerable to a nuclear attack by decentralizing it? The Internet would have been shut down in 1968, as soon as its first $2.2 million for contractors was approved.

Something like that happened last week after Senators Ron Wyden (D-Oregon) and Byron Dorgan (D-N. Dakota) called a press conference to denounce Pentagon plans to register a thousand traders for an online futures exchange to be known as the “Policy Analysis Market.”

The idea of “a federal betting parlor” in which anonymous speculators would place bets on terrorist attacks, assassinations and coups, was ridiculous, said Dorgan — so absurd that he had difficulty “persuading people that the plan was not a hoax.”

Four days later, under heavy criticism in the press, the Defense Department cancelled all its research into the use of markets as predictors of future events. Federal support will continue, of course, in some slightly different guise. But what a sloppy way to proceed!

Almost everybody understands that, under certain circumstances, markets can provide valuable clues to coming events. When in 1983 Soviet Communist Party chief Yuri Andropov worried that Ronald Reagan might be about to launch a preemptive missile assault, he ordered KGB agents in London to monitor the spot market there for blood — on the grounds that a sharply rising price would signal insider knowledge of an impending attack.

What’s new is the engineering approach to the design of market mechanisms that has developed in the past thirty years.

The practical history of intentionally designing markets to elicit information goes back at least to William Vickrey, who in 1961 published a provocative analysis of sealed-bid auctions for Treasury debt in which he plumped for a version in which the winner would pay (or receive, if the contract were for a highway project) only the second-highest price.

The idea was to make things more difficult for those bent on gaming the system by understating their willingness to pay. For this, and other mechanisms designed to encourage truthful revelation of otherwise private information, Vickrey shared the Nobel Award in 1996. (Vanderbilt economist David Lucking-Reiley discovered a few years ago that auctioneers in rare stamp markets had been using the second-price method for decades before Vickrey formally described the way it worked.)

But it was when Vernon Smith joined Charles Plott at Caltech in the early 1970s that experimental economics really took off. The National Science Foundation established a number of laboratories around the country. Smith left Pasadena for Arizona State. But John Ledyard arrived from Northwestern University, and Caltech remained a hotbed of exciting work on bargaining, auctions, the provision of public goods, information aggregation, and individual decision-making.

Smith last year shared the Nobel Award, in recognition of a skein of work going all the way back to the early 1950s — and his first (failed) attempts to replicate monopolistic competition in Edward Chamberlin’s Harvard classroom.

Perhaps the most compelling demonstration of the usefulness of markets as devices for prediction has been developed by the group around Robert Forsythe at the business school of the University of Iowa over the past fifteen years. The  Iowa Electronic Markets opened for business in 1988, backed by the National Science Foundation as a research and teaching device. A “no-action” letter from CFTC chairman Wendy Gramm was required.

The idea behind the small-stakes, real-money futures market was to permit traders to design and trade contracts based on the electoral prospects of political candidates and, in due course, financial events (the Computer Industry Returns Market) and economic indicators (the Federal Reserve Monetary Policy Market). Too bad the Iowans never designed a futures contract on California energy policy!

There are no high rollers in Iowa City — accounts are accepted of as little as $5 to no more than $500. You can make money buying, say, Howard Dean futures at the Democratic convention or Congressional seat-share contracts — but you can loose it, too.

Note that such “decision markets” are practically the opposite of an opinion poll. There is nothing representative about their traders — in the Iowa market they are overwhelming male, well-educated, high-income and young (average age 30). They may not even be registered to vote. Their trades merely sum up what they know, or think they know, about the likely outcome of the election.

Yet in a careful study published three years ago, of outcomes for 49 markets covering 41 elections in 13 countries over the course of a dozen years, the Iowa organizers found their markets consistently outperformed opinion polls in terms of average error, and in a few cases — the 1988 and 1992 presidential elections — dramatically outperformed the polls.

Why? Because, where money is at stake, markets aggregate and summarize a great deal of disparate information about pending events that may affect a particular price — oil, for instance, orange juice, or even blood. No longer is it controversial to set up small money markets to elicit information about the prospects of athletes’ abilities, actors’ reputations or film receipts. The interesting questions now all have to do with how those markets can be gamed.

(Incidentally, Slate magazine outdid itself last week with three sophisticated takes on the controversy by Daniel  Gross, James Surowiecki and Brendan  Koerner — a reassuring assertion of its franchise as journalism’s best online magazine.

(The New York Times, on the other hand, which led the charge against the experimental market, editorialized Wednesday that “The time has obviously come to send John Poindexter packing and to shut down the wacky espionage operation he runs at the Pentagon.”

(“Quite apart with the tone-deafness of equating terrorist attacks with, say, corn futures, the plan would allow speculators — even terrorists — to profit from anonymous bets on future events.

(“The project’s theoretical underpinnings are equally absurd. Markets do not always operate perfectly in the larger world of stocks and bonds,” The Times informed its readers. “The idea that they can reliably forecast the behavior of isolated terrorists is ridiculous.”

(The next day, The Times’ business section ran an exemplary explainer of why the market in terrorism indicators was a good idea, by Berkeley economist and Times columnist Hal Varian.)

The prospectus for the Policy Analysis Market that abruptly vanished from the Web last week was, by all accounts, a glowing invitation to the public relations disaster that befell it — a handful of Web pages describing in lurid terms the looming opportunity to go long on germ warfare or, perhaps, to short a particular king. The exchange was to have begun registering an initial thousand traders last Friday, to begin trading October 1.)

In fact, the choice of contracts to be traded on the PAM would have been up to market participants. Chances are they would have come up with something more sophisticated and valuable than a classic “dead pool.” The plan had been to use the Economist Intelligence Unit’s country political instability index as a basis for trading contracts.

It turns out that two companies had futures markets contracts with DARPA. Only one made it into the news — San Diego’s Net Exchange, founded in 1994 by Caltech’s Ledyard to give advice on high-stakes spectrum auctions. It was Net Exchange’s web page that caused the ruckus, and its consultant, a remarkable George Mason University economist named Robin Hanson, who handled much of the explaining after the fact.

The other contract belonged to MarTek, whose principals are the founders of the Iowa Electronic Market. It was MarTek that organized the DARPA conference last year at which the new possibilities were discussed. Its scientists often have advocated a lower-key approach to the creation of prediction markets, including the use of small private markets, in which the only eligible traders would have been employees of various competing government security agencies.

(Similar experimental markets among corporate insiders are beginning to show significantly better track records forecasting quarterly sales than do official reporting channels.)

There’s no doubt the government will try again — the new techniques are too interesting to ignore. How about a decision market for various approaches to a new and uncertain R&D program? To the allocation of resources within a university? Among military units in the field? First, however, there is plenty of blame to go around.

Retired Rear Admiral John Poindexter, controversial since his days as Ronald Reagan’s National Security Adviser, is the designated fall guy. He will leave his senior DARPA position in the autumn. Another manager with egg on his face is Michael Foster, the DARPA official in charge of the futures project. Caltech’s approach from the beginning was characterized by more than a little swagger and brag. The Iowa crowd, with its fifteen year track-record, gets off scot-free.

The great irony is that the flap should occur just a week after the release of a Congressional report delineating various failures of the Army, Air Force, Navy, Marines, State Department, FBI, CIA, Defense Information Agency, National Security Agency and various other security agencies to share the scraps of intelligence that each had collected in the years before 9/11.

It’s not that markets are perfect. There is, however, reason to believe that properly-designed prediction markets can do a somewhat better job pooling information about possible outcomes than can committee meetings of bureaucrats and inter-service rivals whose nature is to seek to one-up one another. Political decision-makers should have access to both kinds of forecasts.