double-edged sword, empowering both our creative and our destructive natures. It has brought us longer and healthier lives, freedom from physical and mental drudgery, and many new creative possibilities. Yet it has also introduced new and salient dangers.
Stalin's tanks and Hitler's trains used technology. And we still live today with sufficient nuclear weapons—not all of which appear to be well accounted for—to end all mammalian life on the planet.
Bioengineering is set to make enormous strides in reversing disease and aging processes. However, the means and knowledge to create unfriendly pathogens more dangerous than nuclear weapons will soon exist in most college bioengineering labs. As technology accelerates toward the full realization of genetic engineering, nanotechnology and, ultimately, robotics (collectively known as GNR), we will see the same intertwined potentials: a feast of creativity resulting from human intelligence expanded manyfold, combined with grave new dangers. We need to devise our strategies now to reap the promise while we manage the peril.
Consider unrestrained nanobot replication. Nanobot technology requires the coordinated operation of billions or trillions of intelligent microscopic devices to be useful. The most cost-effective way to scale up to such levels is through self-replication, essentially the same approach used in the biological world. But in the same way that biological self-replication gone awry results in biological destruction (cancer, for example), a defect in the mechanism that safely controls nanobot self-replication would endanger all physical entities, biological or otherwise.
The threats of nanotechnology don't stop there. We must also worry about control and access. Organizations, governments, extremist groups or just a clever individual could create havoc with this technology. For example, one could put trillions of undetectable nanobots in the water or food supply of an individual or an entire population. These "spy" nanobots could then monitor, influence and even control our thoughts and actions. Existing "good" nanobots could be influenced through software viruses and other hacking techniques. When there is software running in our brains, issues of privacy and security will take on a new urgency.
People often go through three stages in examining the impact of future technology: awe and wonderment at its potential to overcome age-old problems; then a sense of dread at a new set of dangers that accompany the new technology; followed, finally and hopefully, by the realization that the only viable and responsible path is to set a careful course that can realize the benefits while managing the risks.
The diverse GNR technologies are progressing on many fronts and comprise hundreds of small steps forward, each benign in itself. An examination of the underlying trends, which I have studied for the past quarter century, shows that full-blown GNR is inevitable.
These technologies are advancing on hundreds of fronts, rendering relinquishment completely ineffectual as a strategy.
Sun Microsystem's Bill Joy has eloquently described the plagues of centuries past and how new self-replicating technologies—such as mutant bioengineered pathogens and nanobots run amok—may bring back long-forgotten pestilence. It is also the case, which Joy acknowledges, that it has been technological advances, such as antibiotics and improved sanitation, that have freed us from the prevalence of such plagues. Suffering in the world continues and demands our steadfast attention. Should we tell the millions of people afflicted with cancer and other devastating conditions that we are canceling the development of all bioengineered treatments because there is a risk that those same technologies may someday be used for malevolent purposes? Having asked the rhetorical question, I realize that there is a movement to do exactly that, but I think most people would agree that such broad-based relinquishment is not the answer.
I do think that relinquishment at the right level needs to be part of our ethical response to the dangers of 21st-century technologies. One constructive example of that is the proposed ethical guideline by the Foresight Institute, founded by K. Eric Drexler (creator in the 1980s of the conceptual foundations of molecular manufacturing) and Christine Peterson. It states that nanotechnologists agree to forgo the development of physical entities that can self-replicate in a natural environment. Another proposal would create what nanotechnologist Ralph Merkle calls a "broadcast architecture," in which physical entities would have to obtain the codes for self-replication from a centralized secure server, which would guard against undesirable replication.
As responsible technologists, our ethical and professional guidelines should include such fine-grained relinquishment. Other protections will need to include oversight by regulatory bodies, the development of technology-specific "immune" responses, as well as computer-assisted surveillance by law enforcement organizations.
AS A TEST CASE, we can take a small measure of comfort from how we have dealt with one recent technological challenge. There exists today a new fully nonbiological self-replicating entity that didn't exist just a few decades ago: the computer virus. When this form of destructive intruder first appeared, concerns were voiced that as they became more sophisticated, software pathogens had the potential to destroy the computer network medium they live in. Yet the immune system that has evolved in response to this challenge has been largely effective. Although destructive self-replicating software entities do cause damage occasionally, the injury is but a small fraction of the benefit we receive from the computers and communications links that harbor them. No one would suggest we do away with computers, local area networks and the Internet because of software viruses.
I would describe our response to software pathogens as effective and successful. Although they remain a concern, the danger persists at only a nuisance level. Keep in mind that this success is in an industry in which there is no regulation and no certification for practitioners. And this industry is enormously productive. One could argue that it has contributed more to our technological and economic progress than any other enterprise in human history.
I hasten to point out that the battle concerning software viruses (and the panoply of other software pathogens) is not over and never will be. We are becoming increasingly reliant on mission-critical software systems (for example, the software running our 911 system, transportation, nuclear power plants, hospitals and many others), and the sophistication and potential destructiveness of self-replicating software weapons will continue to escalate. Nonetheless, we have already managed a number of significant challenges without major damage.
FUTURE DANGERS FROM NEW technologies may appear alarming when considered in the context of today's unprepared world. The reality is that the sophistication and power of our defensive technologies and knowledge will grow along with the dangers. When we have "gray goo" (unrestrained nanobot replication), we will also have "blue goo" ("police" nanobots that combat the "bad" nanobots). The story of the 21st century has not yet been written, so we cannot say with assurance that we will successfully avoid all misuse. But the surest way to prevent the development of the defensive technologies would be to relinquish the pursuit of knowledge in broad areas. We have been able to largely control harmful software virus replication because the requisite knowledge is widely available to responsible practitioners. Attempts to restrict this knowledge would have created a far less stable situation. Responses to new challenges would have been far slower