BitFlip27
We Are Building Something We Do Not Yet Fully Understand: Why the People Behind AI Are Warning Us
Artificial intelligence risks are becoming increasingly difficult to ignore.
Artificial intelligence may be the fastest and most consequential technological transformation humanity has ever experienced. In only a few years, AI systems have progressed from relatively specialized research tools into technologies capable of writing software, generating realistic images and video, analyzing scientific data, communicating in natural language, assisting doctors, designing proteins, and increasingly operating as autonomous agents.
And the technology continues to improve.
What makes this moment particularly fascinating—and concerning—is that many of the people warning us about artificial intelligence are not outsiders who simply fear new technology.
Some helped create it.
Bill Gates has repeatedly discussed the risks associated with AI. Geoffrey Hinton, one of the pioneers of modern neural networks, has publicly warned about the possibility of machines becoming more intelligent than humans. Yoshua Bengio, another foundational figure in deep learning, has called for serious attention to catastrophic AI risks. Leaders of major AI laboratories have also acknowledged that sufficiently advanced artificial intelligence could create dangers requiring international attention.
That does not mean artificial intelligence will destroy humanity.
It does mean we should listen carefully.
The central question is no longer whether artificial intelligence will change civilization.
It already is.
The more difficult question is:
How far will this transformation go—and can humanity understand and manage a technology that may be improving faster than our institutions can adapt to it?
Table of Contents
- Why Artificial Intelligence Risks Are Different
- Bill Gates: AI Risks Are Real but Manageable
- Geoffrey Hinton: What Happens When Machines Become Smarter Than Us?
- Yoshua Bengio and the Problem of Autonomous AI
- AI Industry Leaders and the Warning About Catastrophic Risk
- Why AI Is Different From Electricity and Previous Revolutions
- Artificial Intelligence Risks in Biology
- AI-Designed Viruses: Science Fiction Becomes a Laboratory Experiment
- The Difference Between Simulation and Reality Is Shrinking
- The Acceleration Problem
- The Unknown May Be the Greatest Risk
- Should We Be Afraid of Artificial Intelligence?
- What Humanity Should Do
- Conclusion: We Are Participants in the Experiment
Check out these other publications
 |  |
1. Why Artificial Intelligence Risks Are Different
Human civilization has experienced technological revolutions before.
Fire transformed survival.
Agriculture transformed civilization.
The printing press transformed knowledge.
Electricity transformed industry and daily life.
Automobiles transformed transportation.
Computers transformed information.
The Internet connected billions of people.
Smartphones placed much of humanity’s accumulated knowledge into our pockets.
Each transformation produced enormous benefits while introducing new problems.
It is tempting, therefore, to treat artificial intelligence as simply the next chapter in this familiar story.
But there is an important difference.
Previous technologies primarily expanded what humans could do.
Artificial intelligence increasingly expands what machines can figure out how to do.
A hammer does not become better at building houses while you sleep.
Electricity does not study electrical engineering.
A conventional calculator does not decide to learn mathematics.
But modern AI systems are developed using enormous amounts of information and computational power, and researchers are creating increasingly capable systems that can reason through problems, use tools, write software, analyze results, and perform tasks that previously required human intellectual labor.
This creates a different category of technological change.
And it is one reason artificial intelligence risks deserve serious attention.
2. Bill Gates: AI Risks Are Real but Manageable
Bill Gates has been one of the most influential figures in the computer revolution, so his perspective on artificial intelligence deserves attention.
Gates has described AI as revolutionary technology comparable in significance to personal computers, mobile phones and the Internet.
But he has also written extensively about its risks.
In his essay The Risks of AI Are Real but Manageable, Gates discusses concerns including misinformation, bias, employment disruption, cybersecurity and the possibility of increasingly capable AI systems.
His position is not that humanity should abandon artificial intelligence.
Quite the opposite.
Gates argues that societies have successfully managed dangerous consequences of transformative technologies before and can potentially do so again.
That distinction matters.
Recognizing artificial intelligence risks is not the same as opposing artificial intelligence.
Cars kill people, but humanity did not prohibit automobiles. We developed traffic laws, driver’s licenses, seat belts, airbags, crash standards and safer roads.
Electricity can kill people, but we created electrical codes, circuit breakers, insulation standards and professional licensing.
Aviation is inherently dangerous, but commercial aviation became remarkably safe because engineering, regulation, training and investigation evolved alongside it.
The problem with AI is that our ability to build it may be progressing faster than our ability to develop the equivalent safeguards.
Source: Bill Gates, The Risks of AI Are Real but Manageable
https://www.gatesnotes.com/The-risks-of-AI-are-real-but-manageable
3. Geoffrey Hinton: What Happens When Machines Become Smarter Than Us?
Few names carry more weight in modern artificial intelligence than Geoffrey Hinton.
His work on neural networks helped establish foundations for the deep-learning revolution.
Hinton shared the 2024 Nobel Prize in Physics for foundational discoveries that enabled machine learning with artificial neural networks.
And he has become one of the most prominent voices warning about advanced AI.
In his official Nobel interview, Hinton was specifically asked about the greatest risks posed by artificial intelligence and how much time humanity might have before AI becomes more intelligent than humans.
His concern reaches beyond misinformation or automation.
It involves a fundamental control problem:
What happens if humanity eventually creates intelligence greater than its own?
We have never faced this problem before.
Humans have always been the most intellectually capable general-purpose agents controlling human civilization.
We created machines stronger than ourselves.
We created vehicles faster than ourselves.
We created computers capable of arithmetic far beyond ourselves.
But creating something capable of outperforming humans across broad areas of reasoning and strategy would represent something fundamentally different.
Hinton has emphasized that present-day problems—including unemployment and malicious use—matter as well. But his longer-term concern is that sufficiently advanced systems could become difficult to control.
That possibility remains uncertain.
But uncertainty is precisely why the subject deserves serious scientific investigation rather than dismissal.
Source: Nobel Prize, Geoffrey Hinton Interview
https://www.nobelprize.org/prizes/physics/2024/hinton/interview/
4. Yoshua Bengio and the Problem of Autonomous AI
Yoshua Bengio is another foundational researcher in modern deep learning and a recipient of the Turing Award.
He has increasingly focused his attention on AI safety.
One particularly important distinction Bengio and other researchers make involves the difference between an AI that simply answers questions and an AI agent capable of pursuing goals.
Imagine the difference between these two systems.
One AI receives a question, produces an answer and stops.
Another receives an objective, creates a plan, uses software, searches for information, executes actions, evaluates the results, changes its strategy and continues working toward its objective.
The second system is potentially much more useful.
It is also potentially much harder to control.
Bengio and colleagues have argued that unchecked AI agency could create risks ranging from malicious misuse to loss of human control.
The concern is not that today’s chatbot will suddenly become a science-fiction villain.
The concern is the trajectory.
As AI systems gain greater autonomy, planning ability, memory, tool access and real-world permissions, mistakes or misaligned objectives could become more consequential.
Source: Yoshua Bengio, Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?
https://yoshuabengio.org/en/publication/superintelligent-agents-pose-catastrophic-risks-can-scientist-ai-offer-safer-path
5. AI Industry Leaders and the Warning About Catastrophic Risk
In 2023, the Center for AI Safety published an unusually short statement.
It argued that reducing the risk of extinction from AI should be treated as a global priority alongside other civilization-scale risks such as pandemics and nuclear war.
The significance was not merely the sentence.
It was the people who signed it.
Signatories included Geoffrey Hinton, Yoshua Bengio, OpenAI CEO Sam Altman, Google DeepMind CEO Demis Hassabis and many other researchers and technology leaders.
This does not establish that AI extinction is inevitable, probable or scientifically settled.
There is considerable disagreement about the probability and nature of extreme AI scenarios.
But it demonstrates something important.
Serious researchers who understand these systems exceptionally well believe that the possibility deserves investigation.
When engineers building airplanes warn about an unknown structural weakness, we investigate it.
When epidemiologists identify a possible pandemic threat, we investigate it.
When nuclear scientists identify proliferation risks, governments take notice.
AI should not receive less scrutiny simply because some of its potential dangers are difficult to quantify.
Source: Center for AI Safety
https://safe.ai/work/press-release-ai-risk
6. Why AI Is Different From Electricity and Previous Revolutions
Electricity provides a useful comparison with artificial intelligence—but only to a point.
Electrification completely transformed civilization.
Factories changed.
Cities changed.
Transportation changed.
Communication changed.
Homes changed.
Medicine changed.
Virtually every industry eventually changed.
Yet electricity itself was not intelligent.
Electricity did not discover new electrical systems.
It did not write instructions for building better generators.
It did not communicate with engineers.
It did not analyze scientific papers.
Artificial intelligence can participate in the process of technological development itself.
An AI system can already assist a programmer who is developing another AI application.
AI can help researchers analyze scientific literature.
AI can help engineers evaluate designs.
AI can help scientists investigate proteins and biological molecules.
And AI systems can contribute to AI research.
This creates the possibility of technological feedback loops.
Better AI can help humans develop better tools.
Those tools can help develop better AI.
The better AI can then accelerate additional research.
That does not automatically produce the uncontrolled “intelligence explosion” sometimes portrayed in science fiction.
There remain significant limitations involving computing infrastructure, energy, data, algorithms, physical experimentation, manufacturing and human decision-making.
Nevertheless, AI introduces something historically unusual:
the technology can participate in the intellectual work required to improve technology.
That is one reason the current transformation may move faster than previous technological revolutions.
7. Artificial Intelligence Risks in Biology
Perhaps nowhere is the combination of extraordinary promise and potential danger clearer than biology.
For decades, biological research depended heavily on physical experimentation.
Scientists proposed hypotheses, performed laboratory experiments, analyzed results and repeated the process.
Computational biology changed this dramatically.
Artificial intelligence is accelerating the transformation further.
One famous example is AlphaFold.
Proteins are fundamental components of biological systems, and their three-dimensional structures strongly influence their functions.
Determining those structures experimentally can be difficult and time-consuming.
Google DeepMind’s AlphaFold demonstrated that AI could predict protein structures with remarkable accuracy. DeepMind subsequently released predictions covering more than 200 million protein structures.
This has provided scientists with an extraordinary computational resource.
AI is also increasingly being used for protein design, not merely protein prediction.
Researchers are exploring generative models capable of proposing biological molecules with desired characteristics, potentially accelerating drug discovery, biotechnology, enzyme engineering and synthetic biology.
This is enormously promising.
But the same general capability raises an unavoidable question:
If AI becomes increasingly capable of understanding and designing biology, how do we ensure those capabilities are used safely?
This is where artificial intelligence risks become inseparable from biosecurity.
Sources:
Google DeepMind AlphaFold
https://deepmind.google/science/alphafold/
Nature Reviews Bioengineering, AI-driven protein design
https://www.nature.com/articles/s44222-025-00349-8
8. AI-Designed Viruses: Science Fiction Becomes a Laboratory Experiment
This is where the subject becomes especially remarkable.
And we need to be scientifically precise.
In 2025, researchers from Stanford University and the Arc Institute reported using genome language models called Evo 1 and Evo 2 to generate complete genomes for bacteriophages.
Bacteriophages are viruses that infect bacteria.
They are not human viruses.
The researchers used a well-studied bacteriophage called ΦX174 as a design template and computationally generated candidate viral genomes.
Then came the crucial step.
They physically synthesized and experimentally tested hundreds of AI-generated genome candidates.
Sixteen produced viable bacteriophages.
In other words, information generated computationally by an AI system was translated into biological material, and some of those designs functioned in the physical world.
The researchers reported substantial evolutionary novelty among the viable phages. Some exhibited faster lysis dynamics or greater fitness under laboratory conditions than the natural template. A mixture of generated phages was also able to overcome resistance in several E. coli strains.
The research was conducted using a bacteriophage and non-pathogenic bacterial hosts under established biosafety procedures, and its potential benefits are significant. Bacteriophages are being studied as possible tools against antibiotic-resistant bacteria.
But conceptually, the experiment represents something extraordinary.
AI did not merely classify an image of biology.
It did not merely summarize a biology textbook.
It helped generate biological genome designs that were subsequently synthesized and shown to function.
That is a profound milestone.
It also corrects an important misconception.
It would be inaccurate to say that scientists simply “simulated a dangerous human virus in software and then created it exactly in reality.”
That is not what this research demonstrated.
What it did demonstrate is arguably more scientifically interesting: genome-scale generative AI produced candidate designs for bacteria-infecting viruses, and a subset proved viable when researchers experimentally constructed and tested them.
That boundary between digital prediction and physical biology deserves our attention.
Sources:
King et al., Generative Design of Novel Bacteriophages with Genome Language Models
https://www.biorxiv.org/content/10.1101/2025.09.12.675911v1
Arc Institute, How We Built the First AI-Generated Genomes
https://arcinstitute.org/news/hie-king-first-synthetic-phage
Nature, World’s first AI-designed viruses a step towards AI-generated life
https://doi.org/10.1038/d41586-025-03055-y
9. The Difference Between Simulation and Reality Is Shrinking
This biological example illustrates a much larger transformation.
For most of human history, experimentation was constrained by the physical world.
Want to test an aircraft?
Build models and test them.
Want to study a chemical?
Perform experiments.
Want to understand a protein?
Analyze it experimentally.
Computers introduced simulation.
Now AI is making computational exploration dramatically more powerful.
Researchers can increasingly use software to search enormous spaces of possible designs before deciding which candidates deserve physical experimentation.
This does not mean computer simulations perfectly predict reality.
They do not.
Physical validation remains essential.
The bacteriophage experiment itself illustrates this: hundreds of generated designs were considered and nearly 300 were experimentally tested, but only 16 viable phages were obtained.
That is far from perfect prediction.
But consider the direction of progress.
As AI models improve, the digital world may become an increasingly powerful laboratory for exploring possibilities before physical resources are committed.
This could transform:
medicine,
materials science,
chemistry,
energy,
robotics,
aerospace,
engineering,
climate science,
and biology.
The benefits could be extraordinary.
Imagine discovering medications in months instead of years.
Imagine designing new materials for batteries, spacecraft or clean-energy systems computationally before manufacturing them.
Imagine AI identifying treatments for diseases that have resisted decades of conventional research.
These possibilities are reasons to be excited about AI.
But increased capability also increases responsibility.
10. The Acceleration Problem
One of the greatest artificial intelligence risks may not involve any single malicious system.
It may simply be speed.
Human institutions move slowly.
Laws take years to develop.
International treaties can take decades.
Education systems take years to redesign.
Companies take time to restructure.
Scientific norms evolve gradually.
Human culture may take generations to fully absorb transformative technology.
AI development can move on a very different timescale.
A model can be released.
Millions of people can begin using it.
Businesses integrate it.
Developers build applications around it.
Competitors respond.
A new generation appears.
Capabilities that seemed experimental become ordinary.
Then another generation arrives.
This creates an enormous governance problem.
Imagine trying to write traffic laws while automobiles double their speed every year.
Imagine creating aviation regulations while airplanes continuously acquire entirely new capabilities.
That is closer to the challenge society faces with AI.
We are attempting to understand a technology while the technology itself continues changing.
11. The Unknown May Be the Greatest Artificial Intelligence Risk
We naturally try to predict AI’s future using history.
We compare it to electricity.
The Industrial Revolution.
Nuclear energy.
Computers.
The Internet.
Those comparisons are useful.
But they may also create false confidence.
History can tell us how humans responded to previous technologies.
It cannot tell us precisely what happens when machines become capable of performing increasingly sophisticated cognitive tasks.
Humanity has never experienced that before.
That is the part I find most important.
The greatest danger may not be something we already understand.
It may be the fact that we do not know what we do not know.
Imagine someone in 1980 trying to predict social media.
They might have understood computers.
They might have understood telecommunications.
They might even have imagined electronic messaging.
But could they realistically have predicted influencers, viral memes, cryptocurrency, algorithmic recommendation systems, online dating, misinformation networks, streaming culture, smartphones and billions of people carrying Internet-connected cameras?
Probably not in meaningful detail.
Now consider AI.
We are trying to predict the consequences of technology that may itself accelerate scientific discovery and technological development.
Our predictions could be wrong in both directions.
We may overestimate some dangers.
We may underestimate others.
We may also dramatically underestimate the benefits.
That uncertainty is not an argument for panic.
It is an argument for humility.
12. Should We Be Afraid of Artificial Intelligence?
Fear is probably the wrong objective.
Respect
