Transcription* of the full Keynote Address delivered by Charles FRACCHIA, CEO, Black Mesa
At the LMGlobal.Org‘s panel on “Towards a Public Discussion for Shaping the Future of A.I. Regulations in the U.S. & Beyond“, which virtually presented in collaboration with Bons-Ai.Org on November 15, 2025.

Introduction by Adam Bricker, Chairman of Bons-Ai.Org:
“Charles, thank you, and hello everybody. I’d like to introduce our next speaker. Charles Fracchia is the CEO of Black Mesa and a leader working across the for-profit and non-profit sectors at the intersection of AI, cybersecurity, biology, and critical medical manufacturing. His teams have partnered with the U.S. government and international allies to address complex challenges in vaccine manufacturing and other essential sectors. Today he will share recommendations for the safe and effective use of AI in critical infrastructure—an area where the stakes are extremely high and where thoughtful governance matters. Charles, the floor is yours.”

Thank you, Adam, and hello everybody. I’m going to start sharing my screen now. Great, thank you for the lovely introduction and opening remarks.

I lead several teams across the for-profit and non-profit sectors at the intersection of AI, cybersecurity, and biology, especially around vaccine manufacturing and other critical sectors that fill our medicine cabinets. We have worked in this space for the U.S. executive branch and with allies, handling many of the issues that arise there.

What I want to offer today is a set of recommendations for the safe use of AI in critical sectors and infrastructure. I’ll share guidelines and guiding principles that can help us capture the economic and productivity benefits of AI without creating undue harm or risk. In critical sectors such as biology, life sciences, and healthcare, we face some unique challenges that we must address.

I’ll start by giving you my conclusions, then walk through how I arrived at them. I’ll briefly review different types of AI and why it matters to distinguish them. I won’t go deep into technical details, just enough to show that many different systems get labeled “AI,” they behave differently, and therefore they must be regulated and used differently. I’ll then discuss a few examples from healthcare and life sciences to highlight what properties you need to watch out for when you implement these systems in your own countries and environments. After that, I’ll argue for investment in “AI trust labs” – safe environments in which AI can be tested in critical sectors before deployment. Finally, I’ll close with a point about how fast AI is diffusing through society and why education and diffusion matter so much.

So, here are the conclusions up front.

Who is accountable when something goes wrong.

First, AI used in critical infrastructure must be thoroughly tested, and we need responsibility frameworks that clearly delineate who is accountable when something goes wrong. We have too many examples where this wasn’t thought through, and the person who merely plugged in a machine ends up liable, even though they are not the engineer or designer of the AI system. We need to think carefully about responsibility, and we need to do it early.

Industry is currently racing ahead, adopting AI without fully considering the implications.

Second, we must build trusted environments where we can test these technologies: are they operating safely and effectively, what benefits do they bring, and what threats arise from their implementation? Industry is currently racing ahead, adopting AI without fully considering the implications. We need to step back and test these systems in dedicated environments.

How to avoid over-regulation while still protecting the public from harm.

Third, the key is to invest both in these labs and in the people who will gather around them. You don’t necessarily need the latest, biggest GPU clusters. You need environments where people can interact with the technology, experiment with it, and learn its limits. You get innovation benefits from that, and you also learn where the technology breaks, where regulation is needed, and how to avoid over-regulation while still protecting the public from harm.

Now, briefly on types of AI. Many different technologies get called “AI.” We have older techniques like Hidden Markov Models and other numerical systems; you may have heard of convolutional neural networks; you’ve certainly heard of GPTs, where GPT stands for “Generative Pre-trained Transformer.” These are different architectures. The most recent direction involves “Joint Embedding Predictive Architectures,” or JEPAs, which try to build models of how the world works and then test whether those models are correct.

The specific technical details aren’t important here. What matters is that older systems such as Hidden Markov Models behave more like one-directional pipelines: input A goes to B, then to C, then to D with known probabilities. You can trace the path and say, “The model produced this output because of these steps.” That traceability is important because it allows you to explain why a decision was made and to tie responsibility to people and organizations.

These deep systems cannot currently be mathematically guaranteed to stay within safe bounds.

Modern deep learning systems, by contrast, are much deeper and more entangled. The components interact in complex mathematical ways. You cannot reliably go backwards from an output to say exactly why the model decided what it did. That makes it extremely difficult to assign responsibility if something goes wrong. In addition, these deep systems cannot currently be mathematically guaranteed to stay within safe bounds. You cannot prove that they will never produce a particular harmful output.
A classic example comes from image classifiers. Students at MIT showed that you can take a picture of a turtle, add carefully-designed noise that is invisible to the human eye, and cause the neural network to classify the turtle as a rifle. A human can plainly see it is still a turtle. The point is not that accuracy can’t be improved in general; the point is that today we cannot mathematically guarantee that the model will never call the turtle a rifle. These systems can be duped, and we do not yet know how to fully secure them.

So, why does that matter? It means that the specific AI method you deploy – or buy – matters enormously, and context matters. Take self-driving cars. In most practical deployments today, a human still sits behind the wheel. The system assists: it keeps the car in its lane, changes lanes, adapts braking, and so on, but the human is still responsible for taking over if something goes wrong. That is one kind of setup, with a human firmly in the loop.

Other settings are very different. In some applications, as in the next examples, the human decision point is more distant or even absent, and the stakes are much higher. That requires a different design and regulatory approach.

The right question is: how do we integrate AI into the workflow?

Consider radiology. AI image analysis has been widely discussed here. Studies show that in some types of cancer, AI systems can be 22–25 percent better at finding tumors than humans. That’s promising. But the right question is: how do we integrate AI into the workflow? In good designs, the AI pre-analyzes the image and highlights suspect regions for the radiologist: “I think these areas might be tumors; please double-check.” In a trust lab, we can test such systems, learn which cancers they detect well, which ones they miss, and how reliable they are. Crucially, a human remains in the loop, making the final decision.

The lesson is that AI must be designed to augment human decision-making, not to replace it blindly.

This brings speed gains, better detection, and support for tired clinicians. But if you removed the human entirely and told everyone to “just trust the AI,” you’d risk many false positives and, more dangerously, false negatives: missed cancers with severe consequences. The lesson is that AI must be designed to augment human decision-making, not to replace it blindly.

Now, a second example from life sciences. The new U.S. ARPA-H “Catalyst” program, working with FDA, aims to reduce and eventually replace animal testing for drug safety using predictive AI models. Animal testing has historically been necessary for safety but is an imperfect stand-in for humans, which is why we still need clinical trials. The idea now is to use AI to generate predictive data that regulators might rely on directly.

This is very different from the radiology case. Here, the decision point is often singular and high-stakes: once a drug is approved, it goes to market and can impact millions of people. If you get it wrong at that point, the consequences can be catastrophic. The regulatory bar, the validation requirements, and the level of trust must therefore be much higher. Models and data must be validated in fundamentally different ways, and regulators must be confident that submissions are trustworthy and reproducible. That is hard work and still very much in progress.

We need sector-specific AI trust labs…

These examples support my main argument: we need sector-specific AI trust labs, tailored to each critical sector’s risks, regulations, and needs. In each sector – healthcare, life sciences, agriculture, power, water, manufacturing, finance, and others – we need dedicated environments to test AI safely, verify accuracy and reproducibility, and understand operating limits. Once built, these labs can double as innovation hubs and magnets for talent because they concentrate resources and expertise where people interested in the intersection of AI and that sector can come to experiment.

Joint efforts across borders to build and share trust labs in these domains could yield enormous benefits.

This approach is especially powerful when countries collaborate. Different nations and regions often specialize in different critical sectors – say, agriculture in one region, aerospace in another, water safety in another. Joint efforts across borders to build and share trust labs in these domains could yield enormous benefits. These labs are not necessarily massively expensive: we are often talking about investments in the low millions of dollars.

At Black Mesa, we built an early version of such a lab focused on biomanufacturing for medicines and vaccines. The U.S. government invested about 3.7 million dollars in establishing this environment. In its first year, it helped defend around 6.6 billion dollars’ worth of critical medical manufacturing infrastructure. Why? Because it provided a safe environment where we could test, find flaws, and fix them before those flaws could be exploited.

In the U.S., there are officially 16 critical infrastructure sectors…

In the U.S., there are officially 16 critical infrastructure sectors, and there are slightly more if you separate some sub-sectors like vaccine manufacturing. Sectors include chemicals, communications, emergency services, energy, financial services, food and agriculture, and so on. We are already seeing stress and even attacks in some of these areas. We should be building on the order of 16–20 AI trust labs across these sectors to understand how AI will affect their reliability and trustworthiness. Those labs will generate economic activity and resilience at the same time.

People are the key…

Let me close by emphasizing that people are the key. AI is an extremely experiential technology. Many of you have used chatbots; that is just one small slice of AI. But AI is also behind how music and movies are recommended to you. It’s a technology you understand by using it. The more touchpoints and hands-on experience you can give to your population, especially young people and practitioners in critical sectors, the better.

We tend to over-focus on the “very high end”: massive GPU farms and the largest models. But outside a few bleeding-edge research cases, that is not where most of the opportunity lies. There are now many open-source models that can be retrained or fine-tuned and used in specialized applications. The real leverage is in diffusion: how usable are these algorithms in each critical sector, and how do we deploy them safely?

Train people, let them experiment, and let the labs act as magnets.

AI trust labs in healthcare, for example, could test how chatbots might reduce intake burdens in hospitals while still operating safely. The goal is to embed this knowledge locally: train people, let them experiment, and let the labs act as magnets.

The real question is how broadly we diffuse safe, practical use across society.

Costs here can be in the hundreds of thousands to low millions per hub. The key is to distribute these investments across multiple hubs and sectors rather than pouring billions into a single “AI center” that only a small number of people ever touch. The technology is already usable enough that the real question is how broadly we diffuse safe, practical use across society.

It is essential to create AI trust labs… Without them, we will become victims of this technology.

So I’ll end where I began. It is essential to create AI trust labs where critical-infrastructure applications of AI are thoroughly tested and validated. Without them, we will become victims of this technology, always playing catch-up, and lives will be lost because we failed to build safe environments for testing.

At the same time, these labs can act as accelerators for diffusion: they give more people access to the technology, more opportunities to learn and experiment, and more chances to build specialized applications that serve their communities. Nations that invest in this kind of distributed, human-centered infrastructure will be the true winners of the AI revolution. It will not be those who focus everything on a single high-end hub where only a handful of people ever interact with the system.

Thank you.

* * *

A Brief Interview conducted with Charles Fracchia by Adam Bricker, Co-moderator of the session

Adam Bricker asked Charles Fracchia about the rising use of synthetic data in AI.
Mr. Fracchia explained that large language models have already consumed most available written data, and synthetic data is now feeding back into the training pipeline. He warned that this creates bias, risks of misinformation, and “pollution” of future models. Synthetic data can help narrow use cases, but it harms general models. He also noted that we are far from true AGI with current data sources, which is why the field is moving toward multimodal models that learn from text, video, audio, and interactions with the world.

Mr. Bricker then asked about early-stage regulation.
Mr. Fracchia highlighted Australia as a rare example of balanced, light-touch regulation that supports innovation. His primary concern is not over-regulation but the speed gap between rapid AI development and slow government processes. Technology evolves every two weeks, while laws evolve over the years. He emphasized testing frameworks, clear triggers, and practical oversight as more realistic regulatory tools.
He stressed the need to bring practitioners, not only academics, into advisory roles for government and regulatory bodies. Diverse expertise is needed to keep regulation grounded in real-world use.

When Bircan Ünver asked about a unified federal AI Act, Mr. Fracchia said a single, blanket act is neither realistic nor practical. Different sectors: autonomous vehicles, healthcare, manufacturing, defense, move at different speeds and have other risks. He gave examples: autonomous driving has defined test environments under NTSB and NHTSA. At the same time, defense has Directive 3000.09 [a U.S. Department of Defense (DoD) policy that sets the rules for how the military can develop and use autonomous and semi-autonomous weapon systems], which requires humans in the loop for autonomous weapons. Mr. Fracchia recommended a sector-by-sector approach, using existing regulatory authority, building trusted testing environments (“trust labs”), and gradually strengthening oversight. Once sectors mature, elements can eventually be woven into broader policy, but not as one giant act.

_ . _

[*] This transcription is obtained through Zoom-AI Assistant.

Special thanks to Salih Mansur, Volunteer, LMGlobal.Org.