Updated March 5, 2026
0:00 Welcome to the Colaberry AI podcast brought to you by Colaberry AI Research Labs and the Carl Foundation. Today, we're diving deep into some pretty cutting edge AI safety research. Okay. Yeah. We're looking at what the Foresight Institute is funding. 0:13 I found the stuff you shared really fascinating. Oh, cool. Their whole grant program is focused on what they call underexplored areas for making sure advanced AI benefits humanity, so AI safety. Yeah. That's a big one. 0:26 We'll try to unpack these complex ideas simply and clearly so that you can have some moments without getting bogged down by all the technical jargon. Right. Right. Make it accessible. Yeah. 0:36 Exactly. And we'll explore four main areas they're supporting through these grants. Sound good. So first, the Foresight Institute is looking at using AI to help us with AI safety research itself. Okay. 0:48 So, like, AI for AI safety. Yeah. Exactly. It's like, using AI to build even smarter but safe AI tools. Interesting. 0:56 Yeah. One key idea they're exploring is whether we can safely accelerate neurotechnology research. Neurotechnology. Yeah. So technology that interacts with the brain Okay. 1:06 With the goal of creating human aligned software intelligence to better manage the risks that might come with more advanced AI. Right. That makes sense. So they're not just looking at AI in isolation. Right. 1:20 They're considering how our understanding of the human brain and our ability to build technologies that can interface with it Mhmm. Could change how we think about AI safety. Yeah. Like, this idea of speeding up neurotechnology research to create a kind of AI assistant that deeply understands human values that's pretty innovative. It really is. 1:39 And there are a few specific things they're exploring within this area. One is scaling AI enabled research to help develop safe AI in all sorts of ways. Okay. Another one is, scaling efficient forecasting methods. Forecasting. 1:53 So Basically, they're trying to get better at predicting what might happen as AI gets more powerful. Oh, I see. So we can prepare for any problems. Right. To be proactive. 2:02 Exactly. They even use terms like superhuman automated forecasting and brain GPT. Those sound pretty futuristic. They do. Like something out of a movie. 2:10 Right. But they're serious concepts. You know? They wanna use AI to understand and navigate the future of AI itself. That's a really interesting approach. 2:20 It's like, if AI is gonna become so powerful Yeah. Maybe the best way to predict its future and the risks is to use AI's own abilities for analysis and prediction. It makes sense. Right. Superhuman automated forecasting suggests that AI could process so much information and see patterns that we humans might miss, leading to better predictions about how AI will develop. 2:45 Uh-huh. And brain GPT, although more futuristic Yeah. Hints at AI models that could understand intelligence inspired by how our brains work. And then there's super alignment. Super alignment. 2:55 So how do we ensure super intelligent AI AI that's smarter than us acts according to human values? That's the million dollar question. Right? It's a big one. And the Foresight Institute is looking at how using AI to automate research could help find solutions. 3:09 That makes sense. Yeah. But it also raises another question. How do we ensure the AI systems designed to help with AI safety are themselves safe and aligned? Yeah. 3:19 That's a good point. It gets pretty layered, doesn't it? Yeah. It's complex. We'll need to carefully test and validate these AI safety tools. 3:27 Absolutely. Okay. Moving on to the second area. Right. This is where it gets really mind bending. 3:32 We're talking neurotechnology, which, again, is tech that interacts with our brains. Right. The idea is to use it to either work with advanced AI or maybe even provide a human like alternative. So instead of just making AI safer, we're also trying to enhance our own capabilities. Exactly. 3:50 It's about strengthening ourselves in relation to AI. Makes sense. So they're looking at brain computer interfaces or BCIs. Oh, like controlling things with your mind. That's part of it, but they're also thinking about BCIs as a way to boost our thinking abilities Oh. 4:05 Or to create a seamless way for humans and AI to collaborate. Imagine a direct mental link to AI to help solve problems or come up with ideas. That's pretty revolutionary. Combining human intuition and creativity with AI processing power. It is. 4:21 But there are big questions there too. Right? Like privacy, autonomy, and whether everyone would have access to such technologies. Yeah. A lot to consider. 4:29 Then there's whole brain emulation or WBE. Whole brain emulation. That's creating a digital copy of a human brain. So like a digital brain. Yeah. 4:39 Wow. The idea is that these digital brains might be easier to understand and align with human values compared to, you know, AI built from scratch. Because it's based on a human model. Exactly. That's fascinating. 4:52 It makes you wonder is by understanding and replicating the human brain digitally, we might better understand intelligence itself. Uh-huh. And that could help us ensure AI is aligned with human values. Right. And because these emulated brains would be based on human structure Yeah. 5:08 They might inherently understand the world similarly to us. Interesting. The Foresight Institute is also looking at lo fi emulations. Lo fi. These aren't complete digital copies of brains. 5:19 Okay. They use data about brain function, human behavior, and deep learning Oh, I see. To create something that mimics aspects of human intelligence, but maybe more affordably and quickly. That's a practical approach for sure. Yeah. 5:34 Full brain emulation is a huge undertaking. It is. They've got reports and workshops on all these neurotech ideas. Clearly, they're putting a lot of thought into this. Now the third area is all about security. 5:45 Security. Yeah. Just like we try to keep our computers safe, they're looking at ways to make AI systems secure. Oh, that's super important. Like, as AI gets more connected and eponymous Uh-huh. 5:57 Security becomes essential to prevent unintended consequences. Absolutely. We need to be sure it's secure from both outside attacks and internal weaknesses. Right. And they're looking at adapting existing security measures to the unique challenges of AI. 6:11 Okay. So Things like PILO, the principle of least authority. PILO. Meaning, a system only has the permissions needed for its specific task. Right. 6:20 And using hardened hardware to ensure the underlying technology is secure. Those are good first steps, applying established security principle. They are. Pill a helps limit damage if a system is compromised. And strong hardware provides a basic level of trust. 6:34 Exactly. They're also interested in automated red teaming and automated vulnerability discovery. Those sound intense. Yeah. So they want to use AI to find weaknesses in other AI systems before someone with bad intentions can exploit them. 6:50 Oh, wow. So, like, AI hackers testing our AI defenses. That's a good way to put it. That's a powerful idea using AI to find vulnerabilities. It is. 7:00 AI can analyze code and find patterns of weakness much faster than humans. Yeah. Being proactive like that is gonna be critical. And then there's cryptography, which is about secure communication. Right. 7:10 They're looking at techniques that could allow different AI systems to trust each other even if they don't trust the entities behind them. That's tricky. Yeah. The material mentions workshops focusing on the intersection of cryptography, security, and AGI, artificial general intelligence. Okay. 7:25 AI that can do anything a human can intellectually. Oh, right. So cryptography is crucial for building reliable AI ecosystems. Absolutely. Secure communication, verifying identities, ensuring data integrity. 7:39 Exactly. All of that's gonna be essential. Now the fourth area is safe multi agent scenarios. Multi agent scenarios. It's about considering how multiple AI systems and humans interacting with them behave together. 7:51 Okay. Yeah. That makes sense. As AI gets more advanced Yeah. It won't be operating in isolation. 7:57 Right. We need to understand how they interact with each other and with us to avoid harmful outcomes. Definitely. So how do we prevent AI systems from colluding in harmful ways or deceiving each other? That's a big challenge. 8:11 As they get more intelligent Uh-huh. They might develop strategies we haven't even thought of. And instead encourage them to cooperate beneficially. Right. This is where game theory, which looks at strategic decision making, comes in. 8:23 I see. The source material mentions principal agent problems, where one party is supposed to act in the best interest of another but might have conflicting interests, like when you hire someone to do something for you. Exactly. They're exploring how this plays out with intelligent AI agents. So how do we ensure AI truly acts according to our intentions? 8:44 Precisely. There's also research into the foundations of cooperative AI and secret collusion among AI systems. It's a really complex area. It is. They're looking at how to build cooperation and transparency into these systems. 8:57 So we can monitor what they're doing. Right. Okay. Looking at these four areas Yeah. It's clear why the Foresight Institute is focused on these less explored approaches to AI safety. 9:09 Yeah. They're thinking long term, aren't they? They really are. As AI advances quickly, we can't just focus on individual systems behaving well on their own. We need to look at the bigger picture. 9:19 Exactly. Their emphasis on strengthening humans through neurotech and how AI systems interact is a more comprehensive strategy. It really is. They're recognizing that this is a multifaceted challenge requiring solutions at different levels. And they're not afraid to support ambitious ideas, ones that could have a big impact soon. 9:37 They're willing to take risks for potentially big rewards. That's what makes this grant program so unique. Absolutely. You know, they're looking for those potentially game changing projects even if they seem a bit unconventional. Right. 9:49 The ones that can make a real difference in AI safety. You shared some examples of the projects they're funding. It's encouraging to see these ideas in action. Yeah. Tell me more about those. 9:59 Well, at the University of Cambridge, they're working on automating AI safety research work flows. Oh, wow. They're using AI to generate new research ideas and even write code for safety research. That's amazing. Like fighting fire with fire. 10:12 Exactly. Then at University College London, a project called NeuroTrust is using large language models. Those are the things that power chatbots. Right? Yeah. 10:22 To identify reliable science for developing neurotech safely. That's so important given the ethical and safety concerns around brain computer interfaces. It is. A strong scientific basis for neurotech development is essential. Absolutely. 10:36 And then we get even more futuristic. At Nephilabs, they're working on functional low fi whole brain emulation in the mouse. Wow. A step toward understanding how to achieve complex brain emulations. Fascinating. 10:50 And MIT and EON systems are working on a brain emulation review and roadmap to encourage wider discussion and planning for the future. So they're looking at both the nitty gritty and the bigger picture. Exactly. And, of course, there are projects on multi agent scenarios. Groups like the Cooperative AI Foundation and MetaEye are looking at how AI systems might collude and how they learn in social environments. 11:13 Those are really complex questions with with huge implications. They are. It's a really diverse and innovative collection of work. It seems like the Foresight Institute has a very forward thinking approach to AI safety. They do. 11:24 And they're very specific about how they choose which projects to fund. Yeah. How does that work? They look for projects that could make a real impact and value open sharing of research results, which is crucial for progress. Right. 11:37 So everyone can learn from each other's work. Exactly. And they're willing to take a chance on risky ideas if the potential payoff for AI safety is significant. That makes sense. They have to balance risk and reward. 11:49 They do. And just so you know, they have regular opportunities to apply for these grants. The next deadline is March 31. Oh, cool. So if any listeners are inspired They could explore that. 12:01 Absolutely. So we've only scratched the surface here, but these underexplored areas offer a glimpse into some really creative approaches to AI safety. Yeah. It's been eye opening for me. Thinking about all these angles from using AI to automate safety research to neurotechnology, security, and multi agent interactions Yeah. 12:22 What stands out to you as the most intriguing challenge? For me, it's how security and multi agent systems come together. Uh-huh. As AI gets more connected, ensuring those interactions are secure and trustworthy is critical. Definitely. 12:35 The potential for AI systems to collude or manipulate situations in those complex environments Yeah. That's a huge challenge that needs some really innovative solutions. It does. And as you said, we've really only touched on these topics. Each one could be a whole series. 12:49 Absolutely. There's so much to explore. But, hopefully, we've given you a clearer understanding of some of the cutting edge work in AI safety. I think so. And it highlights how many different fields have to work together on this. 13:00 It really does. Thank you for listening in. Subscribe and follow Colaberry on social media. The links are in the description. And check out our website, www.colaberry.ai for more insights like this.
