Large Language Models are changing how non-technical people engage with AI, and those learnings are permeating into the world of defence. Today, a startup called SE3 Labs, which has built a spatial AI platform for people to better understand and command autonomous systems, is emerging from stealth armed with funding and notable traction in the market.
“Stealth” is the term the company is using because it’s talking to the media about its technology and customers, and disclosing investment, for the first time. But under the radar it’s building a business across ground and aerial domains to control swarms of devices for military, public safety and industrial purposes.
The core of its product is a piece of proprietary hardware that sits on a device, using its own sensors and others on the device to develop a picture of a physical environment and the objects within it, to then determine how to respond to that environment. It has also developed a natural language platform (existing LLMs fine-tuned for its purposes) to enable people to read and respond to the spatial AI data more efficiently.
SE3’s business being driven both by demand in the market, and the company’s firm belief that the AI tooling that it’s building is the missing link that will help autonomous systems work better in the future.
“We think autonomy and spatial intelligence will define European defence over the next 50 years,” Lukas Köstler, the CEO and co-founder of SE3 (pictured above), told Resilience Media in an interview.
The startup said that it is being “continuously” used by the Bundeswehr, in Ukraine, and by other (unnamed) allied forces in “multiple” defence contracts.
Köstler confirmed to Resilience Media that the startup has raised €5.5 million from investors that include Lakestar, Seedcamp, the Berlin early-stage founder incubator EWOR (where Köstler is a fellow), the Sequoia Scout Fund, Plug and Play, TwinTrack and others. (The Sequoia connection is one to watch as it moves on to raise more money.) He declined to comment on further fundraising and valuation.
SE3 is also working with third parties to integrate its software. This includes the German/Swiss/American juggernaut Auterion and, in an MOU announced Thursday, with ERC to equip the latter company’s VTOL systems with SE3’s AI.
The name SE3 is a reference to the mathematical framework used to describe the position and orientation of rigid objects moving in three-dimensional space, and this is essentially the focus for SE3 Labs.
SE3 is working in a crowded market, with younger startups, “scale ups” like Helsing and Stark, and prime contractors all building AI-powered solutions. These have been spurred in part by wars in Ukraine and the Middle East, where drones have dominated fighting and AI is the tool that both sides are using to try to gain an edge in every domain, from the physical battlefield to cybersecurity and “grey zone” warfare.
Within that, there has been a wide proliferation of AI-based, spacial-awareness tools for defence, and specifically for autonomous systems to, well, behave autonomously. A number of them have been developed in-house by the companies making the drones and ground vehicles that need them to operate, meaning they are already in theory tightly integrated into the hardware they are intended to enable.
Köstler believes that while some drone makers may be working on their own in-house spatial tooling and systems, there will be a strong market for specialists in the “space” like SE3 offering a modular solution that can be used in conjunction in more customised systems.
And besides this, there are a lot of shortcomings in the spatial AI market that still need fixing.
Specifically, many of these systems are less ‘intelligent’ than you think they might be: they work mostly on pre-programmed routes and beacons, and so when something unexpected is encountered, they are not able to cope with that. On top of this, the sensor data that they produce for analytics, and in turn the responding commands that a human might want to send that hardware, are caught in a data traffic jam. SE3 aims to unpick all of this.
To be clear, SE3 is not the only startup pursuing this concept. Stanhope AI, a London-based spinout from UCL, is building an “active inference” model based on neuroscience research that is intended to help AI systems in physical environments make “decisions” that better respond to unexpected circumstances and improve how humans can communicate with those systems. Stanhope also aims to improve computational and energy efficiency by positioning its processing directly on devices.
SE3 has an interesting backstory that could be one reason why investors and customers are signing up early to work with the startup.
Köstler, who studied mathematics and engineering, said he first started thinking about the idea for SE3 when he was working for Tesla in the US. Inspired by the very technical deep dives that Elon Musk made into the company’s self-driving programme, he knew he wanted to return to Europe to build a company that also solved a hard problem that could benefit that part of the world. He turned first to Professor Daniel Cremers — a distinguished, prize-winning German academic who he had studied under as a post-graduate — and another student of Cremers’, Simon Klenk.
The thinking went a little like this, he said: if we take the advances in LLMs and combine that with the experience we have in 3D computer vision, could we build a new form of AI that really understands three-dimensional space?
“The goal would be to crack the ‘spatial Turing Test,’ where the system could answer a question about a 3D environment — or a battlefield — as well as a human could, given the same data,” Köstler said. Cremers and Klenk were all in, Köstler said, and respectively became his co-founders and respectively the Chief Science Officer and CTO.
The company says that its wider team has been published collectively in 400 publications with over 87,000 citations beyond that. Beyond academic cred, collective industry experience includes time at Nvidia, Tesla Autopilot, BCG, Skydio, and Isar Aerospace.
“Many teams excel either at frontier research or at operational deployment. SE3 is unusual because it brings both together in one culture. The team is deeply technical, close to the customer and unusually low-ego, yet the ambition is enormous: to build technology that becomes a core part of how autonomous systems operate in the physical world.” said Enrico Mellis, who co-led Lakestar’s investment in SE3, in a statement.
The bigger picture with companies like SE3 is that they are laying the groundwork for more foundational AI technology development in Europe, a priority for governments in the region. If technologists are not sourcing and building the full stack from chips and data centres through to foundational AI and the hardware and services built on top of it, then ownership of state-of-the-art solutions for some aspects at least gives more leverage at the table, and that means more resilience.
“Europe’s sovereignty depends on its ability to build and scale the critical capabilities that will define the next era of defence. SE3 is developing one of those capabilities, and we believe it will be essential to Europe’s security, resilience and technological independence,” said Dr. Klaus Hommels, founder and chairman of Lakestar, in a statement.
