How Microsoft built its folding Android phone

Microsoft is returning to making phones this week, as part of an ambitious project to usher in a new era of dual-screen and folding devices. The company has spent around six years developing Surface Duo, its Android-powered device that folds out to be a phone or a miniature tablet. It’s taken Microsoft years to get the hardware and software right, but the company firmly believes now is the ideal time for something new. When it goes on sale tomorrow, we’ll see if the company got it right.

This is the story of how Microsoft’s new folding Android phone came to be.

Panos Panay, Microsoft’s chief product officer, is known for his onstage energy and for constantly feeling pumped, but when it came time to build his dream Moleskine-like device, he spent months walking around in secret with a peculiar piece of hardware in his pocket. “We literally had two pieces of metal and a hinge that we put together,” explains Panay in an interview with The Verge. “We had this piece of metal that I carried around in my pocket for months.”

It was the early and primitive form for what eventually became the Surface Duo, and Panay spent months analyzing things like “fidget factor” and measuring how often he opened and closed the device. “Does it fit in your pocket? Can you sit on it? How big would the screen have to be if it wasn’t a traditional slab you were holding every day?” were some of the many questions he and his team were looking to answer.

Panay has been thinking about a pocketable or a small Surface device for years, even describing the idea as his “baby” at one point, but it was a painful journey to get to the Surface Duo. Work began on the Duo just after Microsoft had canceled the Surface Mini, an eight-inch tablet running Windows. Microsoft had been planning to launch this smaller Surface, but it ultimately wasn’t the right time back in 2014.

“It was emotional to stop,” says Panay. “Products are a reflection of the people that make them, that’s how we talk as a team. These products become who you are at work, and we spend a lot of time at work.”

If there’s anything that Microsoft has learned from trying to harmonize Surface hardware and software, it’s that timing is everything. “One of things that we’ve really developed a strong muscle for is the ability to know timing for when a product is right,” explains Steven Bathiche, who oversees all hardware innovation for Microsoft devices like Surface, Xbox, and HoloLens. “Timing is a thing I’ve learned is the most important thing, it’s more important than the idea itself in fact. All the ingredients really have to be right. This is probably why in the past you haven’t seen some of these ideas really make it through.”

Image: Windows Central

That’s why the Surface Mini never launched. “Mini just wasn’t right because it didn’t have the apps for the form factor,” explains Panay. “There were a lot of challenges for Mini. I still have my Mini, it’s running Windows RT, but it didn’t have everything it needed for that form.” Surface Mini would have run Windows, which meant apps would have needed to be updated for the form factor, and the size and OS choice meant you’d still have to carry around a mobile phone.

While the Surface Mini cancellation was painful, it was an important part of the history of the Surface Duo and influenced what hardware choices were made. Microsoft moved on to a device codenamed “Andromeda” before shifting to the Surface Duo. While the Surface team is reluctant to talk about Andromeda in detail, sources tell The Verge it was a similar dual-screen device that was thicker and bulkier than the Surface Duo. Microsoft had planned to run a custom variant of Windows on Andromeda, and the camera hardware was also different to what exists in the Duo.

“At that time, it was different hardware, it’s not the same hardware… it’s not even remotely close,” says Panay. The key principles of Surface Duo existed in Andromeda, though: two screens side by side, with a hinge that allowed the device to fold out into place. “We’re in so many generations later of development [with Surface Duo], we understand how thin it is, we understand reliability, we understand the robustness of the screens. It has been generationally made, it’s years in the making.”

Surface Mini and Andromeda helped Microsoft experiment with the idea of a pocketable Surface device, and the learnings have helped. The idea of a foldable display, like what’s found on Samsung’s Galaxy Fold, was quickly discarded in favor of the reliability of two flat screens. “It wasn’t difficult for us to realize that taking a screen and folding it wasn’t the right option for this product,” explains Bathiche. “We wanted glass. We wanted glass that wouldn’t scratch because we also wanted to give you a pen. We also wanted a 360-degree hinge, and we wanted to be able to do that without compromising the thickness of the device.”

Microsoft decided early on that there wouldn’t be a cover display, meaning you have to make an intentional choice to unfold the Duo to see the displays and even notifications. These fundamental choices allowed Microsoft to focus on important hardware aspects like keeping the device thin, creating a reliable hinge, and the idea of bringing the dual-screen computing you’re used to on your Windows desktop PC into your pocket.

“One of the things that we learned on Surface Mini that was actually really valuable for us is when you’re designing these super thin structures you want to come up with a mechanical architecture,” says Pavan Davuluri, a distinguished Microsoft Surface engineer. “You’re coalescing and condensing a lot of the mechanical components — structural, thermal, and RF pieces — into a single internal frame.”

This internal frame has allowed Microsoft to spread the Surface Duo components across both sides, while keeping the device just 4.8mm thin. “Most phones are buckets essentially, it’s a bucket that you fill with parts and put glass on the backside,” explains Davuluri. “That’s not how Surface Duo was built. Duo was built with an internal frame that really is the development scheme for integrating all these dual-system components.”

Microsoft experimented with a few different internal architectures. One didn’t even have dual screens, and another was just a phone powering a second screen on the opposite side. “We had another variant where we were using wireless connectivity vs. wired,” says Davuluri, so the second display wasn’t physically connected.

These prototypes would have meant the Duo would end up being thicker or one side would be heavier. “It was definitely easier to make one side thick and put everything on one side, and then make the other side super thin,” admits Bathiche. “We chose the hard way from an engineering standpoint, but we’re really proud of the result of the design… the device is symmetrical, which evokes its function as well.”

These hardware choices weren’t always easy, though. LCD displays would have made a lot of sense over OLED, but they would have added to the thickness of the device. There were intense debates inside Microsoft around the hardware that went into Surface Duo. “The whole LCD or OLED debate was a real one,” explains Bathiche. “I was really worried about it, because I knew some of the challenges we’d have to overcome that OLED didn’t really solve.”

Microsoft created a prototype Surface Duo with OLED displays, and the first hardware sample shocked Surface engineers. “When we got our first prototypes back… we opened it up and looked at it for the first time and realized ‘Holy cow, there’s a color shift that we didn’t essentially account for that happens when you look at OLED offscreen,’” recounts Bathiche. “Displays are like snowflakes, there’s no two alike.”

OLED color shifts aren’t easily noticeable on a single display, but when you put them side by side, it’s a different story. Microsoft had to work with display manufactures to widen the color viewing angles, and configure them to have the same contrast, color uniformity, and timing.

There were also intense debates over the thinness of the Surface Duo. “People wanted to violate thinness every which way, across the board you can pick any of the tech that went into it,” explains Pete Kyriacou, vice president of Surface program management. These debates and internal tension ultimately led to Microsoft creating an incredibly thin device.

“This is the most emotional product we’ve ever created,” says Panay. “A lot of that comes from tension. To get that diamond out, there had to be a lot of it. Microns mattered, not millimeters, microns. I remember being in meetings and being like ‘Come on, Pete, we’re talking less than a millimeter tradeoff that’s all we need to solve this product.” Kyriacou didn’t move on the thin focus, and the team had to find other ways to solve problems. “Maybe that’s why it took years to get to this product,” admits Panay. “There were moments like those because we pushed so many boundaries.”

Some of these hardware choices for dual screens and the device thinness have also led to the Duo missing things like 5G connectivity or NFC support. The camera isn’t what you’d expect to find on a flagship Android device in 2020, largely because of how thin the Surface Duo is. “We had to stay maniacally focused on the weight, the symmetry, and the battery life of the system,” reveals Davuluri. “That, in turn, drove the choice of what kind of sensor we picked, and what kind of optics system had to live in that footprint, and how we had to optimize the camera software experience.”

The camera module inside the Surface Duo is one of the smallest on the market to make sure it fits inside the 4.8mm thickness. Microsoft has optimized for both front and rear photography, but it’s obvious the camera will have some serious limitations.

Microsoft also experimented with other enclosure materials during its early Surface Duo mockup phase. Some prototypes included fabric or metal, similar to what we’ve seen on Surface keyboards. “We liked that fabric enclosure material because it gave us some properties in Surface Mini like our speakers were built out of fabric on that product at the time,” says Davuluri. “There were things we really liked about fabric… but it didn’t meet all of our requirements for our current generation and future generation products.”

Beyond the hardware experimentation, the software and OS powering the Surface Duo were also incredibly important. Microsoft had been experimenting with a custom variant of Windows for its Andromeda device, but the company switched directions and moved to Android. It wasn’t an easy decision to make.

“Bringing Android into the fold, that wasn’t the most simple of conversations all of the time,” admits Panay. “You have to explain that and you want your team on board and people believing it.” Apps were a limiting factor for a Windows- or Windows Phone-powered device, and Panay has previously admitted it’s the key reason the Surface Duo runs Android. Apps and even the Android software running on Surface Duo won’t be perfect just yet, as Google hasn’t fully optimized the OS for this type of hardware, but Microsoft is working with Google to improve Android.

“As we got into working with Android, it wasn’t about just doing things specifically for Duo,” explains Kyriacou. Microsoft has created a dual-screen architecture, drag-and-drop APIs, screen-aware APIs, and even hinge APIs that all make apps light up across both screens. “We wanted to make sure we were working with Google to get that back into the ecosystem, so it’s not a forked version of Android. This is about working with them to make sure this all accrues to app developers and Android.”

Microsoft hasn’t heavily modified or skinned Android with the Surface Duo, either. “Our goal from the beginning was to stay as true to Android as possible,” says Kyriacou. “Mainly for familiarity, but also to make sure the changes we would make for windowing or hinge angle / postures would be part of the Android operating system going forward.”

Software updates and OS tweaks are a lot easier to roll out than hardware changes, so expect to see the Duo improve regularly, especially when Android 11 arrives. Microsoft is also promising three years of Android updates for the Surface Duo, so it will benefit from any work Microsoft and Google are doing to improve Android for years to come.

Microsoft has been working on improving gestures and the keyboard experience on Surface Duo in recent months, and more improvements are on the way. “In addition to our normal fixes, we’ll also be updating features that drive a great dual-screen experience,” adds Kyriacou. “We will be on future versions of Android when the time is right.”

Microsoft’s folding and dual-screen ambitions don’t end with just the Surface Duo. The company is still planning to launch a Surface Neo device, powered by Windows 10X. Microsoft unveiled the Surface Neo last year alongside the Duo, complete with two separate nine-inch displays that fold out into a full 13-inch workspace. It was supposed to launch later this year. “Neo is delayed,” says Panay. “I wanted the right time to bring that product with the right experience. We believe in that concept and form factor and size. It will be a beautiful complement to Duo with Windows and I’m excited about it. It’s a product that’s near and dear to my heart.”

Surface Duo and Neo won’t be the only dual-screen devices Microsoft is creating, either. “I believe that different sizes will happen, and I actually believe different companies will make different sizes too, and I think they should,” says Panay. “We want dual-screen architecture to be prevalent, we want every app to work on these screens, and we’re fundamentally committed to that. It includes a roadmap of multiple sizes.”

How Microsoft balances the choice to run Windows or Android on different sizes will be interesting in the years ahead, especially as the company has been pushing Android closer to Windows with its Your Phone app. You can now run your phone’s Android apps beside Windows apps on a desktop PC, and it’s easy to imagine Microsoft may go further with this integration in the future.

“The next natural evolution is different [dual-screen] sizes,” says Panay. “Whether the larger moves into Windows where it’s appropriate to use the Windows codebase and software, and the smaller form factor uses Android, I think you can safely say that’s the right path with what we’re doing right now as a team.”

Panay believes Duo and dual-screen devices are here to stay, and the hardware will clearly evolve in the future. “Two screens matters. I believe when people start using it they’re going to adapt to these products, they’re going to fall in love with them.”

We might have to wait until foldable glass is a little more reliable before we see it on a Surface device, though. “I think it’s an exciting era of research, it’s one of the things we’re really on top of,” explains Bathiche. “We know all the physics problems that need to be addressed to deliver the experience that we really want to go after, but for us, not yet.”

Microsoft truly believes that the Surface Duo, and devices like it, will change the way people use mobile devices. That belief is rooted in the work the company has been doing in Windows for more than 30 years, allowing PC users to window apps, drag and drop content, and support multiple monitors to multitask.

Bringing that to mobile devices won’t be easy, and Microsoft is hoping the third time’s the charm after Windows Mobile and Windows Phone failed to make a dent in the mobile market. Just like Microsoft had to prove Surface tablets made sense in the first place, the company will once again have to demonstrate that there’s even a need for a device like the Surface Duo.

The future of mobile devices could go in a variety of different directions. Not everyone will have a need for a device like the Surface Duo immediately — or maybe ever. But then not everyone needed to check their email on the go or browse the mobile web when the first stylus-driven smartphones appeared. Microsoft is betting that behaviors will change, or as Panay puts it: “it’s a product that I believe is transformative for the future.”