Paul Kocialkowski's coding blog

Free software, programming and stuff

A hacker's journey: freeing a phone from the ground up, first part

Written by Paul Kocialkowski - - no comments

Every once in a while, an unexpected combination of circumstances ends up enabling us to do something pretty awesome. This is the story of one of those times. About a year ago, a member of the Replicant community started evaluating a few targets from CyanogenMod and noticed some interesting ones. After some early research, he picked a device: the LG Optimus Black (P970), bought one and started porting Replicant to it. After a few encouraging results, he was left facing issues he couldn't overcome and decided to give up with the port. As the device could still be an interesting target for Replicant, we decided to buy the phone from him so that I could pick up the work where he stalled.


Upon receiving the item, I decided to dig a little more into what that device was made of and found that very precise technical documentation about it leaked online (EN_LG-P970_SVC_ENG_110415.pdf). This mostly contains all the electrical schematics of the phone, a precise list of the components used and a few other interesting things. Such in-depth documentation is rather rare for such mainstream devices: we usually get service manuals with some of that information, but seldom something to reach this level of completeness. That really got me interested in the device.


In the meantime, I found a few interesting threads at XDA mentioning the bootloader of a couple of OMAP4 devices from LG. Apparently, LG leaked the private keys for signing the first bootloader on these devices in a release of the bootloader's (copylefted) source code. As often on OMAP devices, the bootloader is X-Loader, a minimalistic copy of U-Boot source code that can run with a small memory footprint. While all of this is great fun, it's nothing we can seriously use in Replicant. At best, it enables developers to play around with the device's bootloader. And we know all too well how much of an issue signed bootloaders represent to software freedom on those devices, so this is kinda nice. Ironically, those private keys aren't the only things that we leaked aside free software releases for these LG OMAP4 devices: that's exactly where the recent PowerVR Series 5 source code leak originates, rising its own share of issues for free software.

Seeing that people were playing around with bootloaders on other LG OMAP devices got me interested in my recently-acquired LG Optimus Black. A quick look at the first bootloader's binary data revealed that it is X-Loader as well, but without the usual headers for signature checking. In addition, the second stage bootloader turned out to be U-Boot on early versions of the devices, which was replaced by LK on the ICS update. Nevertheless, X-Loader and U-Boot are copylefted free software, so I went ahead and tried to grab the released source code from LG. After downloading a few different archives, I finally spotted the one with the bootloaders' source code. All of it looked nice and nearly complete (only a small piece of software specific to LG, called FOTA, was missing from the release and broke the build, but I found equivalent code in one of the OMAP4 device's source code and it turned out to be nothing of interest).

Signature checks

At this point, I just couldn't stand in place as one question was left unanswered: can I actually build and run that code on the device? That is, are signature checks enforced on the LG Optimus Black (P970)? Even if the lack of dedicated header on the binary was a good sign, I am usually not too optimistic about these things, so I wanted to double check. As it turns out, OMAP platforms are also incredibly well documented by TI, so I just grabbed a copy of the Technical Reference Manual (TRM as we call it) and looked for a way to identify clearly whether the device enforces signature checks. Of course, that part is not publicly documented in-depth, but it seemed very clear from the documentation that OMAP SoCs are produced in (at least) two different fashions: a General Purpose (GP) version that doesn't check the first bootloader's signature and a High Security (HS) version that does check those signatures. Usually, GP versions of OMAP SoCs are shipped on reference or development boards, such as the BeagleBoard. They also enable projects like the GTA04 to create a device that runs a free bootloader. However, it is rare to see GP OMAP chips being used on mainstream devices. While very happy from that finding, I still needed a reliable way to figure out whether the Optimus Black is using a GP or HS version of the OMAP3630. And it turns out that a couple of registers hold that information. One of them, CONTROL_STATUS is 4 byte-long and holds the device type with a 1 byte offset, at 0x480022f0. It also holds the boot order information (SYS_BOOT) on the lower byte. On android, it can easily be accessed with devmem:

devmem 0x480022f0 16
The higher byte is 0x03, which clearly indicates a GP device: HS devices usually return 0x02.

This post is part of a series of articles about freeing the LG Optimus Black (P970):

Android with mainline Linux kernel support on the Goldelico GTA04

Written by Paul Kocialkowski - - 3 comments

During the past few months, I've been working on bringing Replicant 4.2 support to the Goldelico GTA04, a montherboard replacement for the Openmoko FreeRunner (GTA02) that is manufactured by Golden Delicious with the intent of being free software friendly. The board design is also released under a free licence. There is an active community of developers and users dedicated to the GTA04 and other similar projects, brougt together under the hood of OpenPhoenux. Other similar projects include the upcoming Neo900, a motherboard replacement for the Nokia N900 using similar hardware as the GTA04 and also aiming to be free software friendly, with a particular emphasis on security and privacy features.

I recently published a blog post covering the status of the Replicant 4.2 port to the Goldelico GTA04 on the Replicant blog, with a nice video highlighting which features already work well. However, in the present post, I would like to cover some of the technical aspects of the port, especially regarding the use of a recent mainline Linux kernel with Android 4.2.

Currently, Android devices ship with Linux 3.4, after a long time of using version 3.0, which started with Android 4.0 Ice Cream Sandwich. To this day, the reference common Android kernel repos have branches with Linux 3.10 and experimental work is done on Linux 3.14. That's pretty close to the Linux 3.12 version we're using on the GTA04!

Opportunistic suspend

The first challenge to overcome was dealing with suspend and resume. Android uses a particular mechanism to implement opportunistic suspend, using wakelocks: both the kernel and userspace can register these locks to prevent the device from going to suspend. In Android kernels, there is another mechanism that allows some non-critical chips to reach suspend state before the rest of the system: this is earlysuspend. As the whole Android system is built around the concept of wakelocks to handle power management, something similar needed to be implemented in the mainline Linux kernel. After efforts from the Android kernel maintainers, wakelocks were implemented in a clean fashion some time ago. In order to implement opportunistic suspend, a separate interface was implemented on mainline Linux, known as autosleep, which uses different nodes than /sys/power/state (that the Android kernels use directly for opportunistic suspend). Starting in Android 4.1, a library was added to Android in order to detect and handle these different modes.

So thankfully, everything was already in place to use autosleep properly on Android 4.2. Except that it didn't work. This turned out to be because of a dedicated capability that was missing from the Android userspace: CAP_BLOCK_SUSPEND. It turned out to be easier to just revert the patch restricting access to wakelocks to users with that capability in the kernel.

Android USB Gadget and Android Debug Bridge

Android also went their own way in implementing USB device drivers for the various things that are used by Android: mass storage, rndis for USB networking, MTP for file access and a couple others, including ADB. The Android Debug Bridge (ADB) is a great way of debugging what's going on with an Android device, providing easy access to the logs, to a shell and file transfer (these are the features of ADB I use the most). All of that is not integrated at all in the mainline Linux kernel, so there was some substantial work to do here. The first thing to do was importing the related commits from the android-3.10 branch of the common Android Linux kernel. All of that built nicely with only minor code corrections, to follow some API changes in 3.12 and some features like rndis or MTP worked right away, but the most interesting part was left not working: that's the Android Debug Bridge. A few kernel versions back, there used to be a dedicated composite function driver for ADB, but a commit by one of Android's kernel maintainers totally gets rid of it, calling it obsolete with no further information. While attempts to restore it failed, I tried to find out in details why it is obsolete and if that meant the final death of ADB, that I found rather convenient. Thankfully, someone found out what happened and wrote about it: the dedicated ADB driver was being replaced by another one using FunctionFS, a more flexible and generic way of implementing such drivers, directly from userspace. It turns out that FunctionFS support for the userspace ADB server was merged nearly entirely in Android 4.2. I had to backport a missing fix to have it fully working properly and also had to import adaptation a patch in recovery to have sideload working with FunctionFS too. A few bits were also needed in the initialization procedure to have things set up right. Once all of that was done, it could finally run flawlessly!

Headset/headphones detection with SoC Jack

So Jack's a good guy. Everyone knows about that. Whether he's saving the homeland from yet another threat or just letting us know something of interest just got plugged in one hole or another, it always feels great to hear from Jack. However, the Android kernel guys didn't seem to appreciate his participation in the show as much as we all do, or at least until recently.

The traditional way of reporting a headset or headphones plug/unplug in Android kernels was using a switch called h2w, reporting these events to userspace. The mainline kernel prefers another approach, using our beloved Jack SoC architecture. It also provides a convenient way of reporting button pushes, which is quite nice. So instead of rewriting it all using the h2w switch, it struck me that there is a frameworks config option to politely ask Android to give Jack some consideration. And when it does that, everything works great, including button press reports!

The Samsung Galaxy back-door was bullshit. Really?

Written by Paul Kocialkowski - - 18 comments

A few days ago, I disclosed (on behalf of the Replicant projet) our research regarding a back-door found in a proprietary program running on Samsung Galaxy devices' applications processor. This back-door lets the modem perform I/O operations on the device's storage.

For the full statement, please refer to the article posted at Free Software Foundation's website. The relevant technical analysis is available at the Replicant wiki and a complementary statement was issued at the Replicant blog.

In the few hours following the publication, an outstanding number of technology-oriented websites relayed the news, including Phoronix, Slashdot, LWN and XDA-Developers. I'm very glad the press found interest in that research and I'm confident it'll help more and more individuals realize the importance of being in control of their computing: that is, to understand what's at stake with free software.

A few recent developments particularly caught my attention: Ars technica bothered to ask an actual security researcher, Dan Rosenberg his thoughts on our findings. Good thing they decided to go deeper than only duplicating the information. On the other hand, Samsung issued a statement about this issue:

Samsung takes the security of its products extremely seriously. We have investigated the claims that have been made and can confirm that there is no security risk. The Free Software Foundation’s recent allegations are based on a false understanding of the software feature that enables communication between the modem and the Application Processor chipset.

Mostly, the point that is argued by Dan Rosenberg is that there is no evidence of any ability for a remote party to use the back-door, nor any known exploit to make use of it remotely. As a matter of fact, we didn't look at how this could be used over the air: this was not the point of our research. The problem we intended to highlight is not so much about how in practical terms an intruder could use this anti-feature remotely to access and modify the data stored on the device, but rather to show that a particular proprietary software implements a feature that could be used to let the modem gain data I/O access over the device. This is where we find the back-door to be: at the interface between the modem and the applications processor. We do consider the modem to be an “unknown” area that offers no guarantee at all regarding security, since it is running proprietary software. Hence, we believe it is relevant to assume the worse and consider it compromised and subject to remote control. Several indications tend to make us think this is actually what is going on: Craig Murray described how a mobile phone had been remotely converted to a spying device in Murder in Samarkand. Considering the recent revelations regarding the practices of several governments' intelligence agencies, we find it hard to believe there is no way modems cannot be remotely compromised.

The goal of our action was to make people aware of that particular issue. One might consider it to have no value, provided they don't think modems can be remotely compromised and others might see it as a crucial security flaw in the event the modem is compromised, as we do. The fact that it was implemented for another purpose or was not intended to be used in malicious ways doesn't change anything at all: an attacker with remote access to the modem will be able to issue the incriminated requests. There is no possible “false understanding”, in the way Samsung seems to imply here.

For the record, we didn't at any point intend to distort the truth to bring attention to our project or our research, nor did we intend to ruin Samsung's reputation. We simply felt it was our moral responsibility to spread the word about it. I believe anyone can decide for themselves whether they have faith in Samsung's good word that this introduces no further security risk, but let it be clear that it doesn't get any more certain than what good faith can provide. We are still looking forward to working with Samsung to make things right, in case they decide to abandon their current position of denial.

Impressions from FOSDEM 2014

Written by Paul Kocialkowski - - no comments

Well, this is mostly going to be about my experience at FOSDEM 2014. Thanks to being a student in a city that features an easy-to-access airport, I was able to attend this year's edition. While my travel schedule was real tight, I only found out the plane was landing not in Brussels, but in Charleroi a few days before departure. Thankfully, it wasn't too late to find another schedule that made it possible for me to arrive at the Friday Beer event around dinner time. The cafe, and the aisle that leads to it were incredibly crowded, to the point that it was barely even possible to make it to the entrance. And once there, despite the fact that the cafe had been reserved for FOSDEM attendees only, I sadly couldn't get in, since the interior of the cafe was apparently full as well. While waiting near the entrance, I was able to see Greg K.H., first of the numerous giants of the free software community I stumbled upon at FOSDEM.

The next day, I was really amazed to see so many people going to FOSDEM, on the way to the event. Public transportation was really filled up with free software hacktivists! Arriving at the ULB campus used for FOSDEM, it really felt spacious and seemed appropriate for an event that big. Lots of interesting discussions took place after the first talk I attended: it was really nice. There were also numerous stands, mostly divided in two buildings, with many interesting people to talk to as well. In the AW building, I enjoyed the Coreboot/Flashrom stand (free software BIOS), Hackable-Devices (apparently focusing on micro-controlers recently), OlinuXino (Allwinner single board computers) and OpenPandora (free software gaming device) with a prototype of the new OMAP5-based version of the device, running apparently really well on GNU/Linux without graphics acceleration blobs. However, the big slice of stands was in the K building, including popular GNU/Linux distros such as Debian or Fedora and desktop environments such as GNOME and KDE. The FSFE was also there, with real good-looking flyers about their Free Your Android campaign, promoting Replicant and F-Droid! I also spent some time at the CaCert booth, and frankly, I was amazed by the depth of the identity verification process. First off, having a single official document to prove of your identity is not enough for these guys, and things get worse when the signature on the ID card (kindly provided by mom at a time I couldn't sign it for myselef) doesn't match the one you produce. Not to mention you have to sign the paper before their eyes, else it's not valid. The ID card itself is also checked to be genuine, with the many UV lights at their disposal and descriptions of the expected results. So I was really surprised how strict the whole process is and I think it's really great that they are taking this very seriously.

The rest of the afternoon we spent in the legal devroom, where I could meet many great people such as John Sullivan, Bradley Kuhn, Karen Sandler and others. John gave a talk about Javascript, with all relevant infos regarding how to escape the Javascript trap, followed by Chris Webber about free network services and introducing the road map for the next MediaGoblin releases, with a really good-looking video explaining the goals. Chris was probably the most enthusiastic speaker I saw at FOSDEM: he was really passionate about his talk and created a wonderful ambiance in the room (summoning that feeling that urges you to get involved in something bigger than you), so kudos to him for being that good. After a while, we headed to dinner with the Coreboot crew, very nice and interesting people. It was a real pleasure spending time in their company.

On Sunday, we had to rush to get to FOSDEM in time for the F-Droid track by Daniel Martí, followed by an introduction to the linux-sunxi community by Olliver Schinagl, who kindly poked the Replicant project during the talk mentioning that I've been promising Replicant for Allwinner devices for the last six months. Olliver's talk really made me realize what an amazing platform Allwinner is, so I just went ahead and ordered a variety of Allwinner devices to port Replicant to, so we should get there in the near future! On the way to lunch, I quickly saw someone I believe to have been Harald Welte: while I would have loved to have thanked him for his great work, he went by faster than it took me to realize who he was. Time went by, and in the afternoon, I could only attend the Lima talk by Luc Verhaegen before leaving. The project is apparently steadily moving forward, however, without any mind-blowing demo this time. I really had to leave fast after the talk, to catch up with my transportation schedule.

I really have the best memories from FOSDEM, it was really nice and there is no doubt I'll attend next year's edition, hopefully presenting a talk about Replicant there. It was also really nice to see people grateful for the work I'm doing on Replicant. Such huge community gatherings are the best to gather the motivation to keep working on a free software project: actually meeting the community brings a whole different picture compared to what contributing to free software usually feels like individually, hacking alone in my dark room.

Missing proprietary firmwares in Android systems

Written by Paul Kocialkowski - - 57 comments

Firmwares are programs that do not run on the main processor of a computer: instead, they runs inside separate chips that have a dedicated functionality. Most of the time, firmwares are proprietary programs, which do not respect the user's freedom. When they are not already installed in the chip they run in, the main processor has to load them into the chip, which requires the firmware to be distributed as a file. Since they are proprietary software, I think firmwares shouldn't be shipped with any operating system, not should any operating system ever advise the user to install them.

However, people sometimes need a functionality that depends on proprietary firmwares so bad that they would rather use a system that ships and encourages the use of proprietary software, including the needed firmware, over a free system. In that case, it is better for their freedom that they use a free system with only the proprietary firmware installed, over another system that contains dozens of other proprietary pieces, including ones that also run on the main processor and are able to compromise the whole system (from a security point of view), on top of not respecting the user's freedom.

On Android devices, people are often facing this choice and look around for instructions on how to install only the missing firmwares so that they can avoid installing a system that contains even more proprietary bits. While these instructions cannot be released on free system's official documentation pages, for the sake of not encouraging the use of proprietary software, it makes sense for me to publish such instructions on my personal pages. I have written scripts that extract the firmwares from CyanogenMod installation zips for a few Android devices and push them to the device: cm-firmwares.git

The procedure to install the firmwares is the following:

  • Grab the CyanogenMod installation zip matching the needed version from the CyanogenMod Downloads website (make sure to select the stable release).
    For instance, the CyanogenMod 10.1.3 installation zip for the Galaxy S 3 (I9300) is at:
  • Clone the cm-firmwares git repository, with the branch matching the CyanogenMod version.
    For instance, for CyanogenMod 10.1.3:
    git clone git:// -b cm-10.1.3
  • Connect the device via USB, make sure ADB is enabled and allowed to run as root on the device. Check that the device is connected and allowed:
    adb devices
    If not, it might be necessary to run the process as root (or prefix the following commands with sudo).
  • Run the script (on the host computer) with the device codename as first argument.
    For instance, for the Galaxy S 3 (I9300):
    ./ i9300
  • Alternatively, it is possible to create a zip containing the firmwares and install it afterwards, using recovery.
    For instance, for the Galaxy S 3 (I9300):
    ./ i9300 zip

If you wish to push only certain firmwares, feel free to edit the text file matching the device codename to remove the ones that are not needed before running the script.