Paul Kocialkowski's coding blog

Free software, programming and stuff

Android

Android with mainline Linux kernel support on the Goldelico GTA04

Written by Paul Kocialkowski - 27 june 2014 - 3 comments

During the past few months, I've been working on bringing Replicant 4.2 support to the Goldelico GTA04, a montherboard replacement for the Openmoko FreeRunner (GTA02) that is manufactured by Golden Delicious with the intent of being free software friendly. The board design is also released under a free licence. There is an active community of developers and users dedicated to the GTA04 and other similar projects, brougt together under the hood of OpenPhoenux. Other similar projects include the upcoming Neo900, a motherboard replacement for the Nokia N900 using similar hardware as the GTA04 and also aiming to be free software friendly, with a particular emphasis on security and privacy features.

I recently published a blog post covering the status of the Replicant 4.2 port to the Goldelico GTA04 on the Replicant blog, with a nice video highlighting which features already work well. However, in the present post, I would like to cover some of the technical aspects of the port, especially regarding the use of a recent mainline Linux kernel with Android 4.2.

Currently, Android devices ship with Linux 3.4, after a long time of using version 3.0, which started with Android 4.0 Ice Cream Sandwich. To this day, the reference common Android kernel repos have branches with Linux 3.10 and experimental work is done on Linux 3.14. That's pretty close to the Linux 3.12 version we're using on the GTA04!

Opportunistic suspend

The first challenge to overcome was dealing with suspend and resume. Android uses a particular mechanism to implement opportunistic suspend, using wakelocks: both the kernel and userspace can register these locks to prevent the device from going to suspend. In Android kernels, there is another mechanism that allows some non-critical chips to reach suspend state before the rest of the system: this is earlysuspend. As the whole Android system is built around the concept of wakelocks to handle power management, something similar needed to be implemented in the mainline Linux kernel. After efforts from the Android kernel maintainers, wakelocks were implemented in a clean fashion some time ago. In order to implement opportunistic suspend, a separate interface was implemented on mainline Linux, known as autosleep, which uses different nodes than /sys/power/state (that the Android kernels use directly for opportunistic suspend). Starting in Android 4.1, a library was added to Android in order to detect and handle these different modes.

So thankfully, everything was already in place to use autosleep properly on Android 4.2. Except that it didn't work. This turned out to be because of a dedicated capability that was missing from the Android userspace: CAP_BLOCK_SUSPEND. It turned out to be easier to just revert the patch restricting access to wakelocks to users with that capability in the kernel.

Android USB Gadget and Android Debug Bridge

Android also went their own way in implementing USB device drivers for the various things that are used by Android: mass storage, rndis for USB networking, MTP for file access and a couple others, including ADB. The Android Debug Bridge (ADB) is a great way of debugging what's going on with an Android device, providing easy access to the logs, to a shell and file transfer (these are the features of ADB I use the most). All of that is not integrated at all in the mainline Linux kernel, so there was some substantial work to do here. The first thing to do was importing the related commits from the android-3.10 branch of the common Android Linux kernel. All of that built nicely with only minor code corrections, to follow some API changes in 3.12 and some features like rndis or MTP worked right away, but the most interesting part was left not working: that's the Android Debug Bridge. A few kernel versions back, there used to be a dedicated composite function driver for ADB, but a commit by one of Android's kernel maintainers totally gets rid of it, calling it obsolete with no further information. While attempts to restore it failed, I tried to find out in details why it is obsolete and if that meant the final death of ADB, that I found rather convenient. Thankfully, someone found out what happened and wrote about it: the dedicated ADB driver was being replaced by another one using FunctionFS, a more flexible and generic way of implementing such drivers, directly from userspace. It turns out that FunctionFS support for the userspace ADB server was merged nearly entirely in Android 4.2. I had to backport a missing fix to have it fully working properly and also had to import adaptation a patch in recovery to have sideload working with FunctionFS too. A few bits were also needed in the initialization procedure to have things set up right. Once all of that was done, it could finally run flawlessly!

Headset/headphones detection with SoC Jack

So Jack's a good guy. Everyone knows about that. Whether he's saving the homeland from yet another threat or just letting us know something of interest just got plugged in one hole or another, it always feels great to hear from Jack. However, the Android kernel guys didn't seem to appreciate his participation in the show as much as we all do, or at least until recently.

The traditional way of reporting a headset or headphones plug/unplug in Android kernels was using a switch called h2w, reporting these events to userspace. The mainline kernel prefers another approach, using our beloved Jack SoC architecture. It also provides a convenient way of reporting button pushes, which is quite nice. So instead of rewriting it all using the h2w switch, it struck me that there is a frameworks config option to politely ask Android to give Jack some consideration. And when it does that, everything works great, including button press reports!

Missing proprietary firmwares in Android systems

Written by Paul Kocialkowski - 22 january 2014 - 60 comments

Firmwares are programs that do not run on the main processor of a computer: instead, they runs inside separate chips that have a dedicated functionality. Most of the time, firmwares are proprietary programs, which do not respect the user's freedom. When they are not already installed in the chip they run in, the main processor has to load them into the chip, which requires the firmware to be distributed as a file. Since they are proprietary software, I think firmwares shouldn't be shipped with any operating system, not should any operating system ever advise the user to install them.

However, people sometimes need a functionality that depends on proprietary firmwares so bad that they would rather use a system that ships and encourages the use of proprietary software, including the needed firmware, over a free system. In that case, it is better for their freedom that they use a free system with only the proprietary firmware installed, over another system that contains dozens of other proprietary pieces, including ones that also run on the main processor and are able to compromise the whole system (from a security point of view), on top of not respecting the user's freedom.

On Android devices, people are often facing this choice and look around for instructions on how to install only the missing firmwares so that they can avoid installing a system that contains even more proprietary bits. While these instructions cannot be released on free system's official documentation pages, for the sake of not encouraging the use of proprietary software, it makes sense for me to publish such instructions on my personal pages. I have written scripts that extract the firmwares from CyanogenMod installation zips for a few Android devices and push them to the device: cm-firmwares.git

The procedure to install the firmwares is the following:

  • Grab the CyanogenMod installation zip matching the needed version from the CyanogenMod Downloads website (make sure to select the stable release).
    For instance, the CyanogenMod 10.1.3 installation zip for the Galaxy S 3 (I9300) is at: https://download.cyanogenmod.org/?device=i9300&type=stable.
  • Clone the cm-firmwares git repository, with the branch matching the CyanogenMod version.
    For instance, for CyanogenMod 10.1.3:
    git clone git://git.paulk.fr/cm-firmwares.git -b cm-10.1.3
  • Connect the device via USB, make sure ADB is enabled and allowed to run as root on the device. Check that the device is connected and allowed:
    adb devices
    If not, it might be necessary to run the process as root (or prefix the following commands with sudo).
  • Run the script (on the host computer) with the device codename as first argument.
    For instance, for the Galaxy S 3 (I9300):
    ./cm-firmwares.sh i9300
  • Alternatively, it is possible to create a zip containing the firmwares and install it afterwards, using recovery.
    For instance, for the Galaxy S 3 (I9300):
    ./cm-firmwares.sh i9300 zip

If you wish to push only certain firmwares, feel free to edit the text file matching the device codename to remove the ones that are not needed before running the script.

Reverse engineering the Elan KTF2K touchscreen driver

Written by Paul Kocialkowski - 17 april 2013 - 3 comments

EKTF2K

I recently acquired an Allwinner A13 unbranded tablet in order to port Replicant to it: this platform is well supported by free software (the Linux kenrel and the u-boot bootloader) and there is an active community of developers working on free software for the Allwinner A1x platforms: linux-sunxi.

The tablet I ended up with contains an Elan EKTF2000 touchscreen, but I couldn't find any touchscreen driver for it in the linux-sunxi kernel tree: the source code was just not released, even though it's marked as being GPL-licensed. Moreover, since the tablet is unbranded, there was no one I could contact to request source code. So I asked around, and it turned out that nobody knew about source code that would have been released for that touchscreen. However, the tablet came with Android preinstalled and there was an ektf2k.ko module.

After some research, I finally found a driver for elan ktf2000 touchscreens written by HTC. It seemed to match mine (both use I2C) and preliminary tests revealed that the same protocol (on top of I2C) was used by my touchscreen. However, it was not quite enough to write an usable implementation for my device: as a matter of fact, the returned coordinates from my touchscreen did not match the screen size: it reported values up to 896x576 while the screen size is 800x640. So the whole issue was about figuring out these values (896 and 576) at run time in order to scale down to the actual screen size.

The preinstalled Android system came with a kernel module called ektf2k.so which is the actual driver. When loaded, I saw this message on the kernel logs:

[elan] __fw_packet_handler: x resolution: 576, y resolution: 896

Which meant that this driver had the code to get the values from the touchscreen chip.

I quickly understood how the touchscreen protocol works by reading the HTC driver, and it turned out that requests were arrays of 4 bytes, with the first one set to 0x53 (indicating a request) and the second one set to a particular command (indicating what we request). Now considering that requests are usually static tables that are defined in code (that's the way it's done in the HTC driver), declared at the beginning of the function, I knew that the static array of 4 bytes corresponding to the request for the size I needed to find out was held somewhere in the ektf2k.ko module.

Thanks to objdump, I decompiled the module (it is legal to perform such reverse engineering in Europe) and looked at the assembly code for the function __fw_packet_handler. I clearly saw the different calls to elan_ktf2k_ts_get_data and printk, but no sign of the data packets. I then looked at the .rodata section, that contains, as its name suggests, the read-only data, where the packets would likely be stored. The string “__fw_packet_handler” is stored at offset 0170 In this section. Right before, I found the following data:

 0160 53000001 53600000 53630000 53f00001  S...S`..Sc..S...

Looks very much like static arrays of data with the first byte set to 0x53! So I tried issuing requests with the commands 0x00, 0x60, 0x63 and 0xf0 and received the height with 0x60 and the width with 0x63! It was not in the most obvious format but 576 is 0x240 and 896 is 0x380, so it was easy to see that the responses were containing these values.

In the end, I completed my implementation, and I now have a fully working touchscreen (with up to 5 contact points) with the free software kernel!
The patch is: input: Elan KTF2K touchscreen driver with CTP bindings

Update: I recently found the source code that partially matches the kernel module preinstalled in my tablet: ektf2k.c

Orientation vector calculation from accelerometer and magnetometer sensors in Android

Written by Paul Kocialkowski - 13 march 2013 - no comments

While reversing the Galaxy Tab 2 sensors, I have been looking for a way to calculate the orientation vector from acceleration and magnetic field vectors: I've looked at any sensors implementation I could find and each time, this was being held into some proprietary component, to the point that the Galaxy Tab 2 has an user-space blob dedicated to this task (orientationd). Since I am not an expert at physics, I soon gave up on writing a free orientationd implementation, which was really a shame given the time I spent making the geomagnetic sensor work properly. I just realized that there was one last implementation I didn't look at, that is the free software user-space program for AKM8975. So many thanks to Asahi Kasei: I was able to reuse that code directly and it worked perfectly at first try.
That's pretty amazing!

All the glory details are in the akmdfs/AKFS_APIs_8975/AKFS_Direction.c file. To put it in a nutshell:

orientation

In the end, I think this piece of code from that free software implementation made my day!

What's up with the Android SDK?

Written by Paul Kocialkowski - 05 january 2013 - 11 comments

A couple days ago, I announced, on the behalf of the Replicant project, the release of the Replicant 4.0 SDK, motivated by some recent license change regarding the Android SDK: Google decided to put an overall non-free license for their SDK. After talking about it on IRC, FSFE member Torsten Grote decided to write an article to raise public awareness: Android SDK is now proprietary, Replicant to the rescue. In the next few hours, that news was being relayed by some online IT media and it sometimes got a bad review, calling our statement a “non-issue”. Let's check our facts and clear out the situation.

It was first brought to our attention that the SDK is being released under a non-free license only a couple days before releasing the Replicant SDK. We didn't know about it before and thus assumed that this was a recent license change. As a matter of fact, we were wrong: we have been told since then that these terms of use have been there all along the way and only a small part about fragmentation have been added with the Android 4.2 release. However, as far as I can remember from the past (and please let me know if I'm wrong), the end-user didn't have to explicitly agree to these terms before downloading the SDK. Now they are required to do so before being able to download the SDK package. That's one first thing that we find unacceptable: we believe that anyone should be able to develop applications for the Android platform without having to agree to such terms. Now let's take a closer look at what the user must actually agree to:

you may not: (a) copy (except for backup purposes), modify, adapt, redistribute, decompile, reverse engineer, disassemble, or create derivative works of the SDK or any part of the SDK; or (b) load any part of the SDK onto a mobile handset or any other hardware device except a personal computer, combine any part of the SDK with other software, or distribute any software or device incorporating a part of the SDK.

These conditions seem totally unacceptable to me and are likely to cause a reaction such as calling the Android SDK proprietary from anyone who values software freedom. However, let's not stop there and let me get back to what is right before these statements in the license text:

You may not use the SDK for any purpose not expressly permitted by this License Agreement. Except to the extent required by applicable third party licenses, 

The last sentence is the meaningful one: it means that basically, the restrictions are not applicable to software that is covered by another free software license. So that's basically how Google can avoid breaking other licenses terms. Moreover, Google is not the copyright holder for all of the software released in the SDK, so they basically have no right to apply such restrictions to it. Huh, we're safe, after all, the Android SDK still is free software. But wait, is it really? Are all the files shipped with the Android SDK proven to be free software? If that was the case, then why would Google waste some time writing down these terms if they actually do not apply to anything in the SDK? So that point gives us fair enough reasons to suspect that there is actually proprietary software in the SDK. Yet another good reason to release a free SDK such as the Replicant SDK. Now let's consider the Android SDK manager utility: it is designed, down to the source code, to check for plug-ins and updates from Google. If I recall correctly (once again, correct me if I'm wrong), there used to be a clear license statement for each components: the Google APIs were shown as non-free software in an explicit way and the emulator images were somewhat shown as containing mainly free software. Now Google changed all this, and all the components show the same EULA terms. Now how can the user make any difference between what's free and what's not in that components list? Sounds harder than it used to be, and like a problem to us. That's why the Replicant SDK won't check for new components from Google. So here are the reasons why we call the Android SDK proprietary and why we think that there is a problem with it. Even though not all of this is a sudden change, why would it be any less relevant to try and raise public awareness about the issues we've spotted?

2013-01-06 Update: I've checked the license of the individual software components shipped with the Android SDK and it turns out that all of them are covered by a free software license. What's the point of that overall proprietary license then?