Paul Kocialkowski's coding blog

During the past few months, I've been working on bringing Replicant 4.2 support to the Goldelico GTA04, a montherboard replacement for the Openmoko FreeRunner (GTA02) that is manufactured by Golden Delicious with the intent of being free software friendly. The board design is also released under a free licence. There is an active community of developers and users dedicated to the GTA04 and other similar projects, brougt together under the hood of OpenPhoenux. Other similar projects include the upcoming Neo900, a motherboard replacement for the Nokia N900 using similar hardware as the GTA04 and also aiming to be free software friendly, with a particular emphasis on security and privacy features.

I recently published a blog post covering the status of the Replicant 4.2 port to the Goldelico GTA04 on the Replicant blog, with a nice video highlighting which features already work well. However, in the present post, I would like to cover some of the technical aspects of the port, especially regarding the use of a recent mainline Linux kernel with Android 4.2.

Currently, Android devices ship with Linux 3.4, after a long time of using version 3.0, which started with Android 4.0 Ice Cream Sandwich. To this day, the reference common Android kernel repos have branches with Linux 3.10 and experimental work is done on Linux 3.14. That's pretty close to the Linux 3.12 version we're using on the GTA04!

Opportunistic suspend

The first challenge to overcome was dealing with suspend and resume. Android uses a particular mechanism to implement opportunistic suspend, using wakelocks: both the kernel and userspace can register these locks to prevent the device from going to suspend. In Android kernels, there is another mechanism that allows some non-critical chips to reach suspend state before the rest of the system: this is earlysuspend. As the whole Android system is built around the concept of wakelocks to handle power management, something similar needed to be implemented in the mainline Linux kernel. After efforts from the Android kernel maintainers, wakelocks were implemented in a clean fashion some time ago. In order to implement opportunistic suspend, a separate interface was implemented on mainline Linux, known as autosleep, which uses different nodes than /sys/power/state (that the Android kernels use directly for opportunistic suspend). Starting in Android 4.1, a library was added to Android in order to detect and handle these different modes.

So thankfully, everything was already in place to use autosleep properly on Android 4.2. Except that it didn't work. This turned out to be because of a dedicated capability that was missing from the Android userspace: CAP_BLOCK_SUSPEND. It turned out to be easier to just revert the patch restricting access to wakelocks to users with that capability in the kernel.

Android USB Gadget and Android Debug Bridge

Android also went their own way in implementing USB device drivers for the various things that are used by Android: mass storage, rndis for USB networking, MTP for file access and a couple others, including ADB. The Android Debug Bridge (ADB) is a great way of debugging what's going on with an Android device, providing easy access to the logs, to a shell and file transfer (these are the features of ADB I use the most). All of that is not integrated at all in the mainline Linux kernel, so there was some substantial work to do here. The first thing to do was importing the related commits from the android-3.10 branch of the common Android Linux kernel. All of that built nicely with only minor code corrections, to follow some API changes in 3.12 and some features like rndis or MTP worked right away, but the most interesting part was left not working: that's the Android Debug Bridge. A few kernel versions back, there used to be a dedicated composite function driver for ADB, but a commit by one of Android's kernel maintainers totally gets rid of it, calling it obsolete with no further information. While attempts to restore it failed, I tried to find out in details why it is obsolete and if that meant the final death of ADB, that I found rather convenient. Thankfully, someone found out what happened and wrote about it: the dedicated ADB driver was being replaced by another one using FunctionFS, a more flexible and generic way of implementing such drivers, directly from userspace. It turns out that FunctionFS support for the userspace ADB server was merged nearly entirely in Android 4.2. I had to backport a missing fix to have it fully working properly and also had to import adaptation a patch in recovery to have sideload working with FunctionFS too. A few bits were also needed in the initialization procedure to have things set up right. Once all of that was done, it could finally run flawlessly!

Headset/headphones detection with SoC Jack

So Jack's a good guy. Everyone knows about that. Whether he's saving the homeland from yet another threat or just letting us know something of interest just got plugged in one hole or another, it always feels great to hear from Jack. However, the Android kernel guys didn't seem to appreciate his participation in the show as much as we all do, or at least until recently.

The traditional way of reporting a headset or headphones plug/unplug in Android kernels was using a switch called h2w, reporting these events to userspace. The mainline kernel prefers another approach, using our beloved Jack SoC architecture. It also provides a convenient way of reporting button pushes, which is quite nice. So instead of rewriting it all using the h2w switch, it struck me that there is a frameworks config option to politely ask Android to give Jack some consideration. And when it does that, everything works great, including button press reports!

As I am porting Replicant, our fully free Android derivate, to the new Goldelico GTA04, I had to deal with ALSA user-space integration in Gingerbread. So let's take a quick look about it:
On Gingebread (and previous versions), user-space audio is done via the libaudio library. The Android framework will basically interact with AudioFlinger that is the component in charge of loading that libaudio library and dealing with it (that's frameworks/base/services/audioflinger/AudioFlinger.cpp).
So libaudio is basically the place where PCM read/write and mixer stuff happens. As we started looking into the various existing ALSA libaudio, my fellow Replicant developer GNUtoo told me about TinyHAL, a clean and flexible audio module that does ALSA and routing from XML configuration files. Too bad, TinyHAL was designed for Ice Cream Sandwich, and the Audio API changed in ICS (basically, it is now a module, like the ones for gps, sensors, lights, etc). So I couldn't use it as-is, but there were various concepts I hoped I would be able to reuse, like the XML routing config or the use of the TinyALSA lib, that is very clean, simple to use and handles ioctl-level ALSA stuff.

In the end, I decided to write my own libaudio, using both TinyALSA and XML config files, that I called TinyALSA-Audio. Audio output and Mixer was quite straightforward and worked quickly. AudioFlinger basically opens the AudioStreamOut at 44100Hz, 2 chans, S16_LE format (signed 16bits, little endian), which works fine with the audio hardware, a TWL4030 Codec here, that didn't complain at all.

Next step was about audio input and recording. At first sight, I thought it would be as easy as audio output, that is just setting the controls, opening the device via TinyALSA, sending the config following what AudioFlinger asks and just read the data. So I basically wrote code that was that simple, but it failed while setting the config: cannot set hw params: Invalid argument was the error. As I didn't see what could possibly be wrong about the params, I decided to take a look at how tinycap, the capture utility that comes with TinyALSA handles things. It actually sets the params at 44100Hz, 2 chans, S16_LE, and recording works then. When I tried to force the params to what AudioFlinger asks, that is 8000Hz, 1 chan, S16_LE, I got the very same error as on my libaudio: cannot set hw params: Invalid argument.

So what was it all about? Does the CODEC only records at 44100Hz, 2 chans? I tried with zygote stopped, and then 8000Hz mono worked. I also checked in the kernel code: the TWL4030 CODEC is supposed to work at 8000Hz mono as well. So I deducted that when the output device is opened, the input device will only work at the same config (rate, chans). What a bummer! AudioFlinger asks for 8000Hz mono, not 44100hz stereo, though that's all I can get when output is opened (which is always the case when zygote is running).

Thanks to the AOSP, there are various others libaudio that I could use to learn things. First thing to know was if that "issue" was specific to TWL4030 or common to all ALSA CODECS. So I did the same test (with tinycap) on the Nexus S and Galaxy S, and the result was that both couldn't record at any other rate/chans that the ones set when opening output. Galaxy Nexus audio module confirmed that too. So what is the solution here? Obviously, it consists in finding a way to return 8000Hz mono data to AudioFlinger while reading 44100Hz stereo data: that's resampling.

Galaxy S and Nexus S libaudio all handled resampling by internal algorithms, which seemed a pain to use and adapt on my libaudio. The solution finally came from the Galaxy Nexus audio module. And I really want to send a big Thank-You to the people who wrote it. First of all, it uses the very same TinyALSA as my libaudio uses. Second thing is that it doesn't embed complicated algorithms for resampling but uses the new Android 4 framework for resampling, that wasn't so hard to understand. behind that engine is the libspeexresampler lib, that is part of speex code. So all I had to do was to backport that Android 4 resampling code, enable to build of libspeexresampler in gingerbread and make use of all that in my lib.

Though, wait a second, when reading AudioFlinger code, it all seemed to indicate that AudioFlinger embeds its own resampling engine, so that when the lbiaudio reports different parameters, it handles resampling to what it wants (mostly 8000hz mono). So I tried to make use of that, in vain. I couldn't figure out why it didn't work, it just didn't. I read the AudioFlinger code several times, made sure the resampler was enabled and all, but in the end, the produced audio was just garbage, so I just gave up on using AudioFlinfer's resampler. After all, if all the AOSP libaudio do not make use of it, there might be a reason. So the thing appears to be totally messed-up and nobody cares enough to fix it but rather implement resampling in the libaudio itself. Not a very good thing for me, I would have preferred to leave resampling to the upper layer, but anyway, I had Galaxy Nexus code that could be adapted to my lib. So that's basically what I did: backporting the Android 4 resampler code to my libaudio, making use of it and using ICS external/speex repo. In the end of this misadventure and with some fine tuning, it all worked.

That's pretty much the end of the story, now I have my TinyALSA-Audio lib that handles input/output routing via mixer audio output and audio input as well, to various rates. Here are some links:

• TinyALSA-Audio source code in my personal git repo: tinyalsa-audio.git
• TinyALSA-Audio source code in Replicant repo: hardware_tinyalsa-audio
• An exemple configuration file (for GTA04): tinyalsa-audio.xml