UPDATE (13 Jan, 2014): koush has merged my pull request into Superuser, so his su app should have better terminal handling now.

As I mentioned in the last post, I’ve had some annoyances with the root shell since the upgrade to Android 4.3. To recap, from Android 4.3, the traditional way of using su on Android no longer works because Android’s zygote process drops many capabilities, and the entire /system partition is mounted nosuid. SuperSU solves this by proxying requests to a daemon who is not a descendant of zygote, but this messes up terminal window sizing. Additionally, I’ve since switched from SuperSU to koush’s Superuser, which actually makes the root shell even worse, as it doesn’t seem to handle the terminal properly, leading root applications to believe they are connected to a pipe rather than a terminal.

I did several experiments and spent some time reading SSH’s source code. Then I clobbered together several utilities and managed to produce the exact result I wanted: A shell which behaves properly, terminal-wise.

From there I made a daemon which does nearly the same thing as SuperSU’s and Superuser’s su daemon, as well as a complimentary client for requesting applications to be run as root by the daemon. The daemon (pts-daemon) is launched at boot time via init, and is therefore already running as root with all Linux capabilities intact.

When you run the client (pts-shell),

• pts-shell establishes a unix socket connection to pts-daemon.
• pts-shell asks the user for the password and sends it along to the pts-daemon via the unix socket.
• pts-daemon responds with an “OK” or “Authorization Failed”. If all is OK, we continue.
• pts-client opens a pseudo-terminal device and tells pts-daemon the name of the newly created pts device, along with a path and command line arguments to an application to run (for example, /system/bin/sh).
• pts-daemon forks a child process. This child process will then become a process group leader (setsid), open the pseudo-terminal device pts-client created, make that pseudo-terminal device its stdin/stdout/stderr, then finally exec the application pts-client wanted to be run.
• Meanwhile, pts-client changes the termios settings on its stdin, doing things like putting it in raw unbuffered mode and disabling echo. Then it starts pumping bytes between its own stdin/stdout and the pseudo-terminal. In addition, it also traps SIGWINCH, sending TIOCSWINSZ to the pseudo-terminal with the new row and column count whenever it receives that signal.

pts-client is able to receive SIGWINCH because it’s attached directly to the terminal emulator. To clarify, this means that pts-client should not be run from inside a root shell, as this would completely negate the whole reason for using it!

All in all I’m quite happy with how this has worked out. You can see the code on GitHub. If you want to try it out yourself, this is how you can install it:

• Download the update zip and load it using ClockworkMod or TWRP.
• In a root shell, run pts-passwd to set the password for the first time. (You can run this again to change the password.)
• From a regular (non-root) shell, run pts-shell <path to application> any time you want a root shell. (eg. “pts-shell /system/bin/sh”)

I’ve only tested it on a Nexus 10 with koush’s Superuser app, although I’m pretty sure it should work with SuperSU’s as well. Do let me know how it works out for you, if you try it.

Updating to 4.3 (the manual way)

Google released Android 4.3 a few weeks ago, and last week, the OTA for my Nexus 10 finally arrived. I eagerly upgraded, and stupidly forgot to backup my internal storage first. It somehow completely slipped my mind that I made several changes to /system (added kernel modules, threw in busybox, bunch of custom scripts). The update restarted the phone into recovery where it got stuck at the “updating” animation. Hoping that it was just taking very long, I went to lunch, but it was still stuck when I got back.

In the end, I installed TWRP. I had ClockworkMod, but adb did not work with CWM. Using adb shell and adb pull, I backed up my internal storage (including my precious Linux chroot) and then turned to performing a manual upgrade.

First, I got the factory image package from Google (Android 4.3 is build JWR66V) then I extract the compressed tarball which contains an img file (bootloader), a zip file (OS partition images), and some scripts. The scripts erases the entire device, and I didn’t really want to start over from scratch, if possible.

So, I erased the boot, cache, recovery, and system partition:

fastboot erase boot
fastboot erase cache
fastboot erase recovery
fastboot erase system

(Note that I did not erase userdata, the “internal storage” partition.)

Next, I flashed the new bootloader, then proceeded to reboot the bootloader:

fastboot flash bootloader bootloader-manta-mantamd03.img
fastboot reboot-bootloader

After the new bootloader started, I initiated the OS update:

fastboot -u update image-mantaray-jwr66v.zip

Note: The provided flash-all.sh script uses “-w”, which erases all user data. “-u” keeps user data.

I’ll admit, I was a bit surprised, but the update went flawlessly. I now have Android 4.3 with all my apps and data intact. I’m guessing that I was rather lucky because this update had no problem with my existing userdata, or perhaps my earlier failed attempt at the OTA update already converted what needed to be converted. Unfortunately, I don’t know the update process well enough to say for sure.

Where’s my root?

As expected after an update, root was gone. I replaced my su binary (using TWRP and adb) and I thought things would go back to the way it was. It didn’t. The old su binaries completely do not work.

Android 4.3 upped its security quite a bit. Two things affect root users: the addition of the nosuid flag to the system mount, and the removal of many capabilities from most of the processes running in the system, especially CAP_SYS_ADMIN, CAP_SETUID, and CAP_SETGID. Fortunately, Chainfire figured out the way forward with 4.3. All I needed to do was to install the new version of SuperSU. He explains what’s changed and why in these three posts.

So, with root restored, I have access to my chroot back, and I can load kernel modules again, right?

No loadable modules?

Nope. The kernel in 4.3 is compiled without loadable module support. I re-compiled the kernel from source, re-enabling loadable module support, and throwing in a few other options which tickled my fancy. (I think there’s 802.1 D and Q support, and one or two smaller drivers compiled in – I don’t really remember everything I changed.)

I found a tool (AnyKernel, I think) which was supposed to allow me to flash any kernel I want using the update.zip method. Unfortunately, that tool (or at least the version of it which I found) was meant for Android 2.3 or below, and the API has changed in ICS.

In the end, I took the original boot.img from the stock firmware image, unpacked it (using this tool – it says Xperia boot tools, but it worked for my Nexus 10 too), replaced the kernel, then repacked it. After doing that, I flashed my new boot.img using fastboot. That seemed to do the trick.

Update: I’ve added support for some 3G dongles (usb_wwan, option, and ppp_async), but I made a mistake and had to recompile, so you’ll have to reflash the boot.img. And since you had to reflash anyway, I made those three modules and cifs built-in, since they’re not large modules anyway.

For the Nexus 10 (ie. manta) there are two files:

• tan-ce_manta_xxxxxx_y.z_boot.img – basically stock boot.img with my kernel (which is for the Nexus 10, with Android 4.3, build number JWR66V or JWR66Y). Essentially stock + loadable modules and a few other options and drivers enabled.
• tan-ce_manta_xxxx_y.z_modules.zip – Collection of the following loadable kernel modules:
  • auth_rpcgss.ko
  • cryptoloop.ko
  • dm9601.ko
  • dm9620.ko (This isn’t in mainline, but my USB to Ethernet dongle uses this.)
  • ebt*.ko – ebtables support
  • exportfs.ko
  • gl620a.ko
  • l2tp_core.ko – support for L2TP tunneling
  • lockd.ko
  • mcs7830.ko – USB ethernet
  • nbd.ko
  • nfs_acl.ko
  • nfsd.ko
  • nfs.ko
  • plusb.ko
  • rndis_host.ko
  • scsi_wait_scan.ko
  • smsc75xx.ko
  • smsc95xx.ko
  • sunrpc.ko

Several useful modules, like ntfs and cifs, are not listed there – this is because I made them built in. Even so, the kernel is only 800KB larger than pure stock.

You can download the files below:

• tan-ce_manta_JWR66Y_boot.img – Nexus 10 boot image, for build JWR66Y
• tan-ce_manta_JWR66Y_modules.zip – Nexus 10 loadable kernel modules, for build JWR66Y

NEW: Update for Android 4.4 (KitKat), KRT16S: tce-manta-KRT16S_boot.zip
See this post for more info.

Also, some have asked me to provide a Nexus 7 kernel as well. I tried to compile them, but I’m told those kernels are only good for boot-loops… If you’re using a Nexus 7, perhaps you can try this kernel. I understand it has support for 3G modems now.

One remaining TTY issue…

I haven’t quite figured this one out yet, and if I do, I’ll post about it. Since su is now being run via proxy, when using it in the Terminal Emulator, all su-ed commands are no longer connected to the pseudo-terminal directly attached to Terminal Emulator.  As such, ncurses apps, and even the line discipline is sometimes messed up. I’ve been toying with the idea of establishing my own pts sessions directly to the shell, but I haven’t quite figured it out yet.

Update: For now I’m using a script to send stty commands for the screen size to the appropriate /dev/pts device. When launching a root shell, I pass it an argument which is the name of the “real” pts device in use by terminal emulator. The script, when run form withing the root shell, reads the screen size from the “real” pts device and configures it’s controlling pts to be the same. You’ll have to re-run the script every time the screen size changes, but it works as a temporary workaround. Here’s the salient parts of the script:

UPDATE (2): I’ve figured out how it works!

# CTTY is an environment variable containing the "real" user facingpts device

if [ "$CTTY" = "" ]; then
    echo "CTTY not set"
    exit 1
fi

# Set the rows and columns
SIZE=`stty -F "$CTTY" size`
ROWS=`echo "$SIZE" | cut -f 1 -d ' '`
COLS=`echo "$SIZE" | cut -f 2 -d ' '`
echo "Setting screen size to $ROWS x $COLS"
stty rows "$ROWS" cols "$COLS"