Updating to 4.3 (the manual way)
Google released Android 4.3 a few weeks ago, and last week, the OTA for my Nexus 10 finally arrived. I eagerly upgraded, and stupidly forgot to backup my internal storage first. It somehow completely slipped my mind that I made several changes to /system (added kernel modules, threw in busybox, bunch of custom scripts). The update restarted the phone into recovery where it got stuck at the “updating” animation. Hoping that it was just taking very long, I went to lunch, but it was still stuck when I got back.
In the end, I installed TWRP. I had ClockworkMod, but adb did not work with CWM. Using adb shell and adb pull, I backed up my internal storage (including my precious Linux chroot) and then turned to performing a manual upgrade.
First, I got the factory image package from Google (Android 4.3 is build JWR66V) then I extract the compressed tarball which contains an img file (bootloader), a zip file (OS partition images), and some scripts. The scripts erases the entire device, and I didn’t really want to start over from scratch, if possible.
So, I erased the boot, cache, recovery, and system partition:
fastboot erase boot fastboot erase cache fastboot erase recovery fastboot erase system
(Note that I did not erase userdata, the “internal storage” partition.)
Next, I flashed the new bootloader, then proceeded to reboot the bootloader:
fastboot flash bootloader bootloader-manta-mantamd03.img fastboot reboot-bootloader
After the new bootloader started, I initiated the OS update:
fastboot -u update image-mantaray-jwr66v.zip
Note: The provided flash-all.sh script uses “-w”, which erases all user data. “-u” keeps user data.
I’ll admit, I was a bit surprised, but the update went flawlessly. I now have Android 4.3 with all my apps and data intact. I’m guessing that I was rather lucky because this update had no problem with my existing userdata, or perhaps my earlier failed attempt at the OTA update already converted what needed to be converted. Unfortunately, I don’t know the update process well enough to say for sure.
Where’s my root?
As expected after an update, root was gone. I replaced my su binary (using TWRP and adb) and I thought things would go back to the way it was. It didn’t. The old su binaries completely do not work.
Android 4.3 upped its security quite a bit. Two things affect root users: the addition of the nosuid flag to the system mount, and the removal of many capabilities from most of the processes running in the system, especially CAP_SYS_ADMIN, CAP_SETUID, and CAP_SETGID. Fortunately, Chainfire figured out the way forward with 4.3. All I needed to do was to install the new version of SuperSU. He explains what’s changed and why in these three posts.
So, with root restored, I have access to my chroot back, and I can load kernel modules again, right?
No loadable modules?
Nope. The kernel in 4.3 is compiled without loadable module support. I re-compiled the kernel from source, re-enabling loadable module support, and throwing in a few other options which tickled my fancy. (I think there’s 802.1 D and Q support, and one or two smaller drivers compiled in – I don’t really remember everything I changed.)
I found a tool (AnyKernel, I think) which was supposed to allow me to flash any kernel I want using the update.zip method. Unfortunately, that tool (or at least the version of it which I found) was meant for Android 2.3 or below, and the API has changed in ICS.
In the end, I took the original boot.img from the stock firmware image, unpacked it (using this tool – it says Xperia boot tools, but it worked for my Nexus 10 too), replaced the kernel, then repacked it. After doing that, I flashed my new boot.img using fastboot. That seemed to do the trick.
Update: I’ve added support for some 3G dongles (usb_wwan, option, and ppp_async), but I made a mistake and had to recompile, so you’ll have to reflash the boot.img. And since you had to reflash anyway, I made those three modules and cifs built-in, since they’re not large modules anyway.
For the Nexus 10 (ie. manta) there are two files:
- tan-ce_manta_xxxxxx_y.z_boot.img – basically stock boot.img with my kernel (which is for the Nexus 10, with Android 4.3, build number JWR66V or JWR66Y). Essentially stock + loadable modules and a few other options and drivers enabled.
- tan-ce_manta_xxxx_y.z_modules.zip – Collection of the following loadable kernel modules:
- dm9620.ko (This isn’t in mainline, but my USB to Ethernet dongle uses this.)
- ebt*.ko – ebtables support
- l2tp_core.ko – support for L2TP tunneling
- mcs7830.ko – USB ethernet
Several useful modules, like ntfs and cifs, are not listed there – this is because I made them built in. Even so, the kernel is only 800KB larger than pure stock.
You can download the files below:
- tan-ce_manta_JWR66Y_boot.img – Nexus 10 boot image, for build JWR66Y
- tan-ce_manta_JWR66Y_modules.zip – Nexus 10 loadable kernel modules, for build JWR66Y
NEW: Update for Android 4.4 (KitKat), KRT16S: tce-manta-KRT16S_boot.zip
See this post for more info.
Also, some have asked me to provide a Nexus 7 kernel as well. I tried to compile them, but I’m told those kernels are only good for boot-loops… If you’re using a Nexus 7, perhaps you can try this kernel. I understand it has support for 3G modems now.
One remaining TTY issue…
I haven’t quite figured this one out yet, and if I do, I’ll post about it. Since su is now being run via proxy, when using it in the Terminal Emulator, all su-ed commands are no longer connected to the pseudo-terminal directly attached to Terminal Emulator. As such, ncurses apps, and even the line discipline is sometimes messed up. I’ve been toying with the idea of establishing my own pts sessions directly to the shell, but I haven’t quite figured it out yet.
Update: For now I’m using a script to send stty commands for the screen size to the appropriate /dev/pts device. When launching a root shell, I pass it an argument which is the name of the “real” pts device in use by terminal emulator. The script, when run form withing the root shell, reads the screen size from the “real” pts device and configures it’s controlling pts to be the same. You’ll have to re-run the script every time the screen size changes, but it works as a temporary workaround. Here’s the salient parts of the script:
UPDATE (2): I’ve figured out how it works!
# CTTY is an environment variable containing the "real" user facingpts device if [ "$CTTY" = "" ]; then echo "CTTY not set" exit 1 fi # Set the rows and columns SIZE=`stty -F "$CTTY" size` ROWS=`echo "$SIZE" | cut -f 1 -d ' '` COLS=`echo "$SIZE" | cut -f 2 -d ' '` echo "Setting screen size to $ROWS x $COLS" stty rows "$ROWS" cols "$COLS"
hello! I just tried your modified boot.img but it didnt work! I just stays in a bootloop! I have the new nexus 7 2013 (flo) I just want module loading enabled to use it with my usb modem and mi arduino card… I would thank you a lot the help you can bring me! Please!
Ah, I suppose it wasn’t completely obvious, but that boot.img is meant for Nexus 10 devices! It will not work for a Nexus 7 device. You’ll have to grab the source and compile your own, or see if anyone else has compiled a kernel for your device.
Oh! I really dind read It was in bold font hehe! I have no idea how to do it for my device but I will try!! Thanks!!
To be fair, I only added the text in bold after you posted your comment.
Anyway, if you’re looking to compile a kernel, I suggest starting here: http://source.android.com/source/building-kernels.html
Note that this is really best done from a Linux PC.
The first part you need to get past is figuring out which git commit to use to build your kernel – otherwise you may end up with a version that isn’t tested with your Android build. Actually, if you look at your kernel version in settings it may already have the “short-form” of the git commit used to compile your kernel.
In the instructions, after you do a make flo_defconfig, but before you do the final make, do a make menuconfig. This will let you change the kernel configuration. Here you should enable loadable modules and set to “M” (loadable module) any other drivers you are interested in.
Hello again! tan-ce I have some questions, I really want to know which are the benefits of enabling module loading, so after that my tablet is going to be able for detecting any device? like my arduino? Or not? So when I compile the kernel, the modules that you listed above are there or I have to make something else? Like flashing those modules?
I really like your article.
I see you have made the updates for Nexus 7 (2013) but I do not see the links. Any chance I can get them? Also have you had any feedback yet?
Hi tan-ce, I’ve tried the kernel and it’s great! I’m also trying to enable usb 3G modem, since you have build environment, would you be so kind to build following 5 loadable kernel modules:
these module is required to enable 3G modem, if you can help it would be great!
Thanks a lot.
Hi, happy to hear that it’s working for you.
I’m pretty sure I enabled usbserial and cdc-acm, but I think I made those as built in modules. For the rest, I’ll take a look and see if I can compile them for you. =)
Ok, I’ve update the post, you should download the new boot.img and modules. All the modules you need are small, so I compiled them in. (ie. no need to insmod/modprobe)
Thanks a lot, tan-ce!
I’ll try it later today.
Ok, I tried the new kernel, now pppWidget work like a charm!
Thanks a lot!
i am using Nexus 7 Flo 2nd generation tab with Kernel version: 3.4.0-g6537a16 and build number JSS15Q. I want to use 3G modem, i have installed PPP widget, it says ” No Driver Found”, i have rooted my device. Please provide me the link to download the following modules so that i can use external USB modem and connect to internet.
So if I make it, me tabletas is going to detect devices Which are supported on Android 4.1 for example? I am going to make a better question: Which are the benefits ofrece enabling module loading ?
Well, which devices are supported is determined by the kernel version, as well as the modules which are enabled. I really can’t give you a full list of all the stuff that’s supported… There’s too many to go and find to list.
But for your question about the benefits of module loading, well, in Linux drivers are kernel modules. So being able to load modules means you’ll be able to add 3rd party drivers (ie. those not already part of the mainline Linux kernel).
Referring to your question a little higher up in this thread, those modules list above are drivers which can be loaded in to this kernel. You’ll have to copy the files on to your tablet, then use a root shell to load them using insmod.
As for your Arduino, it depends, I’m not familiar with Arduinos, but I suspect you should be able to get them working with your tablet.
Oh I’ve tried but I am so so newbbie at this, Im trying another tutorial because I followed the steps in this link: http://source.android.com/source/building-kernels.html but no success, so Im gonna follow this tutorial: http://www.youtube.com/watch?v=IV4jhXWc7AE , so I dont know if there could be any trouble with the kernel version I downloaded from here: https://android.googlesource.com/kernel/msm.git/+/android-msm-flo-3.4-jb-mr2 , I think you are a busy person but I’d really apreciate if you help me, so It could be awesome if you upload de modified boot.img for The 2nd generation nexus 7 (flo)
No promises, but if I have free time I’ll try to build the kernel for you. But two issues:
1. I heard there was a new release of 4.3 for the 2013 Nexus 7 to bugs related to multitouch and GPS. You should apply that update first if you haven’t already.
2. I don’t have a 2013 Nexus 7, so I won’t be able to test the boot.img, but if you could tell me what “Kernel version” and “Build number” says under “About phone”, that would help ensure I build the right kernel. Be sure to update first, as I mentioned in (1)!
Yes I already updated.
-Kernel version: 3.4.0-g365a6e0
-Buil number: JSS15J
I’ve added versions for your Nexus 7! The post above is updated with the links.
Apparently JSS15J is no longer the latest build. Google’s currently at JSS15Q. However, if it’s anything like the recent update to my Nexus 10, it probably was a minor update and didn’t affect the kernel. (Hopefully.)
Any chance of a kernel build with JSS15Q? They made major GPS and touch changes. and I suspect some were at kernel level. Also EDS support?
I’m somewhat certain that the kernel I built came from JSS15Q… but to be sure, could you give me the name of the stock kernel after upgrading to JSS15Q? Then I’ll be able to see whether I compiled from the correct source code commit.
Oops… looks like I took the JSS15J kernel and married it with the JSS15Q initrd earlier…. I’ve fixed this mistake now. You can download the JSS15Q kernel in my post above. Let me know if it works…
It did not boot, just hung on the GOOGLE screen,
Hmm. Ok, give this one a shot… if it doesn’t work, I’m not sure I can help further…
Hello, I’m trying to compile a kernel for the nexus 4 with enabled modules. But when I enable modules with “CONFIG_MODULES=y” compilation fails with this error.
[CODE]arch/arm/mach-msm/lge/mako/board-mako-display.c:900:20: error: ‘lm3530_lcd_backlight_pwm_disable’ undeclared here (not in a function)
arch/arm/mach-msm/lge/mako/board-mako-display.c:902:18: error: ‘lm3530_lcd_backlight_on_status’ undeclared here (not in a function)
make: *** [arch/arm/mach-msm/lge/mako/board-mako-display.o] Error 1
make: *** [arch/arm/mach-msm/lge/mako] Error 2
make: *** [arch/arm/mach-msm/lge] Error 2
make: *** [arch/arm/mach-msm] Error 2
This functions have EXPORT_SYMBOL in “/drivers/video/backlight/lm3530_bl.c b/drivers/video/backlight/lm3530_bl.c”
Do you know what I need to do?
Without enabled modules compile fine.
Thanks for the help.
I’m not sure I can help you with that — I’m not sure why the compile would fail just because modules were enabled.
Did you ever figure out where to fix this issue? I’ve come across the same error when adding the “CONFIG_MODULES=y” to Nexus 4 kernel. I used checkout f43c3d9 for the commit to match up with the factory images for 4.3. From what I can tell, when we enable this option, somehow it causes CONFIG_BACKLIGHT_LM3530 to not get #defined as it normally would.
I didn’t… I don’t own a Nexus 4, so admittedly I wasn’t really motivated to figure this out… but figuring this out probably wouldn’t be a walk in the park, based on what you described… You might need to actually look at source code to see why that happens…
Hi, I use CM10.2 nightly and in this ROM kernel doesn’t have modules enabled.
Can I use your kernel for CM10.2 ->Nexus 10<- manta (it is based on JSS15Q build)?
Could you make recovery install zip file instead of *.img file? It will be easier for a lot of people 🙂
No, this kernel is based off the JWR66V or JWR66Y builds (I believe they use the same kernel). When I have some time I can try to package it into an update zip… I need to figure out how to get that right.
HI tan-ce, i am using Nexus 7 Flo 2nd generation tab with Kernel version: 3.4.0-g6537a16 and build number JSS15Q. I want to use 3G modem, i have installed PPP widget, it says ” No Driver Found”, i have rooted my device. Please provide me the link to download the following modules so that i can use external USB modem and connect to internet.
can someone email me the below zip files to flash
tan-ce_flo_JSS15Q_1.0_boot.img – 2013 Nexus 7 boot image, for build JSS15Q
tan-ce_flo_JSS15Q_1.0_modules.zip – 2013 Nexus 7 loadable kernel modules, for build JSS15Q. this should also contain the below files
for Nexus 7 Flo 2nd generation tab with Kernel version: 3.4.0-g6537a16 and build number JSS15Q to connect to usb modem. my email id is email@example.com
I removed the link because I wasn’t sure if it was working.
You could give it a shot and let me know if it works, but be prepared to reflash your boot partition if it doesn’t work. I don’t have a Nexus 7, so I can’t test it myself.
Thanks Tan-ce, can you confirm if all the files are available for 3g USB dongle usage and more over how to flash the img file. How falsh img extension files? i am not sure, i know to flash only zip files.
I don’t think I’ve uploaded the modules yet, but they’re worthless unless we know the kernel doesn’t actually just cause your device to boot-loop…
Regarding flashing… yeah, I’ve been meaning to get round to making an update.zip… we’ll see if I have some time today or tomorrow. Otherwise, if you want to try first, there’s a guide here for unlocking the Nexus 7: http://forum.xda-developers.com/showthread.php?t=2379618
After which you would have fastboot. Then doing a
fastboot flash boot tan-ce_flo_JSS15Q_1.1_boot.img
should do it.
If you try it, let me know if it works. If it doesn’t, you can restore the factory default boot partition by downloading the stock firmware from here, extracting boot.img and flashing that using fastboot.
I tested it myself.
It does not Boot on nexus 7 2013 with jss15r installed.
Hmmmm … We need an USB 3g kernel for the flo. 😉
Drat. That’s disappointing. I can’t think of any reason why it’s boot-looping… I used the same steps, but with less aggressive changes to the kernel config.
I’m sorry, but without even having the device, it’s kinda hard to debug anything without the hardware… your best bet is probably to look in xda-developers and try to find a kernel there.
If that doesn’t work out, and you’re willing to try several test runs with me, we could go back and forth with me sending you different kernels with different settings hoping to find one that works… if you want to try that, drop me a comment here with your real email address and I’ll contact you and we’ll take this over email.
I have used ElementalX 1.4 kernel for the flo device.
This one has loadable kernel modules since Version 1.3.
Now, we need the kernel modules.
Yes, its hard to work on a device without owning it.
Thanks for all your help.
I have used elementalx kernel 1.4 from xda developers.
This kernel has loadable kernel modules since Version 1.3
Now, we need the modules for 3g usb 😉
Thanks a lot to tan-ce.
It doesn’t work that way… I’ll need the post-compiled source tree for the exact kernel you’re using. It’s not like a DLL or .so, I could give you my kernel modules, but your kernel will reject them because they weren’t compiled for the same kernel.
Have you tried asking elementalx?
I will ASK them.
THX a lot for your hell.
I’m ready to test the kernel on the nexus 7
My mail – [removed]
My name is Dmitry, I speak Russian, I use the Google translator
I think we understand each other.
Can I get the nexus 7 modules and boot links please?
Sorry, looks like my kernel, at least for now, only causes boot-loops. (See my comment above in reply to followmsi)
Did it work ?
Were you able to boot the flo image ???
JSS15R was released some time ago.
Thanks for all your effort tan-ce !!!
You even dont own the device ,)
Why visitors still make use of to read news papers when in this technological world everything is accessible on net?
Your article is very informative. It broadened my view significantly. However the device I am having is Nexus 4, I was wondering whether you can shed some light as to how I am going to make things work on my Nexus 4.
Basically I need the following:
1. cifs module. I want to watch video on my LAN share with subtitles. Dice Player worked for me on 4.2. But it won’t load subtitles on 4.3. That’s why I believe I need cifs module built in. As far as I understand, enabling loadable modules isn’t just enough, I need to find a suitable cifs module file, is that correct? Where do I find it? And is there a tutorial like the kernel building one that teaches how to add modules to the kernel image?
2. Cisco LEAP wifi auth method support. Google dropped this after 4.1.2. I really do not understand. From what I have already read on the internet, this seems to be something more complex than adding cifs module. Can you give me a lead that I can follow?
3. Is there a way to flash 4.1.2 to Nexus 4? I read an article that says it can’t be done. I just wonder because it will fix both problems I mentioned above. I don’t mind running a 4.1.2 version on my Nexus 4. Neither do I mind using custom ROMs. Do you think there are ROMs built from 4.1.2 already out there? Or is there a way to build my own? The biggest problem would be with drivers I suspect.
Thanks. For (1): The Linux CIFS driver is an in-tree driver. So, when you compile the kernel, in the same interface where you enable loadable modules (“make menuconfig”, or if you’re more hardcore, modifying kconfig by hand), there’s options on several modules to either have them disabled (“[ ]”), compiled in (“[*]”), or compiled as a loadable module ([“M”]). Find the CIFS module and select it to be compiled as a loadable module. When you compile the kernel, the modules will also be compiled and left inside. I usually just do a “find -name ‘*.ko'” to find them all.
For (2), unfortunately I know nothing about Cisco LEAP. But a quick Google search found a bug report on AOSP: https://code.google.com/p/android/issues/detail?id=58279. Have you taken a look at that to see if there are any workarounds? Or perhaps a third party app?
(3) The Nexus 4 was released at the same time Android 4.2 was release, which tells me that it’s likely no version of Android prior to 4.2 were ever built for the Nexus 4. I guess your best bet would be to try a 3rd party ROM to see if they support LEAP. Have you taken a look at CyanogenMod?
I’m ready to test the kernel on the nexus 7
My mail – [removed]
My name is Dmitry, I speak Russian, I use the Google translator
I think we understand each othe
Any bootloader with drivers for android 4.4 kit kat for n10?
Am working on it. I need to upgrade my tablet first. Expect it within this next few days.
FYI, if you’re still watching this thread, I’ve released my KitKat kernel: http://blog.tan-ce.com/android-4-4-kitkat-kernel-for-nexus-10/
Have you such a collection of modules compiled for 4.2.2 (JDQ39)? Please help to find compatible module dm9601 or qf9700! I searched a lot and tried, but all modules gave error during insmod. Have intension to compile modules by myself – but I’m a newby and need to find all the sources and virtual machine where to compile too, and step-by-step instructions too. Thanks in advance.
You couldn’t load the modules because the modules has to be made for the specific build of your kernel. So, your best bet would probably be to flash a kernel then try to load the modules for that kernel.
Unfortunately, I don’t have a collection of modules for 4.2. However, you could try to search to see if anyone made modules for your kernel. Are you running the stock kernel?
I was looking for that long time – and haven’t found any. I have stock kernel – 3.4.5-gaf9c307. I’ve found only cif-modules for this kernel, and they work properly.
How to find out the name of the wifi module used in a specific Android phone? For example in Samsung GT-S6312.
Thanks in advance.
Good afternoon. I am very interested in your Manta 3.4 kernel and I have tried for many hours now to flash the boot.img. However I have no access to a PC for several months at least so I have tried to find an Android way. So far I have failed these ways:
1) flash_image binary, installed into /data/bin/ with 755 permisions, declares that the ‘boot’ partition does not exist (command is ‘flash_image boot /path/to/boot.img’) or returns the enigmatic error code 1 depending on which of the two binary versions I tried. Both were linked from the cyanogenmod wiki page:
2) AnyKernel can supposedly convert a boot.img to a flashable zip from within Android, but I got a segmentation fault when I ran the ‘./unpackbootimg boot.img’, which is practically step one:
That happened on two different kernels, one of which is stock. Rom is stock rooted/busyboxed 3.4 JWR66V.
I don’t want to upgrade to 4.4 yet, I’d give up both Adobe Flash and openFileChooser and I’m just not ready to do that. This thing IS my computer for a while. Would you be so kind as to either package the .img into a flashable or help me troubleshoot? Thanks.
Here, give this one a try: http://blog.tan-ce.com/files/4.3-kernel/tce-manta-JWR66Y_boot.zip
Let me know if it does or doesn’t work.
I successfully flashed your new zip as well. Saved me the trouble of symlinking the modules. Thank you!
1) I realize now that my afternoon is not your afternoon as I’m in the Pacific NW. Silly me.
2) I appear to have successfully flashed your boot.img using the DD command. I rebooted and About Tablet shows your 3.4 kernel. My process, for the curious, was as follows:
a) Find and open fstab.manta at the root filesystem “/”.
b) Look for a line containing boot. Mine read as follows: “/dev/block/platform/dw_mmc.0/by-name/boot”. I guessed (!) this to be where the kernel lives.
c) Open a terminal emulator and obtain root (typing “su” if using Android Terminal Emulator).
d) Type “DD if=/path/to/new/boot.img of=/dev/block/platform/dw_mmc.0/by-name/boot” and hit enter.
e) Reboot. Check Settings –> “About Tablet” to see if you are running the kernel you hoped for.
Please note that making a nandroid /before/ all this is an excellent practise.
Now I just have to figure out how to install your modules.
I’m Vincenzo from Italy. I nave a nexus 7 2012 whit stok 4.4
There are a kernel for 3g Dongle USB?
I don’t do Nexus 7 (either version) kernels. Some of the commentators here have mentioned that the ElementalX kernel does have the necessary 3G drivers. You could give that a try.
It’s hard to find your articles in google. I found it
on 14 spot, you should build quality backlinks , it will help you to increase traffic.
I know how to help you, just search in google – k2 seo tricks