Technically minded Weblog

A quick update on my AC810 adventures.

I’ve decided against zRAM – that would require a kernel rebuild and this device has no recovery mode.
Even though “free” showed only a few MB free (max I saw was like 20 MB, usually it shows 5-6 MB), I’ve never ran out of memory when running any software on the device.

Installed some additional software on the device, all built from source, including dependencies:
TUN driver, bash shell, latest hostapd, latest OpenSSL, nano editor and OpenVPN.
I didn’t overwrite any stock binaries just to be safe and instead used a simple wrapper script (which uses LD_LIBRARY_PATH) to run newly installed software.

An interesting idea came to mind when building OpenVPN – what if I could just take one of the WiFi APs (this device has two, one for 2.4 GHz and one for 5 GHz band) and bridge it to a TAP interface instead of doing routing to a tunnel ? Devices connecting to the bridged WiFi would then connect directly to my home network instead of being routed through a tunnel.
A true home WiFi away from home experience without any setup required !
Since OpenVPN is CPU-bound, I’m not expecting to get full speed, but around 40 mbps should be achievable with good 4G reception. More than enough for my needs (remote work and game streaming).

At first, I wanted to preserve dual-band WiFi and create a third SSID (WiFi driver claims to supports this) for the VPN bridge, but Qualcomm’s binary driver has a bug which causes WPA authentication responses from the client device to always be sent to the primary WiFi SSID instead of the secondary SSID, which causes authentication failures. Third SSID only works as Open WiFi which won’t do for me.
I have explored a possibility of using ath10k open source driver from Driver Backports instead of the binary blob (this device seems to use Atheros QCA6174 for WiFi) but again, I cannot install it without rebuilding the kernel due to cfg80211.ko being built into kernel which conflicts with Driver Backports.

So in the end, I had to dedicate one of the existing WiFi APs for VPN bridge. I’ve decided to use the 5 GHz AP for VPN bridge for now and leave 2.4 GHz AP for normal WiFi use but I’m not sure whether I will leave it that way.

UPDATE: OpenVPN was doing 24mbps max with CPU maxed out and a lot of packet loss. Switching from OpenVPN to SoftEther increased speeds to over 40mbps (and zero packet loss). Maybe it can do even more, will need to test in a different location with better 4G reception.
SoftEther was pain in the but to cross-compile (required manual editing of Makefile) and get working though and I also needed to use preloadable_libiconv hack to get past the iconv() stuff as libc on the modem did not have gconv enabled.

I’ve recently bought myself a Netgear AC810 mobile hostpot. It is a rather old model but it’s lightyears better than those cheap Huawei soapbar-style modems which our ISPs provide. Also, since 4G is not my main connection and is only used on-the-go, I didn’t want to invest in a higher-end model, like Nighthawk M1 or M2.

Now, since I’m a computer geek, I wanted to take a look what’s under the hood of that small black box with an LCD screen on it.
I know that this thing, like most 4G routers out there runs some version of Linux. Now how do I access the shell ?
After some research, I found that there’s an AT terminal running on port 5510. Since this modem uses a Qualcomm chipset, it is probably based on the same software as those Sierra Wireless modems. Fortunately, the unit I have had all advanced AT commands (which are usually locked by a password, which in case of my model should be “whistler”) unlocked out the box.
But AT!CUSTOM=? returned an error. So there is some lock present. Fortunately, a keygen is available to enable custom commands on Qualcomm/Sierra Wireless modems.
https://github.com/bkerler/SierraWirelessGen (use model MDM9x40)

With this keygen, I was able to unlock AT!CUSTOM commands (commands will again be locked after a reboot). On my particular unit, AT!CUSTOM=”ADBENABLE”,1 succeeded but did not actually enable ADB. But AT!CUSTOM=”ENABLETELNET”,1 did enable Telnet. I was able to get a password-less root shell on port 23. Bingo !

Now, for the specs:
SoC: Qualcomm MDM9640 (aka. Snapdragon X12), single core ARM Cortex-A7, 1.19 GHz
RAM: 160 MB
Storage: 512 MB, divided into multiple partitions, usable space is formatted as UBIFS. There’s a total of about 126 MB of usable free space available. Root is writable as are most other locations.
Kernel: Linux 3.10.49

Router’s interface is purely web-based. Even the LCD interface is done using a tiny DirectFB based web browser (WebKit based, judging from the logs) which simply loads a web page (http://127.0.0.1/lcd/index.html) during boot. This browser seems to have some kind of custom API accessible via JavaScript. I have not yet fully reverse-engineered it but I do plan to do so.

For compiling custom software, Linaro v13.08 ARM toolchain should work, it has the same version of libc (eglibc 2.17) as that used on the router:
https://releases.linaro.org/archive/13.08/components/toolchain/binaries/
There are GPL sources provided, but compiling GCC toolchain from schratch is a large amount of work and the result probably wouldn’t work any better than the Linaro toolchain.

I was pleasantly surprised that Netgear developers have enabled the option to include a kernel configuration in /proc/config.gz, this should help greatly when building new modules for modem’s kernel. /proc/config.gz is a very rare sight on production devices. This modem is the first device I’ve ever seen to have /proc/config.gz enabled out of the box.

What I’ve done so far:
* Customized dropbear SSH server, https://github.com/zcutlip/dropbear-hacks (replace MIPS compilers with ARM ones in the build script) which works without relying on getpwnam(). Now, I can add a password and stop relying on Telnet for shell access.

Ideas:
* zRAM module. OS is extremely poorly optimized. With GUI and SSH running there’s only about 5 MB of RAM free !
* DnsCrypt for secure DNS lookups.
* OpenVPN client for VPN passthrough.
* Password lock for LCD UI

Will update.

Neighbor of mine (who is a part-time video editor) recently had his Mac Book die on him and asked me to do some data recovery for him from a bunch of old HDDs, all, of course, formatted as Apple HFS+. Since he plans to replace his dead Mac with a Dell XPS, he needed me to extract all data from his old drives and reformat them as NTFS. Even though I’ve never ever owned any Apple hardware, I have no problem recovering data from Mac formatted drives, thanks to Paragon’s NTFS/HFS+ Linux driver (free version with limited functionality available here).

But there was one small problem. Two of the hard drives (2TB WD Blacks) were originally used as an Apple software RAID-0 array. While Paragon’s driver can mount HFS+ file systems, it cannot reconstruct or operate RAID arrays. Also, Apple’s RAID format is proprietary and there is no support for it on OS’es outside of OS X/macOS. Since I don’t have access to a Mac, I had no way to recover data from those two HDDs.

Out of curiosity, I’ve plugged in both RAID-0 HDDs to my main rig (booted to Ubuntu 18.04) anyway to check how they’re formatted. Both drives showed as GPT partitioned disks with three partitions on them: EFI, “Apple RAID” and “Apple boot”. It was pretty clear to me that the actual data I needed to extract was on “Apple RAID” partitions. Now how to I get Linux to recognize them as a RAID-0 array ?

I do know that RAID-0 is basically a stripe, alternating chunks of data between multiple drives.
Linux has it’s own RAID implementation in form of mdadm, which, apart from it’s own format, is also capable of reading some other RAID array formats, like Intel’s firmware RAID. But unfortunately, it does not support Apple RAID.
Since RAID-0 is just a stripe, that got me thinking: is there another way to get Linux to treat two block devices as a stripe ? Why yes there is. Say hello to Device mapper. Device mapper is a kernel driver (more precisely – a kernel framework) which allows you to remap physical block devices into virtual ones, including mapping multiple devices as a single device and even doing transparent block device encryption (in form of dm-crypt). This framework is widely used in various Linux-based OSes. For example Android uses dm-crypt for adoptable storage and app containers on SD cards. Device mapper is also used by LVM as a more flexible way to partition HDDs.
And, you probably guessed it, Device mapper supports mapping disks as stripes !

Could it be that Apple just uses a simple stripe for RAID-0 instead of some custom format ?
Again, out of curiosity I’ve tried mapping the two “Apple RAID” partitions as a Device mapper stripe using a simple script which I found online (can’t find the link right now, sorry). And to my pleasant surprise, the resulting stripe was happily recognized by Paragon’s driver as an HFS+ volume ! I did try to open a few files and all were perfectly readable.

I’m now in the process of copying all data from that RAID-0 array to another location, after which I will dismantle the array and reformat both HDDs as NTFS.

So it turns that at least Level 0 of Apple’s software RAID isn’t as proprietary as it seems.