Odroid Xu4 PortaLAN
- 1 Summary
- 2 Machine purpose
- 3 Hardware
- 4 Power
- 5 Physical Buildout
- 6 OS
- 7 Applications
- 8 Setting up wireless client
- 9 Building a Kernel / Updating a kernel
- 10 Setting up bridging between eth0 and wlan1
- 11 HostAPD
- 12 dnsmasq
- 13 pureftp
- 14 lighttpd
- 15 services
- 16 Maintaining
- 17 Project thoughts
I wanted to build a machine that would provide me with the ability to build out a portable version of what I had made and was trying to enhance at our local Makerspace/Hackerspace JaxHax that sadly folded a few years ago. This machine provides fun and useful services to whatever group I go to. Services range from simple internet access point to filesharing/distribution, PXE boot for images and rescue services, along with many other services (see machine purpose section). I really enjoy not only building neat machines but having a really cool machine that allows people to connect and share and interact.
One of major points for this project is to help enhance the local Jacksonville community with something that allows me to come in, sit a machine down, and have something that I can say I'm bringing and people will know will work. This will allow them to bring their equiptment so we can collaborate, work and/or play together. Jacksonville's social-tech scene is fairly devoid of actual tech, and this project is one of my most-involved and best stabs at resolving this problem.
This is still a live document and project, being added to and refined, look for it to be updated in the future.
This is what the machine will do when its running/clients connect to it
It will connect to a pre-defined wireless internet connection then it will provide the following services:
- DHCP - to clients inside it's network
- DNS - local addressing along with blocking of virus/malware/porn/fakenews via Stephen black's hostfile https://github.com/StevenBlack/hosts
- Routing - to allow clients to get to the interwebs and to each other
- Access Point - spawn an access point using HostAP
- (future) TFTP/PXE - allow clients to boot off the network for things such as os images for local organizations+troubleshooting (like linux/windows images) and potentially things like vulnerable images that people could boot vm's of and attack.
- Fileserver - allow clients to download (and potentially upload) material such as conference videos/audio/ctf and other material
- FTP - pureftpd (one I'm using, more security oriented) or proftpd or vsftpd
- HTTP - apache/lighttpd
- (future) bittorrent - rtorrent/opentracker
- (potential) emule?/other p2p host
- (future) IRC server/other local chat server
- (future) Proxy for gaming/OS update services (high bandwidth) to save on bandwidth and speed up downloads a lot
Harbor Freight knock off pelican case 1800 - https://www.harborfreight.com/1800-watertight-protective-case-9-316-in-63518.html
Odroid Xu4: https://ameridroid.com/products/odroid-xu4
32GB EMMC: also from ameridroid
2x Wireless card TL-WN722N v1: <link removed> - I would advise STRONGLY against getting v2 or v3 (rtl8188eu, 2357:010c) of this card. Only v1 (atheros) chipset seems to work alright for a non-hacked up version of hostapd. Others require custom drivers and hostap builds that do not seem to play well with the XU4. Most sellers that say they give v1 seem to still give v2/3, there does not seem to be a way to tell on the outside of the packaging.
Wireless card alternative - Alfa AWUS036NEH (rt2800usb) - https://www.amazon.com/gp/product/B0035GWTKK/ works for hostapd and client for me.
1x 32GB MicroSD: https://www.amazon.com/gp/product/B073JWXGNT
60mm fan (cooling/airflow): https://www.amazon.com/gp/product/B00N1Y493Q
Lenovo advanced plug -> square (sacrafice) for power: https://www.amazon.com/gp/product/B00FYYOWVU (this will be used with one of the 20V 90W power bricks I have laying around)
2x USB3 SATA iii drive atapters with external power & UASP: https://www.amazon.com/gp/product/B06WWLCYC3
2x Buck 12V regulator - https://www.pololu.com/product/2855
Buck 5V regulator - https://www.pololu.com/product/2865
USB 2 - Gearhead USB 2 hub
USB 2 1' extension - https://www.amazon.com/gp/product/B01HN1GSE0s (for wireless cards)
2x drive, adapters come with 12v2A bricks, drive manufacture says 7W read, bricks can deliver 24W, some say rule of thumb is 25W/Drive, so lets calculate in the max possible to not have brownout, 50W
xu4 - 5v up to 4A - 20W
5V 2A overhead on selected regulator for other devices (fans/whatever): 10W
Total power from power brick (peak): 80W
Total power from power brick (nominal): TBD:
Total power supplied from IBM standard (large) brick: 90W
In this section I will be showing the buildout, it took me about 12 hours for the primary build all-told, much of the time was spent with needling over things like trying not to destroy the gigabit switch and its tiny data leads. This is ordered by the order that I built it in.
Here we have most of the parts laid out that I was working on building.
Some parts were replaced such as the white usb cables (did not work with the wireless adapter), usb 3 hub replaced with usb 2 (drives directly connected to xu4 now).
This is the ODroid XU4. They are coming out with a newer board here pretty soon that does all kinda fancy things and will cost more, but this is what I had at the time and built my project around, if you wanna build a similar box, you may want to look at the newer project board, but the XU4 is storied and has most/all of the bugs worked out by the community (and here, thats why you're here :).
I pulled this gigabit switch out of its case, stripped most of the plastic off of one of the ports (left some on for stability since the board is pinned through the pcb by only 2 plastic pegs). Heated up the pins and pulled the pins out of their holes and soldered a cable into them after homing it all out using a multimeter to determine where the cables and pins on the sacraficial cable lined up to be sure. I ended up accidently pulling off 2 of the traces so I had to solder directly into the transformer on those leads. I tried to be careful with the soldering but they were so close and the wires so small that they were shorting, so tried hot glue to isolate them from each other. That did not work, so I ended up just using flags made out of gaffers tape and that seems to work well. If the leads short out on this the switch just protects itself by restarting, not good for sure but keeps the magic smoke from getting out.
Here are the slots I made for the air to go over the hard drives (perhaps I should have made more, the XU4 seems to need more cooling then the harddrives). For the slots I just used a drill and hand slid them then passed some sand paper through to knock the edges down. For the fan I made a template by tracing out the fan, then holding it where I wanted it to be and using a punch to punch guide holes before drilling those. I may expand the holes later to let more air through but for now its an effective finger-detterant with the small holes.
On the initial build I had to figure out how to get everything to fit, and all the connectors to go where I needed them to go, this eventually meant that I needed to stand the XU4 and the switch up on their ends. There are a few problems with this 1> the boards would interact with cables going behind them 2> the boards would interact with the smooth curve of the box's inner wall (pushing them higher) 3> there would be no effective stabilization on the top of the board, leaving only the bottom pinned, and the top to flop around and would be potentially damaging. So my solution was to make shims and use standoffs. I was doing this at about 1am, so there was not much open, and was thinking about something that was sturdy that I could work around the house, I had some aluminum bar, but that was about 4mm thick and I needed something *thin* and *sturdy*. I was thinking a rigid plastic but couldnt think of something I had around the house... Then while kinda poking around, I found a pile of my old 5.56 casings I had collected when I went to the range one time and was thinking of trying my hand at doing refills (until I found out that because 5.56 casing is so thin that is makes it pretty hard to do, and the tools to do it are fairly expensive) so that project got shelved and I just have a bag of cases laying about for whatever purpose... and purpose found! Cut off the thick butt with the primer with a hack saw, just hammered it flat (so its pretty much double-walled) tossed it in the drill press (you could just use a drill) and popped 2 holes in it and bam, shims done! I had to bend em in somewhat and remove some material from the top of the box because its thicker then the bottom but all told it works really really well. I think this would work better then plastic or aluminum, brass is strong but malliable so it makes a good material for this usage.
For our external power, I am using a old 90W 20V IBM thinkpad adapter I had laying around, the system's buck transformer output theoretical max output is 80W, so 90W should be enough to overcome that along with any inefficiencies that may happen in the transfer between 20V -> 12/5V so its pretty perfect for this. I just grabbed a IBM Advanced connector (the female barrel connector) to the slimline connector (the rectangle), lopped off the rectangle right at the end, and stripped it. The wires are not huge, but just spliced some new ones in and ran em to the buck transformers.
This is a pciture of the first run, making things work, everything seemed to work at this point! I was able to take it down to my local 2600 meeting and show off the initial build, for the components in there at this point this was working to the point of where it was for the last meeting except the components were in a nice easy to carry case.
Now I started mounting the second hard drive on the top of the case, not the greatest space, but when dealing with a tiny case, you gotta find space where you can. I put rubber grommets on the top and bottom of the screw to help with shock absorbsion.
- Yellow 12v
- White 20v
- Red 5v
- Black neutral
This is the teardown of the tplinks that I was attempting to use. The driver status of these guys is pretty poor at the moment in linux, so I ended up needing to find alternatives, but here are the pictures for these guys none-the-less. The mounting position is pretty perfect for em, when I get a new card I'll have to determine where I can put them, hopefully they are similar.
This is a shot of the antenna cables I'm using to get the signal outside the box, I could have tried to port the antenna out right from where they lay but this is more elegant, also safer and looks nicer. I offset the outside connectors so you could move the antennas into a few configurations (assuming that there are ethernet cables beside it so you couldnt put anything the other direction). One point of this being that you could put the box on its bottom like here or its back or top, where the only connections lie on the two sides (power on one and connections on the other).
And here's the full view at the end of the main-build process, another picture or two will be forthcoming with the modifications made with different cables and cards I've had to use in the mean time. I'm really happy how it all came out!
Core OS: Funtoo
Why Funtoo/Gentoo: You can optimize and customize funtoo/gentoo very well, and their documentation and community support is very good. I enjoy playing with linux, feel free to build the odroid on the default ubuntu that comes with it, or armbian, or whatever you want, but with this I have fine tuned control, and its fun!
Why Funtoo over Gentoo: Funtoo is pretty much just a flavor of gentoo, from what I've read, they split up some of the releases of packages a bit better that may break the distro so sometimes it may be a bit more stable and not break on just some random occasion so I wanted to give it a shot. This build doc should for the most part work for gentoo as well excepting for some things like git inside of portage and profiles instead of eselect, but we'll see as we move through it, it's an adventure.
Initial image pull and deployment
Per Funtoo's and Gentoo's buildout pages, they suggest you start with the ubuntu image, I have worked on building a from-scratch method of installing, but its tedious to do since I have never done it before, so in the mean time to get it working I'll go with the method provided.
We will be using these materials on the web for our build of Funtoo:
Other resources people may find useful for a buildout:
- https://blogs.s-osg.org/install-ubuntu-run-mainline-kernel-odroid-xu4/ - Samsung Open Source Group install guide for Ubuntu with mainline Kernel
- https://wiki.gentoo.org/wiki/Xu4 - Gentoo buildout docs
- https://archlinuxarm.org/platforms/armv7/samsung/odroid-xu4 - Arch Linux - I had issues with their image in the past (it was armbian inside, such wierdness), but just checked and its back to being arch it seems.
- https://www.armbian.com/odroid-xu4/ - Armbian for xu4 based on ubuntu or debian (check the other downloads section)
Building from scratch
This section will hopefully be built out in the future.
If you are interested, on some neat stuff about the back-end on how ODroid works, expand this section. If not, skip to the next section, the section on building from scratch has not been completed.
I don't want these to go to waste, so I'll go ahead and include these in the wiki, if you want to do a scratch build, these resources will give you a leg up, I'll probably get back to this at some point, or maybe not.
Links to information I was diving into at the time
- http://linux-exynos.org/wiki/Samsung_Exynos_5422 - Project for the chipset that the XU4 uses
- http://linux-exynos.org/wiki/ODROID/u-boot - how to build the u-boot resources, sd-fusing, and firmware
- http://linux-exynos.org/wiki/ODROID/Hardkernel_Linux_kernel - Hardkernel's kernel, don't know how useful this would be, the normal gentoo kernel seems to be usable
- http://linux-exynos.org/wiki/Installing_a_rootfs - installing rootfs, specifically this works for gentoo, they have other operating systems in there too
- https://wiki.odroid.com/odroid-xu4/odroid-xu4 - the xu4 wiki has a ton of useful information (no kidding) but felt it should be included here, the linux-exynos project is more of a low-level project whereas the odroid wiki is more specific to the odroid itself, both resources are very useful.
- I built a gentoo VM and started going through building the cross-development toolchain ( https://wiki.gentoo.org/wiki/Cross_build_environment and https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Full ) but I started getting in pretty deep to that and started getting annoyed with my lack of progress, this is where I need to restart if I come back to the from-scratch probably.
Not gonna get really into how to build toolsets here (unless I do a from-scratch build for ya) but the links above have all that material and how to do it, enjoy.
The official Funtoo/Gentoo Build way
Ubuntu pull and flash, gutting, Funtoo laydown
First we will start with pulling the most-current Ubuntu image from ODroid, checking the md5sum and flashing that to our SDCard.
- Image lists: https://wiki.odroid.com/odroid-xu4/os_images/linux/ubuntu/ubuntu
- Image info for the one I'm starting with: https://wiki.odroid.com/odroid-xu4/os_images/linux/ubuntu_4.14/20171212
I pulled the newest version of minimal image for the XU4:
On the host machine:
On the odroid:
Here I then stuck the card into the odroid, booted (had to reboot once on mine) then logged in.
I wanted to make sure the card worked, the odroid worked, and the / had resized to the full size of the card, I shut off the box gracefully with shutdown -h now and put the card back in the host.
On the host machine:
At this point, I was able to have a working Funtoo setup and continue with the install doc - https://www.funtoo.org/Install
Now that we have a basic running system, we can set up some customization that will let us run the XU4 the way that we want.
Start by setting up the network so we can get our starter packages and initial setup stuff. You can either do that with a...
Static IP address:
Or DHCP address (just start dhcpcd, it listens everywhere):
Now lets update portage, do our make.conf optimizations for the XU4, and then update all the packages.
Here's a little script I tossed together that just watches the temp and frequencies (in Cx1000 and MHZx1000) to see if you are thermal throttling. I have a full-sized heatsink with no fan, but currently have a 80mm pointing at it for the buildout.
Now set up your timezone and hostname of your system:
Build yourself a new user:
And finally set yourself up a nice motd by editing the /etc/motd and reboot:
Now onto more specialized things :)
Things I needed to install and why after initial install:
- net-fs/nfs-utils - mounting nfs mounts
- sys-block/parted - gpt partition larger then 4tb
- sys-apps/hdparm - changing my wd-red drive's rest state to be less efficient (more performance, less sleepy)
- sys-apps/ethtool - helping view my physical ethernet's status
- net-analyzer/vnstat - (add USE flag 'gd' for graphics generation) for viewing speed of traffic over interface and stats
- sys-process/cronie - cron daemon
- sys-apps/mlocate - helps find files on the filesystem
- net-wireless/iw - wireless network configuration utility
- net-wireless/wpa_supplicant - allows connecting to encrypted access points
- net-wireless/hostapd - creates a local access point using one of the wireless cards
- app-editors/vim - I like the vim editor, funtoo comes with nano and vi
- app-admin/sysklogd - system logger daemon
- app-admin/logrotate - log rotator
- net-wireless/wireless-tools - wireless tools
- dev-util/strace - for low level troubleshooting of processes
- sys-apps/lshw - great tool for checking hardware and driver status
- net-dns/bind-tools - host, nslookup, ect
Setting up wireless client
wireless config: wpa_supplicant + dhcp: https://wiki.gentoo.org/wiki/Wpa_supplicant
If you would like being more or less secure bout connecting to ap (using bssid mac, not letting repeaters in, ect): https://wiki.gentoo.org/wiki/Wpa_supplicant#Editing_manually
Building a Kernel / Updating a kernel
You may want to do this for updates, or to compile a new driver in (I wanted both)
Go ahead and back up the /boot stuff we made from ubuntu and such
Thanks for the assist from this post https://forum.odroid.com/viewtopic.php?f=52&t=1674 that used this article as a template: https://zozs.se/2013/05/23/tp-link-wn725n-in-arch-linux-arm/
another doc: http://odroid.us/mediawiki/index.php?title=Step-by-step_Native_Compiling_a_Kernel http://linux-exynos.org/wiki/ODROID/u-boot https://github.com/hardkernel/linux/tree/odroidxu4-4.14.y/Documentation/admin-guide
note I'm skipping his step 1 where I'm syncing the os kernel (because I'm gonna use this one for the OS kernel)
Goto https://github.com/hardkernel/linux/ and select the branch you want to use, I am using odroidxu4-4.14.y so my url ends up being https://github.com/hardkernel/linux/tree/odroidxu4-4.14.y
On our currently running odroid system, lets just fill in some variables so its easy for everyone to just copy paste things, change yours if they are different:
Lets now fetch the new kernel using git and link it.
cd to the new linux directory, copy the xu4 default config over to be our default kernel config
If you want to see/edit the config, you can install ncurses by doing this:
Lets build our kernel
Now we need to make the initrd, and package it to where the uboot system can read it then copy both to the /boot filesystem (not overwriting the ones that are currently running the system because we want those as backup, you *could* remove those because they are in memory).
you will need to merge dracut (to make initrd) and u-boot-tools (to manipulate initrd image for uboot)
I'm calling mine <name>-xu4-4.14.18.021318 (that's xu4-<kernel>.<date>). Name yours whatever you want.
Lets modify our boot.ini a bit (change the zimage, uinitrd at least, I removed a bunch of comments and a few options/if statements)
If your kernel does not work for whatever reason, just put your storage in another machine, mount /boot, grab the boot.ini backup you made at the beginning of the section and replace it here, and as long as you didnt replace the uInitrd and zImage (or if you did, grab those from the backup), ittl just work and you'll be good to go.
Here you need to put your zImage.whatever name and initrd name that you made in the fatload lines you see below.
During my first public test of it, it went down, and I saw this process taking up all of the processor, I couldn't figure out what it was doing, couldn't strace it. Later on I got a clue, someone said it was related to a module, so I looked up the module and this guy was for a touch interface... I don't even have one of those, so on the module blacklist it goes! modprobe -r'd it and problem went away, blacklisted the module and we were good to go.
top looked like this
So just make a file like this, so it doesnt come up at boot
If you need to do it right now:
I was having problems with getting the version 3 of the tplink usb wireless cards working, the new kernel allowed me to have those work (at least seemingly) out of the box, further testing is necessary. The ID for those cards is 2357:010c. They don't have a description but thats the OS' fault not the kernel, the device shows up now and thats fantastic!
Setting up bridging between eth0 and wlan1
I want to have a single network where clients both physical and wireless can connect and access resources and each other, for this I will be using both my physical interface eth0 and my wireless interface wlan1.
Bridge configuration for wlan0/eth0 - https://www.funtoo.org/Networking#Bridge_Configuration
Make sure you have the right utilities, I had bridge-utilities already
Due to things like hostap seeming to be picky about how the rc-update script was named, had to change to netif from net.device... so their documentation is inconsistent.
Its dumb you need to do this, but if you dont ittl autoconfig... Then deny dhcp to the interfaces we dont want dhcp auto-configured to.
Setting up hostap daemon for software access point running off of wlan1.
Gentoo hostapd document: https://wiki.gentoo.org/wiki/Hostapd
Currently this setup will run you maxed out at about 65Mb/s on 802.11n, you will need to add variables to allow for usage of ht capabilities. Those are described further in this document: https://wireless.wiki.kernel.org/en/users/Documentation/hostapd#Wireless_Interface .. My config will be added here later once I figure out what works for mine consistently.
I use this for DHCP and DNS resolutions.
PureFTPd setup, mostly default for the moment.
LigHTTPd setup, setup to allow for directory services and allow people to traverse.
Whats usually running on the system
In /etc/profile I added a line to the end to make the default editor vim:
In cron I added hdparm lines to turn off spindown and power savings for both hard drives because I don't want them doing that (with a long sleep to allow them to spin up first else it will not work)
Also there is a mount -a in there as well, because the drives take too long to come up for the boot process so I just wait a lil bit and mount them. The XU4 boots very fast.
I started vnstat monitoring of my network interfaces for bandwidth:
Enabling important services at start and doing some configs:
You would probably want to back up this system before taking to things like defcon and restore it after you get back just to make sure noone has a long-term intrusion
To backup the system fully before going to a risky con or something, take whatever you want to off the large disks, and then pull the main os storage volume, mount on a linux machine, then:
Another way you can backup (and preferred normally) would just be to build an rsync job or use some other utility to ship the data off to some other device using either includes or excludes to keep things like /proc /dev /sys /tmp and other 'virtual' directories from being backed up.
This would remove all the data on the device you pointed it at, so be careful. Take your trusted backup and restore it over the contents of the card.
- ego sync
- emerge -uDN @world
Ideas on what you could change to make the project different to fit your needs:
- If you wanted a larger box, you could conceal the antennas inside the box without having problem with signal (by getting the antennas away from the devices and power), this also makes it easier to add other systems like batteries or capacitors internally so if you need to switch from one power source to another the server doesn't go down. (like a mini-battery backup)
- You could make it waterproof by adding heatsinks and thermal gates, or a watercooling system, if you are going to environments that require a very wet based system (like emergency support)