I wanted to share a story of a segmentation fault I helped track down this weekend. I thought the final root cause of the segfault was interesting because of how unrelated it was to the code I was trying to debug.

I’ve been maintaining a Linux fork of obs-ios-camera-source, which is an OBS plugin that allows you to use an iPhone or iPad’s camera and microphone as a video and audio source in OBS. It works in conjunction with the “Camera for OBS Studio” app in the App Store. This kind of thing is useful for online streamers who want to use their phone’s camera instead of buying a separate camera. For those of you who don’t know, OBS is short for Open Broadcaster Software. A lot of streamers use it to handle broadcasting their stream. It allows you to capture audio and video, mix it all together, do all kinds of cool things with it, and then record the final result and/or stream it to sites such as YouTube and Twitch.

Getting this plugin working on Linux wasn’t really complicated, because it was already well-written without much platform-specific code. After all, the existing codebase was already operational on both macOS and Windows. It mostly just required tweaking a few compile/link options to make the code run happily on Linux.

Anyway, I’m pretty sure a good number of people have been using my Linux port of this plugin without issues. I know it works fine for me when I test with it in Ubuntu 18.04 or 20.04. I’ve helped people on other distros get it working too. I don’t really do any streaming myself — maybe someday though!

On Friday, GitHub user rrondeau reported an issue: after a half a year of the obs-ios-camera-source plugin working without a problem, it suddenly started causing OBS to segfault on his computer (currently running Fedora 33). He provided a stack trace that showed that the segfault was happening because of something initiated by the plugin. Afterward, he used GDB to get a better stack trace that provided more info about the functions being called and the parameters being passed:

#0  0x00007fffee7abc64 in socket_send () at /usr/lib64/samba/libsamba-sockets-samba4.so
#1  0x00007fff88b7813c in send_packet (sfd=50, message=8, tag=1, payload=0x1b22e60, payload_size=488) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/libusbmuxd/src/libusbmuxd.c:400
#2  0x00007fff88b782a6 in send_plist_packet (sfd=50, tag=1, message=0x1ae53e0) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/libusbmuxd/src/libusbmuxd.c:431
#3  0x00007fff88b7851b in send_list_devices_packet (sfd=50, tag=1) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/libusbmuxd/src/libusbmuxd.c:499
#4  0x00007fff88b79367 in usbmuxd_get_device_list (device_list=0x7fffffffc740) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/libusbmuxd/src/libusbmuxd.c:938
#5  0x00007fff88b725e1 in portal::Portal::addConnectedDevices() (this=0x1909378) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/portal/src/Portal.cpp:109
#6  0x00007fff88b72684 in portal::Portal::reloadDeviceList() (this=0x1909378) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/portal/src/Portal.cpp:126
#7  0x00007fff88b722db in portal::Portal::Portal(portal::PortalDelegate*) (this=0x1909378, delegate=0x1909240) at /home/rrondeau/git/perso/obs-ios-camera-source/deps/portal/src/Portal.cpp:57
#8  0x00007fff88b67053 in IOSCameraInput::IOSCameraInput(obs_source*, obs_data*) (this=0x1909240, source_=0x1aee000, settings=0x19210a0)
    at /home/rrondeau/git/perso/obs-ios-camera-source/src/obs-ios-camera-source.cpp:74
#9  0x00007fff88b66358 in CreateIOSCameraInput(obs_data_t*, obs_source_t*) (settings=0x19210a0, source=0x1aee000) at /home/rrondeau/git/perso/obs-ios-camera-source/src/obs-ios-camera-source.cpp:371
#10 0x00007ffff6259c2a in obs_source_create_internal () at /lib64/libobs.so.0
#11 0x00007ffff626bb81 in obs_load_source_type () at /lib64/libobs.so.0
#12 0x00007ffff626e3c2 in obs_load_sources () at /lib64/libobs.so.0
#13 0x000000000049e750 in OBSBasic::Load(char const*) (this=0xa370b0, file=0x7fffffffd040 "/home/rrondeau/.config/obs-studio/basic/scenes/Untitled.json")
    at /home/rrondeau/git/perso/obs-studio/UI/window-basic-main.cpp:973
#14 0x00000000004a2976 in OBSBasic::OBSInit() (this=0xa370b0) at /home/rrondeau/git/perso/obs-studio/UI/window-basic-main.cpp:1783
#15 0x000000000047feff in OBSApp::OBSInit() (this=0x7fffffffd690) at /home/rrondeau/git/perso/obs-studio/UI/obs-app.cpp:1415
#16 0x0000000000482503 in run_program(std::fstream&, int, char**) (logFile=..., argc=1, argv=0x7fffffffdd68) at /home/rrondeau/git/perso/obs-studio/UI/obs-app.cpp:2052
#17 0x0000000000484203 in main(int, char**) (argc=1, argv=0x7fffffffdd68) at /home/rrondeau/git/perso/obs-studio/UI/obs-app.cpp:2697

The actual segfault was happening inside of a function called “socket_send” in libsamba-sockets-samba4.so, which was being called by a function in libusbmuxd, which is bundled as part of the obs-ios-camera-source plugin source code and is used for communicating with iOS devices over USB. When I first saw this in the stack trace, my mind thought “Huh…that’s weird. Why does libusbmuxd use Samba’s library for its socket code instead of providing its own?” (Samba is an implementation of the Windows file sharing protocol used by pretty much every Linux distribution)

I tested and couldn’t reproduce the issue in Ubuntu. I know basically nothing about Fedora, but I faked my way through grabbing a Fedora 33 virtual machine, installing OBS, and compiling the plugin. I ran into the exact same issue that he was seeing.

Before I had a chance to look deeper and understand what was going on, rrondeau beat me to the correct conclusion: code in Samba’s library was mistakenly being called. libusbmuxd has a function called socket_send, but clearly libsamba-sockets-samba4’s function that is also named socket_send was accidentally being called instead.

Honestly, that’s all we really needed to know. Renaming libusbmuxd’s socket_send function to something else, and updating all references to it to use the new name, fixed the issue. I still wanted to understand why this suddenly became an issue when it had been working fine prior to that. Why were we calling into Samba libraries? Why does an iOS USB multiplexing library even consider talking to a library associated with Windows file sharing?

Not knowing the answer to that question bothered me. I decided to dig deeper and understand exactly what was going on. I started by using ldd, which lists all dynamic libraries used by a program or library:

[fedora@fedora33 build]$ ldd obs-ios-camera-source.so 
	linux-vdso.so.1 (0x00007fffa599a000)
	libobs.so.0 => /lib64/libobs.so.0 (0x00007f0a3f688000)
	libavcodec.so.58 => /lib64/libavcodec.so.58 (0x00007f0a3e2db000)
	libavutil.so.56 => /lib64/libavutil.so.56 (0x00007f0a3e036000)
...
	libsamba-sockets-samba4.so => /usr/lib64/samba/libsamba-sockets-samba4.so (0x00007fd6af4b7000)
...

I truncated the output because it spit out a very long list of libraries. As we can see from ldd’s output, obs-ios-camera-source.so depends on libsamba-sockets-samba4.so. ldd lists all recursive dependencies as well, and I couldn’t find any references to “samba” in the plugin source code, so this was likely an indirect dependency instead. I confirmed this by using readelf to show only the direct dependencies:

[fedora@fedora33 build]$ readelf -d obs-ios-camera-source.so | grep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libobs.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libavcodec.so.58]
 0x0000000000000001 (NEEDED)             Shared library: [libavutil.so.56]
 0x0000000000000001 (NEEDED)             Shared library: [libobs-frontend-api.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

At this point I used ldd and readelf to walk through the tree of dependencies and figure out what was actually linking against the Samba libraries. I later learned that I could have installed lddtree (part of the pax-utils package) to do this automatically. Either way, this led me to discover that the Samba libraries were being included through libsmbclient, which was a dependency of libavformat (part of FFmpeg). libavformat is a dependency of libobs.

Repeating this experiment on Ubuntu showed that libavformat on Ubuntu does not depend on libsmbclient. This explains why I couldn’t reproduce the issue on Ubuntu. So why does Fedora’s (well, RPM Fusion‘s) version of libavformat depend on libsmbclient?

It turns out that it’s a compile-time option for FFmpeg. libavformat contains code for talking with Windows servers using libsmbclient, but it’s an optional thing that you can choose to enable at compile time. Clearly Ubuntu chooses not to enable it, but RPM Fusion does. Actually, I found the exact post on RPM Fusion’s commits mailing list where the patch was added for enabling SMB support in FFmpeg. This patch is what led to the whole issue happening. If Ubuntu’s version of FFmpeg was being built with SMB support, we would have seen this a long time ago. This commit to RPM Fusion was made on December 31, 2020, which explains why rrondeau had only recently begun seeing the problem.

The root cause here is that the obs-ios-camera-source plugin was linking against two libraries that both provided a function named socket_send: libsamba-sockets-samba4 (indirectly through libobs) and libusbmuxd. libusbmuxd was being linked statically, but that doesn’t prevent functions in it from being resolved through dynamic linking rules anyway. So even though libusbmuxd was a static library with its own internal implementation of socket_send, it was using libsamba-sockets-samba4’s implementation instead.

rrondeau and I settled on changing what we had control over: the libusbmuxd source code embedded inside of the plugin’s source code. We went with simply adding a “usbmuxd_” prefix before all of the socket_ functions. There may be a more complex way of forcing it to use its own internal version of socket_send through linker options, but I feel that this is probably the simplest solution. It’s easy to implement, and it gets the job done.

This segfault turned out to be a pretty simple issue to solve and diagnose. Is it really worthy of a blog post? Maybe, maybe not. I could definitely foresee someone else running into this issue with another combination of libraries. socket_create, socket_close, socket_send, etc. are such generic names that it may happen again. This is a great opportunity to remind everyone: don’t use generic function names like this in your shared libraries, at least not in your exported symbols! You could easily run into a situation similar to this one. In my opinion, prefixes are definitely a good idea for your library’s exported symbols. In this case, both libusbmuxd and Samba were breaking that guideline.

This can be tricky because dynamic libraries on Linux export all symbols by default unless you specify otherwise. This is backwards from how Windows works with DLLs. Windows DLLs require you to specify which functions are being exported. I actually like that approach better! Here’s an interesting reference on how to customize the visibility of your Linux dynamic library’s symbols.

libusbmuxd already fixed this on their end quite a while ago — they now only export functions intended to be public, which have a usbmuxd_ or libusbmuxd_ prefix. I think the version included with the plugin’s source code is quite a bit older. For fun, I tried applying the visibility fixes from the linked patch to the plugin’s embedded libusbmuxd source code. The patches don’t apply cleanly because the embedded libusbmuxd code is actually built using CMake, so I have to add the compiler flags to CMakeLists.txt. After doing that, it does indeed cause libusbmuxd’s internal socket_send function to be called instead, and thus fixes the segfault.

What do you think? Would it make sense to try to convince the Samba project to rename their exported socket functions, or would I be barking up the wrong tree? I suspect that Samba’s socket library is actually intentionally exporting these functions so that other Samba libraries can call the socket functions. Would renaming Samba’s exported socket functions to give them less generic names cause a ton of incompatibilities given how long those function names have existed? Is it too late at this point? Am I wrong to think that Samba’s exported socket functions should have a “samba_” prefix or something like that?

I like my EarPods. Yeah, not the new fancy wireless ones. Just the standard wired earbuds that have come with iPhones for a long time. At one point I realized I prefer to use my EarPods during meetings. I was actually using my iPad instead of my computer to join meetings simply because I could use the EarPods.

Read the rest of this entry

I was looking at one of my classic Macs a few weeks ago, and noticed that my Ubuntu 18.04 netatalk server wasn’t showing up in the Chooser anymore. If you’re not familiar with netatalk, it’s an implementation of Apple Filing Protocol (AFP) that runs on Unix-like operating systems such as Linux and NetBSD. It allows other operating systems to act as Mac file servers. Version 2.x, which I use, supports the ancient AppleTalk protocol. This allows it to work with really old classic Macs that don’t even have a TCP/IP stack installed. Support for AppleTalk was removed in version 3.x, so that’s why I’m still using 2.x.

I checked out the server, and noticed that atalkd wasn’t running.

doug@miniserver:~$ ps ax | grep atalkd
3351 pts/0 R+ 0:00 grep --color=auto atalkd

Hmmm….why wouldn’t atalkd be running? I went ahead and tried to restart netatalk:

Read the rest of this entry

A while ago, I put 16 GB of RAM into one of my computers. The computer is using a Foxconn P55MX motherboard with a Core i5 750. It’s old and could probably be replaced, but it still works for what I need.

Here’s the interesting part. This motherboard doesn’t officially support 16 GB of RAM. The specs on the page I linked indicate that it supports a maximum of 8 GB. It only has 2 slots, so I had a suspicion that 8 GB sticks just weren’t as common back when this motherboard first came out. I decided to try anyway. In a lot of cases, motherboards do support more RAM than the manufacturer officially claims to support.

I made sure the BIOS was completely updated (version 946F1P06) and put in my two 8 gig sticks. Then, I booted it up into my Ubuntu 16.04 install and everything worked perfectly. I decided that my theory about the motherboard actually supporting more RAM than the documentation claimed was correct and forgot about it. I enjoyed having all the extra RAM to work with and was happy that my gamble paid off.

Then, a few months later, I tried to boot into Windows 10. I mostly use this computer in Linux. I only occasionally need to boot into Windows to check something out. That’s when the fun really started.

Read the rest of this entry

After I upgraded Ubuntu MATE from 16.04 to 18.04, I noticed a really annoying behavior. Clicking in the empty area of a scrollbar (between the handle showing where your current position is and the up or down arrow) used to scroll up or down a page at a time. After the 18.04 upgrade, this behavior changed, and now it moves the handle directly to the position where you clicked. Some people like this, some people don’t. I don’t. Unfortunately, they didn’t decide to make this a configurable option in the graphical settings for some strange reason.

It turns out that this is actually a GTK 3 behavior — it’s caused because MATE recently changed to use GTK 3. Luckily, it’s an easy fix:

Go into the directory of the theme you are currently using. So for my current configuration, I went into this directory:

/usr/share/themes/Ambiant-MATE/gtk-3.0

Edit the “settings.ini” file. Add the following line to the bottom of it:

gtk-primary-button-warps-slider = false

Log out and back in, and you’re done.

After upgrading my machines from Ubuntu MATE 16.04 to 18.04, I noticed that something was subtly wrong with my top menu/panel: the network notification icon was missing. Something about the 16.04 to 18.04 upgrade process broke it. It happened on both of the machines I upgraded, so I don’t think it was a random thing that only I was experiencing.

Note: my fix for this will reset your panel layouts, so take a screenshot and/or save any info about the layout of your panels before following the steps below so you will have a reference for restoring it back the way you want it afterward.

To start out, I will share some background info: if you are used to the default menu layout of the top panel with the Applications, Places, and System menus, they decided to change this in 18.04. That layout is called “Traditional”, and Ubuntu MATE 18.04 now defaults to a new layout called “Familiar”. I personally don’t like the Familiar layout because it doesn’t have the Places menu, but I guess enough people liked it that they wanted to make it the default.

Anyway, the fix for this is to reset your panel to one of the default configurations, which will add everything back properly. But like I said in bold above, it will also erase any customizations you have done. Here’s the fix:

  1. Open up MATE Tweak. In the Traditional layout, it’s in the System -> Preferences -> Look and Feel menu.
  2. In MATE Tweak, click on the Panel category.
  3. In the top dropdown menu, choose the layout you want. To match Ubuntu 16.04’s default, choose Traditional. To match Ubuntu 18.04’s default, choose Familiar.
  4. Confirm that you want to change the layout by clicking OK.
  5. All done! Your network notification icon should be restored, and now you can reapply any of your customizations to the panel.

Another way to fix it is to right-click on the panel and choose Reset Panel, but I think the method I described above is better because it lets you pick which layout you want to start from.

I recently upgraded a couple of my machines from Ubuntu MATE 16.04 to 18.04. On both of them, I noticed that after the upgrade, the login window did not look correct. There was no background image. It didn’t look like a fresh 18.04 install. Plus, any changes I did in the Administration -> Login Window program didn’t take effect. I did some digging, and figured out what was going on.

Sometime between 16.04 and 18.04 (it appears that this happened starting in 17.10), Ubuntu MATE decided to stop using lightdm-gtk-greeter, and instead switched to using slick-greeter. It seems to be that on update installs, lightdm-gtk-greeter is not being removed, so it’s defaulting to using it rather than slick-greeter.

I was able to solve this very easily by running the following command:

sudo apt purge lightdm-gtk-greeter

After that, I logged out, and Ubuntu MATE’s correct login screen appeared. It’s as simple as that!

At work, we still have a Windows 98 computer that we use for programming a few very old products that we no longer sell but still support. I tried to do some research so we could move this process onto a newer computer, but I wasn’t able to find a suitable tool. Thus, I’m stuck maintaining a Windows 98 computer until we no longer have to support the product. I do any necessary development work on a different computer, but I still need to get the program binaries onto the Windows 98 machine. The computer doesn’t have a floppy or CD drive, and Windows 98 didn’t come with a USB mass storage driver. Honestly, all of those solutions would be inconvenient for me anyway.

It’s super insecure, but you can enable file sharing on the Windows 98 computer and connect to it from Linux. Programming binaries to old microcontrollers is literally the only thing this computer is used for, so I’m not too concerned about the security pitfalls of SMB.

There are several tools designed for accessing an SMB share from Linux. Most modern desktop environments provide an interface to access Windows file shares. From the command line, I have used smbclient from Samba, which also works great in most situations. When we recently added subnets to our work network and the Windows machine was on a different subnet from my development machine, I had to come up with a solution to connect to it because it no longer showed up automatically. I didn’t want to set up a WINS server to enable cross-subnet Windows file sharing just for this one little application. I couldn’t get smbclient to connect to the Windows 98 computer properly across subnets in this scenario, so this blog post will focus on how to use Linux’s built-in CIFS support so you can use the standard “mount” command, which does work across subnets if you do it right.

First of all, you will need to ensure you have mount.cifs installed. In Ubuntu 16.04, you can type the following command to install it:

sudo apt install cifs-utils

Here is a sample mount command:

sudo mount -t cifs //192.168.2.7/MYSHARE /tmp/windows -oservern=MYSERVER,guest,vers=1.0

Here is an explanation of each parameter:

  • -t cifs
    • This specifies to the mount command that we’re trying to mount a CIFS share.
  • //192.168.2.7/MYSHARE
    • This is the SMB server and share you are trying to mount. 192.168.2.7 is the IP address of the server. MYSHARE is the name of the share on the server. Note that I am using forward slashes here, not backslashes like you would expect on Windows.
  • /tmp/windows
    • This is the local directory to mount the share on. Make sure you create the directory before trying to run this command.
  • -oservern=MYSERVER…
    • “-o” specifies options. Everything following is a comma-separated list of options:
    • servern=MYSERVER
      • This option specifies that the name of the server you are connecting to is MYSERVER. You have to specify the name of the server you’re trying to connect to if you try to mount a Windows 98 share, or else it won’t work. So this option is super important.
    • guest
      • This specifies that we’re trying to connect as a guest. Otherwise, you can use the username= and password= options. It’s probably not a good idea to use the password= option though, because it leaves your password visible in plain text; there is a credentials= option that works a lot better by letting you specify a filename containing credentials.
    • vers=1.0
      • This tells the kernel to use the SMBv1 protocol. It used to be the default before Linux 4.13 came out, but now Linux defaults to a newer protocol for security reasons.

There are many other parameters that you may need to use for various reasons. You can get information about all of the available parameters by typing this command into your terminal:

man mount.cifs

Note that prior to Linux 4.13, I did not have to specify the protocol version as 1.0. It just worked. But with Linux 4.13 and later, it defaults to a newer protocol version for security reasons. If you try to connect to a Windows 98 share without specifying protocol version 1.0, the mount command will hang and eventually fail. I was able to determine what was going on by looking at a Wireshark trace, which then led me to the mount.cifs manpage to discover how to specify an older protocol version.

I have noticed that if you try to replace an existing file, you will get an error. So in my experience, you have to delete the existing file first. This is not a big deal in my use case.

To unmount it when you’re done:

sudo umount /tmp/windows

Hope this helps someone out there!

After upgrading my Ubuntu server from 14.04 to 16.04, I noticed that legacy AppleTalk clients connecting to my netatalk (afpd) server showed an empty volume list; I could no longer access my “Home Directory” share that showed up by default in 14.04.

After some research, I discovered that the 14.04 to 16.04 upgrade brought netatalk from version 2.2.2 up to 2.2.5. It turns out that 2.2.5 has a bug which causes AppleTalk clients to not see anything from /etc/netatalk/AppleVolumes.default; only /etc/netatalk/AppleVolumes.system is read. Here is a link to the bug report I posted.

You could rebuild netatalk 2.2.5 with the latest unreleased patches as I mentioned in my first comment of the bug report above, or you can do what I did: just put your volumes into /etc/netatalk/AppleVolumes.system instead. I’m unsure if the “~” share works properly when it’s in /etc/netatalk/AppleVolumes.system, so I hardcoded it to my home directory path just in case. It works fine!

Recently I had an idea: what if I maintained a local server to host my Ubuntu install ISOs so I could load them through the PXE boot capability available on all of my computers? I would never need to create an Ubuntu install USB drive ever again — instead, I could just tell the BIOS or UEFI to boot through the network card, pick the ISO I want from a list, and boot right into it. Every x86/amd64 computer I’ve ever used (with the exception of Macs) has had an option in the BIOS/UEFI for booting through the Ethernet port, but it’s something I’ve never been able to try, and tutorials on it are usually old and/or incomplete. I finally decided to try to figure it out. I got it working after a lot of trial and error.

I found plenty of tutorials, but none of them had everything that I needed to know in order to get this setup working. Most of the tutorials that were Ubuntu-specific applied to really old releases (10.04 and below), and generally didn’t support both UEFI and BIOS simultaneously. I can verify that the process I figured out will work for both legacy BIOS and modern UEFI systems, and it will properly boot into an Ubuntu 16.04 desktop live ISO. Specifically, I have tested it with an Ubuntu MATE live ISO, but it should work with all other variants too.

dnsmasq

These instructions will use dnsmasq, which is a nice little DNS/DHCP/TFTP server program that you can install in Linux. If you are using something like DD-WRT, there is a good chance you already have dnsmasq! However, for these instructions it’s apparently important that you use the latest version as of this writing, which is 2.76. I didn’t test with earlier versions, but according to sources online, version 2.76 contains bug fixes for PXE booting with UEFI. So to start, you need to make sure you have dnsmasq 2.76. If your server is an Ubuntu 16.10 server, you can simply install dnsmasq with apt–it’s already the latest version. Otherwise, you may need to manually grab the .deb from 16.10, or compile it yourself. Keep in mind that desktop versions of Ubuntu run dnsmasq in the background for DNS, so you may have to set up something special if you want to run this server from a desktop install of Ubuntu without interfering with the existing dnsmasq. I’m not 100% sure–I did this on a server install so that wasn’t an issue.

If you already have a different DHCP server program, you can probably adapt these instructions to work with it, but you’re on your own to figure out the required syntax.

Proxy DHCP or not?

There is a decision you will have to make which will affect how you configure dnsmasq. Do you want to run dnsmasq as a normal DHCP server, or a proxy DHCP server? Let me explain the choices. As a forewarning, this may take a while to fully explain, but it’s all useful information.

When I say “normal DHCP server”, what I mean is that dnsmasq will replace your existing DHCP server. So you would set up dnsmasq to assign IP addresses to all of your computers and devices on your network, and turn off the DHCP server built into your home router. This new DHCP server provided by dnsmasq will recognize when your computer is trying to netboot, and will tell it to use TFTP to download a file from a server to boot. As you will see shortly, this is probably your best option. It’s understandable, though, if you don’t want to mess around with the DHCP server that you already have which is working just fine with no problems whatsoever. If you’re in that camp, there is another option available: a proxy DHCP server.

If you already have a DHCP server on your network that you can’t configure for PXE but you still want to use (e.g. the DHCP server in your home router), you can set up dnsmasq as a proxy DHCP server that works alongside your existing DHCP server. Your existing DHCP server will still be responsible for assigning IP addresses to all of your computers and devices, but the proxy DHCP server will provide the necessary information that PXE clients need for booting (the TFTP server IP address and file path). You will not have to change any configuration on your existing DHCP server; the proxy just provides additional information that all PXE-enabled BIOS/UEFI implementations are supposed to support. PXE was designed to support this use case, and every computer I’ve tested so far works with it just fine.

With both of these options available, the choice may seem obvious to you: use it as a proxy DHCP server! It’s a low-risk choice because you’re not messing with the configuration of anything else already on your network. In theory, this would be a great idea. In practice, though, there are problems with booting in proxy mode with UEFI. It’s not actually dnsmasq’s fault at all. It’s not the PXE protocol’s fault, and it’s not even the fault of your computer’s UEFI implementation. The fault lies in shim and grub. Let me explain further.

shim is a bootloader that serves a single basic purpose: to load grub. The reason it exists is because it’s possible to have it signed by Microsoft so that it can load on a UEFI computer that has Secure Boot enabled. So for Ubuntu, Canonical provides a version of shim signed by Microsoft that trusts binaries signed by Canonical. This signed shim can then load a version of grub that has been signed by Canonical, and grub will load a signed kernel, and so forth. The trick with shim is that it needs to know where to find grub. It attempts to auto-detect how it was loaded, and looks for a file called grubx64.efi next to itself. This works great when it was loaded from a hard drive or SSD. If it was loaded through netboot, it will try to load grubx64.efi from the same TFTP server and directory from which it was loaded, which it determines by looking at the original information provided by the DHCP server. Unfortunately, shim doesn’t currently detect this information properly when it is booted from a PXE proxy server. UEFI provides the information necessary to figure it out, but shim doesn’t currently look for it.

You could be like me and say, “who cares about Secure Boot?” Instead, just disable Secure Boot on your computers, and boot directly to grub instead of using shim. The problem is that grub has the exact same issue, and it needs to know how it was booted in order to figure out where to get grub.cfg. There is probably a way to work around this problem by embedding an intermediate grub.cfg into your grub binary (using grub-mkstandalone) that knows where to look for the real grub.cfg, but you won’t be using shim so you won’t support clients with Secure Boot enabled.

The easiest thing to do is just not use proxy mode for now if you want to support UEFI clients, especially UEFI clients with Secure Boot enabled. I will still explain how to set up proxy mode at the end of this post if you’re interested, but just know that if you do it, you will need to spend a bit more time figuring out how to get grub to grab its config file and any other supporting files over TFTP when netbooting a UEFI computer. I didn’t bother to figure this out, though, because…

There is actually a third option that may work OK for you if you were hoping you could use a proxy server: use dnsmasq as a normal DHCP server alongside your existing DHCP server, but tell it to assign IPs outside of the range that your existing DHCP server provides, and also tell it to ignore non-PXE clients. That way, it will allow your other server to handle all normal DHCP leases, but it will still be able to respond with the boot info when it sees a PXE client. Your other DHCP server will also respond to the PXE client’s boot request, but I believe the PXE client will ignore your other server because it doesn’t provide any PXE boot info. It works OK in my testing, anyway!

You may be thinking, “Doug, you’re crazy! Two DHCP servers on the same subnet?” I know it sounds weird, but it’s safe to have two DHCP servers on the same network as long as they don’t assign any overlapping IP addresses. Some people even recommend using such a setup intentionally for redundancy. For example, if your existing DHCP server assigns addresses from 192.168.1.100 to 192.168.1.199, you could safely tell your new DHCP server to assign addresses from 192.168.1.200 to 192.168.1.249 with no conflicts at all–normal DHCP clients will use whichever server responds first. It would definitely be wise to ensure your new DHCP server assigns the same default gateway and DNS servers that the existing server assigns, but if you’re configuring it to only respond to PXE clients, it’s probably not a huge deal because the PXE boot process probably isn’t going to need them on a simple home network.

Do initial setup

Regardless of what type of DHCP server you’re going to run, there is a bunch of common setup to do. Let’s get started!

TFTP directory

First of all, let’s create a TFTP directory on your server, and change it so it’s owned by you (replace “doug” with your username):

sudo mkdir /tftpboot
sudo chown doug:doug /tftpboot

Download and extract the live install ISO

Now, we need to download the install ISO. You can store it wherever you want, but the location you choose won’t end up mattering; we’re going to extract the contents of this ISO into a directory on your computer. Start by downloading the Ubuntu .iso that you want to make bootable. In my case, I downloaded Ubuntu MATE 16.04.2 LTS:

wget http://cdimage.ubuntu.com/ubuntu-mate/releases/16.04.2/release/ubuntu-mate-16.04.2-desktop-amd64.iso

After grabbing the ISO, you should mount it loopback so you can extract files from it:

mkdir /tmp/iso
sudo mount -oloop,ro ubuntu-mate-16.04.2-desktop-amd64.iso /tmp/iso

Now, we need to do several things. First of all, we need to put the kernel and initrd into your TFTP directory so TFTP clients will be able to access it. I like to make a directory in the TFTP server for each ISO I’m hosting:

mkdir /tftpboot/ubuntu-mate-16.04.2-desktop-amd64
cp /tmp/iso/casper/{vmlinuz.efi,initrd.lz} /tftpboot/ubuntu-mate-16.04.2-desktop-amd64/

Note that even though the kernel is named with a .efi extension, it will still boot a normal BIOS system just fine. The bootloader will know how to handle the file either way.

Now, you need to save a copy of all of the contents of the ISO somewhere locally that you can serve with NFS. I prefer to make a directory on the root of my drive for each ISO, but do what you want. Afterward, you can unmount the ISO.

sudo mkdir /ubuntu-mate-16.04.2-desktop-amd64
sudo cp -R /tmp/iso/* /tmp/iso/.disk /ubuntu-mate-16.04.2-desktop-amd64/
sudo umount /tmp/iso

Set up an NFS server

Set up your system as an NFS server so the PXE clients will be able to load the contents of the ISO that you extracted:

sudo apt-get install nfs-kernel-server

Add the following line to /etc/exports (I’m unsure which of the options chosen here are absolutely necessary, but they worked for me):

/ubuntu-mate-16.04.2-desktop-amd64           *(ro,sync,no_wdelay,insecure_locks,no_root_squash,insecure,no_subtree_check)

And finally, restart the NFS server so it recognizes your changes:

sudo service nfs-kernel-server restart

At this point, you are serving the contents of the ISO inside the directory /ubuntu-mate-16.04.2-desktop-amd64 with NFS, and you no longer need the .iso file. Note that some people might prefer to just keep the .iso file mounted loopback at all times instead of extracting the files from it. If you prefer to do it that way, you can probably set something up in /etc/fstab to automatically mount it, and then set up /etc/exports to share the mounted directory instead. Whatever you prefer!

Now, we need to set up some bootloaders that the PXE clients will load. We will use PXELINUX for BIOS clients and shim/grub for UEFI clients. Later on, we will set up dnsmasq to decide which bootloader to serve based on how the client identifies itself.

PXELINUX

For legacy BIOS clients, we are going to use PXELINUX. This is a very popular PXE bootloader based on SYSLINUX. It only supports BIOS clients, so this will only handle the BIOS half of things.

Start out by downloading SYSLINUX, which includes PXELINUX. Extract pxelinux.0 and a few libraries from it and place them into the TFTP server directory:

wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.xz
tar -xJf syslinux-6.03.tar.xz
cp syslinux-6.03/bios/core/pxelinux.0 /tftpboot/
cp syslinux-6.03/bios/com32/lib/libcom32.c32 /tftpboot/
cp syslinux-6.03/bios/com32/libutil/libutil.c32 /tftpboot/
cp syslinux-6.03/bios/com32/elflink/ldlinux/ldlinux.c32 /tftpboot/
cp syslinux-6.03/bios/com32/menu/vesamenu.c32 /tftpboot/

pxelinux.0 will be the file that is sent to BIOS PXE boot clients. After loading, it will be responsible for grabbing the other library files (ending in .c32) from the TFTP server and then it will display a list of menu choices, which we will define in a config file. Create the directory that will contain the config file:

mkdir /tftpboot/pxelinux.cfg

Now, create the file /tftpboot/pxelinux.cfg/default, which will contain the commands needed to boot into the live environment from the NFS share of the Ubuntu ISO contents. In the example below, your NFS server is 192.168.1.2:

DEFAULT vesamenu.c32
MENU TITLE Network boot

LABEL ubuntu-mate-16.04.2-desktop-amd64
  MENU LABEL ubuntu-mate-16.04.2-desktop-amd64
  KERNEL ubuntu-mate-16.04.2-desktop-amd64/vmlinuz.efi
  APPEND initrd=ubuntu-mate-16.04.2-desktop-amd64/initrd.lz root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.1.2:/ubuntu-mate-16.04.2-desktop-amd64 splash -- 

You can add as many “LABEL, MENU LABEL, KERNEL, APPEND” groups as you want to provide multiple selectable options.

shim and grub

For modern UEFI clients, we can’t use PXELINUX. Apparently, SYSLINUX also supports UEFI netbooting, so you could use it, but I read that the current version as of this writing (6.03) is very buggy with UEFI PXE booting. Instead, let’s use grub. Using grub will also allow you to use shim and support Secure Boot if you care about that.

Download Ubuntu’s version of shim-signed, and copy it into your TFTP directory. There are many ways to obtain it, such as downloading the dpkg file and extracting shim.efi.signed out of it; here is an easier way that works as of this writing:

wget https://launchpad.net/ubuntu/+archive/primary/+files/shim-signed_1.19~16.04.1.tar.xz
tar -xJf shim-signed_1.19~16.04.1.tar.xz
cp shim-signed-1.18~16.04.1/shim.efi.signed /tftpboot/shim.efi

Do the same with a signed version of grub, which Ubuntu’s signed shim will be able to load in a Secure Boot environment:

wget http://archive.ubuntu.com/ubuntu/dists/xenial/main/uefi/grub2-amd64/current/grubnetx64.efi.signed
mv grubnetx64.efi.signed /tftpboot/grubx64.efi

Create the directory that will contain grub’s config file:

mkdir /tftpboot/grub

And finally, create the file /tftpboot/grub/grub.cfg, which will contain the commands needed for booting into the Ubuntu live environment. Like before, the NFS server in this example is 192.168.1.2.

menuentry "ubuntu-mate-16.04.2-desktop-amd64" {
  linux ubuntu-mate-16.04.2-desktop-amd64/vmlinuz.efi root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.1.2:/ubuntu-mate-16.04.2-desktop-amd64 splash -- 
  initrd ubuntu-mate-16.04.2-desktop-amd64/initrd.lz
}

You can add as many “menuentry” items as you want to provide multiple selectable options.

Set up dnsmasq

We are now finished setting up the TFTP directory. All bootloaders and config files are in place. Now, we just need to set up the DHCP/TFTP server and tell it to boot BIOS clients using pxelinux.0, and UEFI clients using shim.efi. First, install dnsmasq 2.76 or greater using whatever method is most convenient for you. Based on which strategy you chose earlier, go to the subsection that matches the setup you want, and set up your dnsmasq.conf based on the given example.

Replace your existing DHCP server

Here is a (mostly) complete dnsmasq.conf file that sets up a full DHCP server. Note you will probably need to add additional options for the default gateway and DNS servers, but I’ve only supplied the parts of the config file necessary for PXE booting to work properly. Find a normal dnsmasq tutorial for info on the options to add for fully setting up a DHCP server. In the file below, it’s assumed that the server running dnsmasq is 192.168.1.2 and it is assigning IP addresses in the range of 192.168.1.50 to 192.168.1.99. Also, note that the configuration includes support for EFI32, which is a pretty uncommon architecture and will probably require a different build of grub and shim, if anyone has even made such a thing. You probably won’t have to worry about EFI32, so you can completely remove the lines referring to it if you want.

Another interesting tidbit is that this configuration doesn’t actually use PXE; it just uses the normal TFTP server and filename fields that are a part of DHCP. All PXE clients I’ve ever tested (both BIOS and UEFI) support this without any problems though. The reason I did it this way is because I discovered during testing that UEFI clients don’t seem to work properly when you actually use PXE with this setup. I never figured out why, but since it works to just set the DHCP boot server and filename, there’s no need for PXE.

# Don't function as a DNS server
port=0

# Log lots of extra information about DHCP transactions
log-dhcp

# Enable the built-in TFTP server
enable-tftp

# Set the root directory for files available via TFTP
tftp-root=/tftpboot

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override

# Inspect the vendor class string and match the text to set the tag
dhcp-vendorclass=BIOS,PXEClient:Arch:00000
dhcp-vendorclass=UEFI32,PXEClient:Arch:00006
dhcp-vendorclass=UEFI,PXEClient:Arch:00007
dhcp-vendorclass=UEFI64,PXEClient:Arch:00009

# Set the boot filename based on the matching tag from the vendor class (above)
dhcp-boot=pxelinux.0,,192.168.1.2
dhcp-boot=net:UEFI32,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI64,shim.efi,,192.168.1.2

# Give out IPs from 192.168.1.50 to 192.168.1.99
dhcp-range=192.168.1.50,192.168.1.99

If for some reason you’re directly booting into grub instead of shim, replace all instances of shim.efi with grubx64.efi instead.  This won’t work with Secure Boot enabled, so I’d definitely recommend using shim for maximum compatibility.

Set up a proxy DHCP server

This config file sets you up as a PXE proxy server. Like the above example, it is assumed that the dnsmasq server’s IP address is 192.168.1.2. I am also assuming that the other DHCP server is 192.168.1.1. Unlike the above example though, this example does actually use PXE, and it works properly with both BIOS and UEFI. Unfortunately, as I explained earlier in this post, shim and grub won’t play nicely because they won’t be able to automatically detect that they were loaded from a PXE proxy. Because of that problem, this configuration is mostly useless if you need to support UEFI. I’m just providing it for reference, especially to show the difference between setting up normal DHCP TFTP server/filename booting and actual PXE booting. If shim and grub are ever fixed to support PXE proxies properly in the future, this configuration will be much more useful.

The “Test PXE” and “Testing PXE” text can be set to whatever you want.

# Don't function as a DNS server
port=0

# Log lots of extra information about DHCP transactions
log-dhcp

# Enable the built-in TFTP server
enable-tftp

# Set the root directory for files available via TFTP
tftp-root=/tftpboot

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override

# Set the boot file and server IP based on the architecture
pxe-service=x86PC, "Test PXE", pxelinux.0, 192.168.1.2
pxe-service=BC_EFI, "Test PXE", shim.efi, 192.168.1.2
pxe-service=X86-64_EFI, "Test PXE", shim.efi, 192.168.1.2

# PXE menu. The first part is the text displayed to the user. The second is
# the timeout, in seconds.
pxe-prompt="Testing PXE", 1

# We are proxying for the DHCP server 192.168.1.1
dhcp-range=192.168.1.1,proxy

Add a second DHCP server with a different IP range (and ignore non-PXE clients)

This method should allow you to preserve your existing DHCP server while still supporting PXE. Because of the problems with shim and grub mentioned above, I’d personally recommend using this method instead of the proxy method if you don’t want to mess with your existing DHCP server. Use the same configuration from the “Replace your existing DHCP server” section, but add the following lines to the config file, just after the dhcp-no-override line:

# Set a tag if it's a PXE client
dhcp-match=set:IsPXEClient,60,"PXEClient"
# Ignore non-PXE requests, to allow the existing DHCP server to handle them instead.
dhcp-ignore=tag:!IsPXEClient

And of course, update the dhcp-range setting to make sure it doesn’t overlap with your existing DHCP server’s IP address range.

As described earlier, the dnsmasq server in this configuration will ignore non-PXE clients, so your existing DHCP server will be fully in charge of assigning IP addresses to your computers and devices, with the exception of PXE boot attempts. Both your existing server and the dnsmasq server will respond to PXE requests, but your server should “win” because the existing DHCP server won’t respond to the PXE request with any of the required PXE information. I can’t guarantee this will work with every PXE client, but it seems to work fine with every computer I’ve tried to netboot.

Run dnsmasq and try it out

At this point, you have set up dnsmasq using one of the three strategies listed above. Now it’s time to try it out! Ensure that dnsmasq will start up and run. While testing, I like to leave the service stopped and instead run it manually with the -d flag so I can see all of the debug output:

sudo service dnsmasq stop
sudo dnsmasq -d

Now, tell a computer on your network to boot! Usually your computer has a boot menu that lets you pick various boot options. Sometimes the F12 key works, sometimes the Esc key works. Or, try making a VMware virtual machine and booting it without a hard drive. In many computers, you have to enable the PXE boot ROM or the “Networking Stack” in order to make it work. There’s almost always an option somewhere! Remember, this only works over Ethernet; if you have built-in Wi-Fi, it’s probably not going to work unless you have a really fancy BIOS that knows how to connect to a Wi-Fi network — I’ve never heard of such a thing.

If everything works OK, you should end up at a menu screen served by PXELINUX on a BIOS computer or grub on a UEFI computer. Pick the Ubuntu option you added to the config file earlier, and press enter to begin booting!

Sometimes, downloading the kernel and initrd can be slow. If you see a black screen and nothing’s happening, don’t panic. Open up Wireshark and see if anything’s happening, and be patient. In some cases, such as a VMware VM set up for EFI, grub takes forever to load the kernel and initrd over TFTP and sits with a black screen for a long time. dnsmasq doesn’t actually print out anything about a TFTP file transfer in progress; it only prints out a message after the file transfer succeeds. At some point, Ubuntu will load and attempt to mount the ISO contents over NFS.

Once I know that everything is working OK, then I kill the debug dnsmasq with ctrl-C and run it as a service instead (assuming I have it installed that way):

sudo service dnsmasq start

You might want to remove some of the debug options from the dnsmasq config file such as log-dhcp once you know everything is working OK, but keep in mind if you ever need to do any troubleshooting, it will be useful to turn it back on.

After installing, fix /etc/network/interfaces

When you install Ubuntu from an NFS mount, it does something weird: it sets up /etc/network/interfaces to manage your Ethernet card instead of allowing NetworkManager to handle it. It’s really easy to fix though. After booting into your new Ubuntu install, edit /etc/network/interfaces and remove the lines relating to your Ethernet card. The only two non-comment lines left when you’re finished should be:

auto lo
iface lo inet loopback

This is just a minor annoyance that is easy to solve. It’s probably possible to customize the Ubuntu install scripts to fix this automatically, but I spent way too much time figuring out how to netboot. I don’t feel like solving that tiny problem now. Post a comment on the blog if you figure out how to do it and I’ll add instructions to this post.

All done!

Not too bad, right? I hope this post can serve as a reference tool for sysadmins, developers, power users, etc. who want to netboot install modern Ubuntu systems. Maybe I didn’t search hard enough, but I couldn’t find any reference online with all of this information in one place. Let me know if you have any suggestions on ways I can improve these instructions, or if things change as time goes on! Maybe someday, shim and grub will support PXE proxy servers and I won’t have to tell people to steer clear of that option.

Special thanks

It wouldn’t be right if I didn’t give credit to the following sites that helped me figure out how to make this work and also helped me understand why things weren’t working properly: