Recently I had an idea: what if I maintained a local server to host my Ubuntu install ISOs so I could load them through the PXE boot capability available on all of my computers? I would never need to create an Ubuntu install USB drive ever again — instead, I could just tell the BIOS or UEFI to boot through the network card, pick the ISO I want from a list, and boot right into it. Every x86/amd64 computer I’ve ever used (with the exception of Macs) has had an option in the BIOS/UEFI for booting through the Ethernet port, but it’s something I’ve never been able to try, and tutorials on it are usually old and/or incomplete. I finally decided to try to figure it out. I got it working after a lot of trial and error.

I found plenty of tutorials, but none of them had everything that I needed to know in order to get this setup working. Most of the tutorials that were Ubuntu-specific applied to really old releases (10.04 and below), and generally didn’t support both UEFI and BIOS simultaneously. I can verify that the process I figured out will work for both legacy BIOS and modern UEFI systems, and it will properly boot into an Ubuntu 16.04 desktop live ISO. Specifically, I have tested it with an Ubuntu MATE live ISO, but it should work with all other variants too.

dnsmasq

These instructions will use dnsmasq, which is a nice little DNS/DHCP/TFTP server program that you can install in Linux. If you are using something like DD-WRT, there is a good chance you already have dnsmasq! However, for these instructions it’s apparently important that you use the latest version as of this writing, which is 2.76. I didn’t test with earlier versions, but according to sources online, version 2.76 contains bug fixes for PXE booting with UEFI. So to start, you need to make sure you have dnsmasq 2.76. If your server is an Ubuntu 16.10 server, you can simply install dnsmasq with apt–it’s already the latest version. Otherwise, you may need to manually grab the .deb from 16.10, or compile it yourself. Keep in mind that desktop versions of Ubuntu run dnsmasq in the background for DNS, so you may have to set up something special if you want to run this server from a desktop install of Ubuntu without interfering with the existing dnsmasq. I’m not 100% sure–I did this on a server install so that wasn’t an issue.

If you already have a different DHCP server program, you can probably adapt these instructions to work with it, but you’re on your own to figure out the required syntax.

Proxy DHCP or not?

There is a decision you will have to make which will affect how you configure dnsmasq. Do you want to run dnsmasq as a normal DHCP server, or a proxy DHCP server? Let me explain the choices. As a forewarning, this may take a while to fully explain, but it’s all useful information.

When I say “normal DHCP server”, what I mean is that dnsmasq will replace your existing DHCP server. So you would set up dnsmasq to assign IP addresses to all of your computers and devices on your network, and turn off the DHCP server built into your home router. This new DHCP server provided by dnsmasq will recognize when your computer is trying to netboot, and will tell it to use TFTP to download a file from a server to boot. As you will see shortly, this is probably your best option. It’s understandable, though, if you don’t want to mess around with the DHCP server that you already have which is working just fine with no problems whatsoever. If you’re in that camp, there is another option available: a proxy DHCP server.

If you already have a DHCP server on your network that you can’t configure for PXE but you still want to use (e.g. the DHCP server in your home router), you can set up dnsmasq as a proxy DHCP server that works alongside your existing DHCP server. Your existing DHCP server will still be responsible for assigning IP addresses to all of your computers and devices, but the proxy DHCP server will provide the necessary information that PXE clients need for booting (the TFTP server IP address and file path). You will not have to change any configuration on your existing DHCP server; the proxy just provides additional information that all PXE-enabled BIOS/UEFI implementations are supposed to support. PXE was designed to support this use case, and every computer I’ve tested so far works with it just fine.

With both of these options available, the choice may seem obvious to you: use it as a proxy DHCP server! It’s a low-risk choice because you’re not messing with the configuration of anything else already on your network. In theory, this would be a great idea. In practice, though, there are problems with booting in proxy mode with UEFI. It’s not actually dnsmasq’s fault at all. It’s not the PXE protocol’s fault, and it’s not even the fault of your computer’s UEFI implementation. The fault lies in shim and grub. Let me explain further.

shim is a bootloader that serves a single basic purpose: to load grub. The reason it exists is because it’s possible to have it signed by Microsoft so that it can load on a UEFI computer that has Secure Boot enabled. So for Ubuntu, Canonical provides a version of shim signed by Microsoft that trusts binaries signed by Canonical. This signed shim can then load a version of grub that has been signed by Canonical, and grub will load a signed kernel, and so forth. The trick with shim is that it needs to know where to find grub. It attempts to auto-detect how it was loaded, and looks for a file called grubx64.efi next to itself. This works great when it was loaded from a hard drive or SSD. If it was loaded through netboot, it will try to load grubx64.efi from the same TFTP server and directory from which it was loaded, which it determines by looking at the original information provided by the DHCP server. Unfortunately, shim doesn’t currently detect this information properly when it is booted from a PXE proxy server. UEFI provides the information necessary to figure it out, but shim doesn’t currently look for it.

You could be like me and say, “who cares about Secure Boot?” Instead, just disable Secure Boot on your computers, and boot directly to grub instead of using shim. The problem is that grub has the exact same issue, and it needs to know how it was booted in order to figure out where to get grub.cfg. There is probably a way to work around this problem by embedding an intermediate grub.cfg into your grub binary (using grub-mkstandalone) that knows where to look for the real grub.cfg, but you won’t be using shim so you won’t support clients with Secure Boot enabled.

The easiest thing to do is just not use proxy mode for now if you want to support UEFI clients, especially UEFI clients with Secure Boot enabled. I will still explain how to set up proxy mode at the end of this post if you’re interested, but just know that if you do it, you will need to spend a bit more time figuring out how to get grub to grab its config file and any other supporting files over TFTP when netbooting a UEFI computer. I didn’t bother to figure this out, though, because…

There is actually a third option that may work OK for you if you were hoping you could use a proxy server: use dnsmasq as a normal DHCP server alongside your existing DHCP server, but tell it to assign IPs outside of the range that your existing DHCP server provides, and also tell it to ignore non-PXE clients. That way, it will allow your other server to handle all normal DHCP leases, but it will still be able to respond with the boot info when it sees a PXE client. Your other DHCP server will also respond to the PXE client’s boot request, but I believe the PXE client will ignore your other server because it doesn’t provide any PXE boot info. It works OK in my testing, anyway!

You may be thinking, “Doug, you’re crazy! Two DHCP servers on the same subnet?” I know it sounds weird, but it’s safe to have two DHCP servers on the same network as long as they don’t assign any overlapping IP addresses. Some people even recommend using such a setup intentionally for redundancy. For example, if your existing DHCP server assigns addresses from 192.168.1.100 to 192.168.1.199, you could safely tell your new DHCP server to assign addresses from 192.168.1.200 to 192.168.1.249 with no conflicts at all–normal DHCP clients will use whichever server responds first. It would definitely be wise to ensure your new DHCP server assigns the same default gateway and DNS servers that the existing server assigns, but if you’re configuring it to only respond to PXE clients, it’s probably not a huge deal because the PXE boot process probably isn’t going to need them on a simple home network.

Do initial setup

Regardless of what type of DHCP server you’re going to run, there is a bunch of common setup to do. Let’s get started!

TFTP directory

First of all, let’s create a TFTP directory on your server, and change it so it’s owned by you (replace “doug” with your username):

sudo mkdir /tftpboot
sudo chown doug:doug /tftpboot

Download and extract the live install ISO

Now, we need to download the install ISO. You can store it wherever you want, but the location you choose won’t end up mattering; we’re going to extract the contents of this ISO into a directory on your computer. Start by downloading the Ubuntu .iso that you want to make bootable. In my case, I downloaded Ubuntu MATE 16.04.2 LTS:

wget http://cdimage.ubuntu.com/ubuntu-mate/releases/16.04.2/release/ubuntu-mate-16.04.2-desktop-amd64.iso

After grabbing the ISO, you should mount it loopback so you can extract files from it:

mkdir /tmp/iso
sudo mount -oloop,ro ubuntu-mate-16.04.2-desktop-amd64.iso /tmp/iso

Now, we need to do several things. First of all, we need to put the kernel and initrd into your TFTP directory so TFTP clients will be able to access it. I like to make a directory in the TFTP server for each ISO I’m hosting:

mkdir /tftpboot/ubuntu-mate-16.04.2-desktop-amd64
cp /tmp/iso/casper/{vmlinuz.efi,initrd.lz} /tftpboot/ubuntu-mate-16.04.2-desktop-amd64/

Note that even though the kernel is named with a .efi extension, it will still boot a normal BIOS system just fine. The bootloader will know how to handle the file either way.

Now, you need to save a copy of all of the contents of the ISO somewhere locally that you can serve with NFS. I prefer to make a directory on the root of my drive for each ISO, but do what you want. Afterward, you can unmount the ISO.

sudo mkdir /ubuntu-mate-16.04.2-desktop-amd64
sudo cp -R /tmp/iso/* /tmp/iso/.disk /ubuntu-mate-16.04.2-desktop-amd64/
sudo umount /tmp/iso

Set up an NFS server

Set up your system as an NFS server so the PXE clients will be able to load the contents of the ISO that you extracted:

sudo apt-get install nfs-kernel-server

Add the following line to /etc/exports (I’m unsure which of the options chosen here are absolutely necessary, but they worked for me):

/ubuntu-mate-16.04.2-desktop-amd64           *(ro,sync,no_wdelay,insecure_locks,no_root_squash,insecure,no_subtree_check)

And finally, restart the NFS server so it recognizes your changes:

sudo service nfs-kernel-server restart

At this point, you are serving the contents of the ISO inside the directory /ubuntu-mate-16.04.2-desktop-amd64 with NFS, and you no longer need the .iso file. Note that some people might prefer to just keep the .iso file mounted loopback at all times instead of extracting the files from it. If you prefer to do it that way, you can probably set something up in /etc/fstab to automatically mount it, and then set up /etc/exports to share the mounted directory instead. Whatever you prefer!

Now, we need to set up some bootloaders that the PXE clients will load. We will use PXELINUX for BIOS clients and shim/grub for UEFI clients. Later on, we will set up dnsmasq to decide which bootloader to serve based on how the client identifies itself.

PXELINUX

For legacy BIOS clients, we are going to use PXELINUX. This is a very popular PXE bootloader based on SYSLINUX. It only supports BIOS clients, so this will only handle the BIOS half of things.

Start out by downloading SYSLINUX, which includes PXELINUX. Extract pxelinux.0 and a few libraries from it and place them into the TFTP server directory:

wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.xz
tar -xJf syslinux-6.03.tar.xz
cp syslinux-6.03/bios/core/pxelinux.0 /tftpboot/
cp syslinux-6.03/bios/com32/lib/libcom32.c32 /tftpboot/
cp syslinux-6.03/bios/com32/libutil/libutil.c32 /tftpboot/
cp syslinux-6.03/bios/com32/elflink/ldlinux/ldlinux.c32 /tftpboot/
cp syslinux-6.03/bios/com32/menu/vesamenu.c32 /tftpboot/

pxelinux.0 will be the file that is sent to BIOS PXE boot clients. After loading, it will be responsible for grabbing the other library files (ending in .c32) from the TFTP server and then it will display a list of menu choices, which we will define in a config file. Create the directory that will contain the config file:

mkdir /tftpboot/pxelinux.cfg

Now, create the file /tftpboot/pxelinux.cfg/default, which will contain the commands needed to boot into the live environment from the NFS share of the Ubuntu ISO contents. In the example below, your NFS server is 192.168.1.2:

DEFAULT vesamenu.c32
MENU TITLE Network boot

LABEL ubuntu-mate-16.04.2-desktop-amd64
  MENU LABEL ubuntu-mate-16.04.2-desktop-amd64
  KERNEL ubuntu-mate-16.04.2-desktop-amd64/vmlinuz.efi
  APPEND initrd=ubuntu-mate-16.04.2-desktop-amd64/initrd.lz root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.1.2:/ubuntu-mate-16.04.2-desktop-amd64 splash -- 

You can add as many “LABEL, MENU LABEL, KERNEL, APPEND” groups as you want to provide multiple selectable options.

shim and grub

For modern UEFI clients, we can’t use PXELINUX. Apparently, SYSLINUX also supports UEFI netbooting, so you could use it, but I read that the current version as of this writing (6.03) is very buggy with UEFI PXE booting. Instead, let’s use grub. Using grub will also allow you to use shim and support Secure Boot if you care about that.

Download Ubuntu’s version of shim-signed, and copy it into your TFTP directory. There are many ways to obtain it, such as downloading the dpkg file and extracting shim.efi.signed out of it; here is an easier way that works as of this writing:

wget https://launchpad.net/ubuntu/+archive/primary/+files/shim-signed_1.19~16.04.1.tar.xz
tar -xJf shim-signed_1.19~16.04.1.tar.xz
cp shim-signed-1.18~16.04.1/shim.efi.signed /tftpboot/shim.efi

Do the same with a signed version of grub, which Ubuntu’s signed shim will be able to load in a Secure Boot environment:

wget http://archive.ubuntu.com/ubuntu/dists/xenial/main/uefi/grub2-amd64/current/grubnetx64.efi.signed
mv grubnetx64.efi.signed /tftpboot/grubx64.efi

Create the directory that will contain grub’s config file:

mkdir /tftpboot/grub

And finally, create the file /tftpboot/grub/grub.cfg, which will contain the commands needed for booting into the Ubuntu live environment. Like before, the NFS server in this example is 192.168.1.2.

menuentry "ubuntu-mate-16.04.2-desktop-amd64" {
  linux ubuntu-mate-16.04.2-desktop-amd64/vmlinuz.efi root=/dev/nfs boot=casper netboot=nfs nfsroot=192.168.1.2:/ubuntu-mate-16.04.2-desktop-amd64 splash -- 
  initrd ubuntu-mate-16.04.2-desktop-amd64/initrd.lz
}

You can add as many “menuentry” items as you want to provide multiple selectable options.

Set up dnsmasq

We are now finished setting up the TFTP directory. All bootloaders and config files are in place. Now, we just need to set up the DHCP/TFTP server and tell it to boot BIOS clients using pxelinux.0, and UEFI clients using shim.efi. First, install dnsmasq 2.76 or greater using whatever method is most convenient for you. Based on which strategy you chose earlier, go to the subsection that matches the setup you want, and set up your dnsmasq.conf based on the given example.

Replace your existing DHCP server

Here is a (mostly) complete dnsmasq.conf file that sets up a full DHCP server. Note you will probably need to add additional options for the default gateway and DNS servers, but I’ve only supplied the parts of the config file necessary for PXE booting to work properly. Find a normal dnsmasq tutorial for info on the options to add for fully setting up a DHCP server. In the file below, it’s assumed that the server running dnsmasq is 192.168.1.2 and it is assigning IP addresses in the range of 192.168.1.50 to 192.168.1.99. Also, note that the configuration includes support for EFI32, which is a pretty uncommon architecture and will probably require a different build of grub and shim, if anyone has even made such a thing. You probably won’t have to worry about EFI32, so you can completely remove the lines referring to it if you want.

Another interesting tidbit is that this configuration doesn’t actually use PXE; it just uses the normal TFTP server and filename fields that are a part of DHCP. All PXE clients I’ve ever tested (both BIOS and UEFI) support this without any problems though. The reason I did it this way is because I discovered during testing that UEFI clients don’t seem to work properly when you actually use PXE with this setup. I never figured out why, but since it works to just set the DHCP boot server and filename, there’s no need for PXE.

# Don't function as a DNS server
port=0

# Log lots of extra information about DHCP transactions
log-dhcp

# Enable the built-in TFTP server
enable-tftp

# Set the root directory for files available via TFTP
tftp-root=/tftpboot

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override

# Inspect the vendor class string and match the text to set the tag
dhcp-vendorclass=BIOS,PXEClient:Arch:00000
dhcp-vendorclass=UEFI32,PXEClient:Arch:00006
dhcp-vendorclass=UEFI,PXEClient:Arch:00007
dhcp-vendorclass=UEFI64,PXEClient:Arch:00009

# Set the boot filename based on the matching tag from the vendor class (above)
dhcp-boot=pxelinux.0,,192.168.1.2
dhcp-boot=net:UEFI32,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI,shim.efi,,192.168.1.2
dhcp-boot=net:UEFI64,shim.efi,,192.168.1.2

# Give out IPs from 192.168.1.50 to 192.168.1.99
dhcp-range=192.168.1.50,192.168.1.99

If for some reason you’re directly booting into grub instead of shim, replace all instances of shim.efi with grubx64.efi instead.  This won’t work with Secure Boot enabled, so I’d definitely recommend using shim for maximum compatibility.

Set up a proxy DHCP server

This config file sets you up as a PXE proxy server. Like the above example, it is assumed that the dnsmasq server’s IP address is 192.168.1.2. I am also assuming that the other DHCP server is 192.168.1.1. Unlike the above example though, this example does actually use PXE, and it works properly with both BIOS and UEFI. Unfortunately, as I explained earlier in this post, shim and grub won’t play nicely because they won’t be able to automatically detect that they were loaded from a PXE proxy. Because of that problem, this configuration is mostly useless if you need to support UEFI. I’m just providing it for reference, especially to show the difference between setting up normal DHCP TFTP server/filename booting and actual PXE booting. If shim and grub are ever fixed to support PXE proxies properly in the future, this configuration will be much more useful.

The “Test PXE” and “Testing PXE” text can be set to whatever you want.

# Don't function as a DNS server
port=0

# Log lots of extra information about DHCP transactions
log-dhcp

# Enable the built-in TFTP server
enable-tftp

# Set the root directory for files available via TFTP
tftp-root=/tftpboot

# Disable re-use of the DHCP servername and filename fields as extra
# option space. That's to avoid confusing some old or broken DHCP clients
dhcp-no-override

# Set the boot file and server IP based on the architecture
pxe-service=x86PC, "Test PXE", pxelinux.0, 192.168.1.2
pxe-service=BC_EFI, "Test PXE", shim.efi, 192.168.1.2
pxe-service=X86-64_EFI, "Test PXE", shim.efi, 192.168.1.2

# PXE menu. The first part is the text displayed to the user. The second is
# the timeout, in seconds.
pxe-prompt="Testing PXE", 1

# We are proxying for the DHCP server 192.168.1.1
dhcp-range=192.168.1.1,proxy

Add a second DHCP server with a different IP range (and ignore non-PXE clients)

This method should allow you to preserve your existing DHCP server while still supporting PXE. Because of the problems with shim and grub mentioned above, I’d personally recommend using this method instead of the proxy method if you don’t want to mess with your existing DHCP server. Use the same configuration from the “Replace your existing DHCP server” section, but add the following lines to the config file, just after the dhcp-no-override line:

# Set a tag if it's a PXE client
dhcp-match=set:IsPXEClient,60,"PXEClient"
# Ignore non-PXE requests, to allow the existing DHCP server to handle them instead.
dhcp-ignore=tag:!IsPXEClient

And of course, update the dhcp-range setting to make sure it doesn’t overlap with your existing DHCP server’s IP address range.

As described earlier, the dnsmasq server in this configuration will ignore non-PXE clients, so your existing DHCP server will be fully in charge of assigning IP addresses to your computers and devices, with the exception of PXE boot attempts. Both your existing server and the dnsmasq server will respond to PXE requests, but your server should “win” because the existing DHCP server won’t respond to the PXE request with any of the required PXE information. I can’t guarantee this will work with every PXE client, but it seems to work fine with every computer I’ve tried to netboot.

Run dnsmasq and try it out

At this point, you have set up dnsmasq using one of the three strategies listed above. Now it’s time to try it out! Ensure that dnsmasq will start up and run. While testing, I like to leave the service stopped and instead run it manually with the -d flag so I can see all of the debug output:

sudo service dnsmasq stop
sudo dnsmasq -d

Now, tell a computer on your network to boot! Usually your computer has a boot menu that lets you pick various boot options. Sometimes the F12 key works, sometimes the Esc key works. Or, try making a VMware virtual machine and booting it without a hard drive. In many computers, you have to enable the PXE boot ROM or the “Networking Stack” in order to make it work. There’s almost always an option somewhere! Remember, this only works over Ethernet; if you have built-in Wi-Fi, it’s probably not going to work unless you have a really fancy BIOS that knows how to connect to a Wi-Fi network — I’ve never heard of such a thing.

If everything works OK, you should end up at a menu screen served by PXELINUX on a BIOS computer or grub on a UEFI computer. Pick the Ubuntu option you added to the config file earlier, and press enter to begin booting!

Sometimes, downloading the kernel and initrd can be slow. If you see a black screen and nothing’s happening, don’t panic. Open up Wireshark and see if anything’s happening, and be patient. In some cases, such as a VMware VM set up for EFI, grub takes forever to load the kernel and initrd over TFTP and sits with a black screen for a long time. dnsmasq doesn’t actually print out anything about a TFTP file transfer in progress; it only prints out a message after the file transfer succeeds. At some point, Ubuntu will load and attempt to mount the ISO contents over NFS.

Once I know that everything is working OK, then I kill the debug dnsmasq with ctrl-C and run it as a service instead (assuming I have it installed that way):

sudo service dnsmasq start

You might want to remove some of the debug options from the dnsmasq config file such as log-dhcp once you know everything is working OK, but keep in mind if you ever need to do any troubleshooting, it will be useful to turn it back on.

After installing, fix /etc/network/interfaces

When you install Ubuntu from an NFS mount, it does something weird: it sets up /etc/network/interfaces to manage your Ethernet card instead of allowing NetworkManager to handle it. It’s really easy to fix though. After booting into your new Ubuntu install, edit /etc/network/interfaces and remove the lines relating to your Ethernet card. The only two non-comment lines should be:

auto lo
iface lo inet loopback

This is just a minor annoyance that is easy to solve. It’s probably possible to customize the Ubuntu install scripts to fix this automatically, but I spent way too much time figuring out how to netboot. I don’t feel like solving that tiny problem now. Post a comment on the blog if you figure out how to do it and I’ll add instructions to this post.

All done!

Not too bad, right? I hope this post can serve as a reference tool for sysadmins, developers, power users, etc. who want to netboot install modern Ubuntu systems. Maybe I didn’t search hard enough, but I couldn’t find any reference online with all of this information in one place. Let me know if you have any suggestions on ways I can improve these instructions, or if things change as time goes on! Maybe someday, shim and grub will support PXE proxy servers and I won’t have to tell people to steer clear of that option.

Special thanks

It wouldn’t be right if I didn’t give credit to the following sites that helped me figure out how to make this work and also helped me understand why things weren’t working properly:

As someone who works with a lot of development boards, most of which come with a USB-to-serial chip of some kind, I deal with an annoying problem pretty often in Linux (currently, Ubuntu 16.04). When I plug the development board into my computer, I’m blocked from doing anything with the USB serial port it creates. The first error I usually get is that permission was denied:

$ cat /dev/ttyUSB0
cat: /dev/ttyUSB0: Permission denied

This is a common problem which is easy to solve — the problem is that the port’s permissions and ownership don’t allow me to access it:

$ ls -l /dev/ttyUSB0
crw-rw---- 1 root dialout 188, 0 Oct 25 13:45 /dev/ttyUSB0

I usually just add myself to the dialout group after I install Ubuntu to get around this issue. But…this isn’t what this blog post is about. The problem is that after I fix this, log out, log back in, and plug the board back into my computer, I run into this next problem:

$ cat /dev/ttyUSB0
cat: /dev/ttyUSB0: Device or resource busy

What’s interesting about this problem is that if I wait for 15-30 seconds and try again, it works. The problem goes away on its own. I did some sleuthing with the “lsof” command and figured out why this happens. The reason is that ModemManager opens the port when you plug it in, checking to see if a modem is attached to the serial port. I personally find this really annoying.

Luckily, it’s pretty easy to fix with a udev rule. udev already comes with several default rules which prevent ModemManager from looking at some USB serial devices (see the files /lib/udev/rules.d/77-mm-usb-device-blacklist.rules and /lib/udev/rules.d/77-mm-usb-serial-adapters-greylist.rules). Following the pattern in these files, you can create your own blacklist rule which you can put in /etc/udev/rules.d. In this example, I’m going to prevent an imaginary USB-to-serial adapter with USB vendor ID 0x1234 and product ID 0x5678 from being checked by ModemManager.

Create the file /etc/udev/rules.d/99-sample-usb-blacklist.rules and enter the following content, all on a single line:

ACTION=="add", SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device", ATTRS{idVendor}=="1234", ATTRS{idProduct}=="5678", ENV{ID_MM_DEVICE_IGNORE}="1"

That’s all there is to it. This rule checks for a device in the subsystem “usb” with a DEVTYPE of “usb_device” and a matching USB PID/VID combination. If it finds a matching device, it sets an environment variable called ID_MM_DEVICE_IGNORE to 1, which will tell ModemManager to ignore it. Reload the udev rules with the following command:

sudo udevadm control --reload-rules

Now unplug and replug your development board (or USB-to-serial adapter or whatever it is) and you should notice that ModemManager didn’t interfere.

Luckily, this problem is not as common today as it used to be. Modern versions of ModemManager come with a default rule that “greylists” all USB devices with the FTDI vendor ID so they are only checked if you manually scan for modems–see the 77-mm-usb-serial-adapters-greylist.rules file I mentioned earlier. Most development boards that I’ve seen use FTDI chips, so that rule takes care of them. I still occasionally run into this problem with non-FTDI chips or older versions of Ubuntu, though, and the rule pattern I gave above should fix it.

Note that the ID_MM_DEVICE_IGNORE environment variable needs to be set on the USB device, and not the tty device. In other words, your rule needs to say SUBSYSTEM==”usb” and not SUBSYSTEM==”tty”. I have spent an embarrassing amount of time trying to troubleshoot a rule that didn’t work because of this requirement. If you already have another rule that sets custom permissions on the tty device, you need to create a separate rule to add the ID_MM_DEVICE_IGNORE environment variable to the actual USB device rather than the tty device.

Recently, my dad’s TracFone Huawei H110C cell phone (with over 900 minutes and 200 service days) was a victim of mistaken identity. It was loaned out to someone else, and their dog thought it was a chew toy.

BrokenFront

I was called to the rescue. The phone still worked, but the display was all screwed up so I couldn’t see anything. My first thought was to transfer the service (phone number, minutes, service days) to a new phone, but I discovered that you need to enter and read back codes on the old phone in order to transfer your service, which you obviously can’t do with a broken display. I also read on forums that it can be time-consuming to work with TracFone to recover your minutes from a broken/lost/stolen phone, so I didn’t really want to mess around with that.

I tore apart the old phone to try to see if it was possible to replace the display. I was able to remove the LCD display module, but I couldn’t find that particular display available for purchase anywhere online. It may have been custom-made for Huawei. A brand new H110C costs less than $10 at Wal-Mart, so I decided the easiest approach would be to buy a new H110C and remove its display, then plug the new display into the old, broken phone and hope that it would work long enough to get the minutes transferred off of it.

It worked! I was able to enter the codes needed to transfer the service to another new H110C, which I also bought at Wal-Mart. Rather than reassemble the first new phone and transfer the service to it, I decided it would make more sense to buy a second new phone because some components in the phone are glued together, and you have to sort of mangle a metal shield in the phone in order to get to the display. So this process required me to buy two new H110Cs at Wal-Mart for a total price of $19.76. Not bad! Other than setting off the alarm at Wal-Mart when I left because the employee who helped me forgot to do something to the phone boxes, buying the phones was a painless process.

I hope I can help someone else out who runs into this problem, so here’s a tutorial on removing/replacing the display in a Huawei H110C.

Get the phone out of the box

Remove the phone from the box. Leave the battery out (or if it’s already installed, remove it).

NewPhone1

 

Remove the six screws from the back

There are six screws on the back of the phone. Remove them. They are Torx T5 screws, so you will need an appropriate screwdriver. One of them (right center in the picture) is covered with a white sticker as proof that you haven’t opened it. We’re going to void our warranty here anyway, so no big deal.

Screws

 

Separate the two halves

Removing the screws will not automatically separate the plastic pieces that hold the phone together. Carefully slide an old credit card or similar tool in between the two halves to release the latches that hold it all together. Work your way around the entire phone until the two halves are separated, and then lift the top of the case away.

Split

Halves

 

Remove the button board

The buttons are on a small flexible white circuit board. It is glued (and/or taped?) to the metal shield underneath it. Carefully peel it up off the shield. There is a connector on the bottom of it that connects to the main phone circuit board below. The connector should easily disconnect as you peel the button board away. It’s difficult to not bend the board, but try to keep it as straight as possible because we will need to use it again when we turn on the old phone.

Peel

KeysRemoved

KeyBoard

 

Pull the bottom half of the phone out of the plastic case

At this point you should remove the main phone board and display from the bottom case. If you push it around enough it should come out. I struggled with it a little bit, but eventually I was able to snap the circuit board out.

RemovedFromCase

 

Remove the metal shield

In order to get to the connector that connects the display to the board, you will need to remove the metal shield. This shield is probably an EMI/RF shield blocking interference to/from the phone. There are four tabs that hold it in place–two on each side. Peel them back, and then pop the metal shield off. This may take some force, so be careful. I used a small set of needle-nose pliers. On the first phone I opened (the broken one), I mangled the shield pretty badly. I did a better job on the second one, but I wouldn’t count on necessarily putting it back together when you’re done unless you’re really desperate to only buy one new phone instead of two.

PeelBackClips

MetalCoverRemoved

 

Disconnect the display from the main board

The display is connected to the main board through a small gray connector attached to a ribbon cable. It is easy to remove. Gently pry it up (I used a small screwdriver).

DisconnectDisplay1

DisconnectDisplay2

 

Remove the display

Removing the display is pretty difficult because it is also glued/taped down. I couldn’t figure out exactly what they used, but it seemed to be some sort of really sticky tape. I carefully stuck a thin screwdriver between the phone board and the display, and very carefully pried it up a little bit at a time to break the bonding. Eventually it lifted off the board. Be very careful here, especially with the brand new phone, because you don’t want to damage the new display.

DisplayRemoved

 

Repeat this process on your broken phone

Do the same thing to remove the broken display from the old phone. As you can see, the old one was damaged by a dog bite.

BrokenDisplay

 

Put the new display into the old phone

Basically, just do these steps in reverse. Plug the new display and one of the button boards into the old phone’s main board. I left the metal shield off because I knew I would only have the old phone turned on for a few minutes. I also covered the bottom of the button board with Kapton tape to ensure it would stay insulated from any metal underneath, although I don’t know how important that was.

 

Write down everything about the old phone

You will need the MEID of the old phone during the transfer process. TracFone will probably already have your old phone’s MEID on file, but just in case, write it down or take a picture of it (it’s inside the battery compartment, on the back of the main circuit board).

 

Turn on the old phone

Put the old phone (with new display and button board) into a case, insert a battery, and turn it on. You can press the little buttons on the button board that correspond with the phone’s keys. Cross your fingers! Here’s what I saw after a second:

Success

 

Begin the transfer process

Now, go to TracFone’s website and start the process of transferring the old phone’s service to the second new (unopened) phone. The TracFone site will walk you through entering some codes on the old phone and typing the result codes into their webpage to determine the number of remaining minutes and service days.

While you have the old phone turned on, be sure to write down the old contacts and settings so you can put them into the new phone.

 

All done!

After that, I was instructed to dial a number on the new phone to activate it. Activation worked painlessly and the new phone worked great. At that point I was able to turn off the old phone and get rid of it. Remember, lithium-ion batteries are dangerous if they get damaged, so don’t put them in the garbage–take them somewhere that can dispose of them properly. In all honesty, the dog was probably lucky that he didn’t puncture the battery.

The tape/glue/whatever was pretty annoying, and the metal shield was a pain to remove, but other than that, this phone wasn’t very difficult to take apart. It was definitely worth the less-than-$20 price to buy a brand new phone and another new LCD donor phone in order to recover all those minutes and service days from the old phone. I hope this little tutorial helps someone out there!

 

I recently repaired a Macintosh IIx I bought on eBay. It wouldn’t power on at all–both the rear power button and the keyboard power button did nothing. I was already aware that this system needs fresh batteries in order to boot, and I had removed the old batteries and replaced them with new ones, which didn’t make a difference. The real culprit: all of the surface-mount aluminum electrolytic capacitors had leaked electrolyte all over the logic board. This is a very common problem in older computers. These capacitors have a nasty habit of leaking after about 20 years or so. The electrolyte ate away at traces and several traces near capacitors were broken. Several IC pins were corroded really bad as well and had lost contact to their solder pads.

It was quite a journey figuring out how to fix it. The power button didn’t do anything, so I didn’t have a clue where to look. Luckily, an important figure in the Classic Mac community, Gamba, created a schematic of the power circuit. Sadly, Gamba passed away, but his site is still operational. I think there is a small error in his schematic, so I went ahead and fixed it. The fixed schematic is depicted below. The only change is the positioning of J18-15 and TS1 on the right side:

macIIsch_fixed

I began looking at this schematic and realized I didn’t have a clue what was going on in any of it. All of those NAND gates, diodes, and transistors looked intimidating. I buckled down and did my best to read the schematic and figure out what every piece does. After lots of experimentation and research, I feel like I understand this circuit pretty well now. I’ve decided to write this post to help explain the circuit and maybe help people in the future who are trying to figure out why their II/IIx won’t start up. Note: I am not an electronics expert. I’m trying my best, but I could be wrong about some details. Corrections would be appreciated!

Note that the diagram’s component numbering is for the Mac II. The IIx is slightly different. I would provide more detail, except my IIx logic board is a prerelease one that has the same component numbering as the II. So my IIx’s logic board’s component numbering matched this diagram. Other IIx systems do not match.

Let’s get started!

Overview

Here’s a broad overview of the different pieces of the circuit:

Overview

The ADB portion of this circuit is unrelated to this discussion, so I will skip it. Don’t worry–I have included the keyboard power button as part of my description of the power on circuit.

When I first looked at this diagram, for some reason I thought the top half was for the keyboard and the bottom half was for the rear power button. I was totally wrong. The keyboard and rear power button share most of the same power on circuitry. Now, I’m going to cover each section in more detail.

Batteries

Batteries

The two batteries B1 and B2 (3.6V 1/2 AA lithium) are in series with each other. This means they combine together to create a 7.2V voltage across ground and the positive terminal of B1. The three diodes seem to serve two purposes:

  1. Drop the voltage a little bit
  2. Protect against reverse polarity (putting the batteries in backwards)

In particular, D5 and D6 take the 7.2V voltage from the batteries and drop it down to about 7V or so. Normally these diodes would have a higher voltage drop, but you will see that while sitting idle, there isn’t much current being drawn from them, so the diode voltage drop is pretty low. D4 drops the 3.6V voltage a little bit, but I think its main purpose is to protect against reverse polarity. If you put B1 and B2 in backwards, D4 and D5 will prevent negative voltages from making their way to any other part of the logic board.

The 1000 μF capacitor is (I believe) the capacitor on the battery replacement board, and it can be found on both the II and the IIx, despite what Gamba’s diagram says. These computers originally came with the two batteries soldered directly to the logic board. When they needed to be replaced, there was a little circuit board that a service provider would solder on to replace the original batteries. It had two battery holders and a 1000 μF capacitor. I don’t fully understand the purpose of the capacitor, but it might be to make life easier on the batteries when the power button is pressed.

So this portion of the circuit takes two batteries as input, and as output provides a 3.3V-ish voltage rail and a 7V-ish voltage rail. I say “-ish” because I measured anywhere from 6.8V to 7V when I was debugging this circuit. But it’s somewhere in that ballpark. The 3.3V voltage is for powering the PRAM and clock while the computer is turned off. The 7V voltage is for turning the power supply on when you press either of the computer’s power buttons.

If you’re troubleshooting this portion of the circuit, make sure the cathode of D6 has 7V or so, and the cathode of D4 has 3.3V or so. If not, there’s something wrong with some of the traces and/or components pictured above. Check continuity between everything in the diagram above, and if needed, test D4, D5, and D6.

Clock/PRAM

ClockPRAM

This portion of the circuit is responsible for powering the clock and PRAM. On most (all?) of the Macs that came after the II/IIx, this is the *only* thing the single 3.6V battery is used for, at least as far as I know.

Recall that I already explained that there is about 3.3V on the cathode of D4 coming from battery B2. The 4.7KΩ resistor R17 is limiting the amount of current that will be supplied to UB6, the clock/PRAM chip. So…when the computer is turned off, the battery B2 is powering UB6. D3 is preventing the battery voltage from going into the +5V rail (which will be 0V when the power supply is turned off).

When the power supply turns on, the +5V rail actually has 5V on it. D3 will allow the 5V to go through into UB6, so the +5V rail will power UB6. I believe the 4.7KΩ resistor (R17) makes it so that the 5V rail does most of the work powering UB6 instead of the battery when the power is on. D4 is also (I believe) preventing the circuit from sending +5V into the battery circuit when the computer is powered on.

This part of the circuit is pretty simple! I don’t think a problem in here would necessarily prevent the computer from turning on, but it might prevent settings from being remembered while the power is off.

Power On

PowerOn

OK, here is where things really get interesting and more complicated. This is also where it will become obvious that both batteries must be installed in order for the II/IIx to boot. Recall that the cathode of D6 has about 7V on it or so. It powers UB2 (which is a 74HC132 containing four NAND gates–three are used, one is unused). UB2 doesn’t draw much current, so that’s why the diodes mentioned in the battery portion of the circuit don’t have much of a voltage drop. Without the batteries, these NAND gates will not be powered at all, and therefore the power circuit won’t work. It’s a good thing UB2 doesn’t draw much current, because otherwise the batteries would run out of juice too quickly.

Supplying 7V to the 74HC132 is a bit risky — it’s within the allowed range, but the datasheet I have recommends a maximum of 6V. Whatever–it seems to work.

The 7V rail also goes to the emitter of transistor Q3, which is a PNP transistor. We’ll see in a minute, but when the power button is not being pressed, the transistor is turned off, so it shouldn’t be drawing any current.

Additionally, the 7V rail is directly connected to one of the inputs on each of the three NAND gates. In effect, it means each one of these NAND gates is really acting as an inverter (e.g. a logic 0 at the other input becomes a logic 1 at the output, and a logic 1 becomes a logic 0). This is because:

  • 1 NAND 1 = 0
  • 0 NAND 1 = 1

I’ll go into this in a bit more detail when I describe what happens when you press the power button.

Finally, the 7V rail goes through the 100KΩ resistor R18 to the second input of the leftmost NAND gate. This is a pull-up resistor that keeps the second input of the NAND gate pulled up to a logic 1 by default. The 100Ω resistor R3 is also connected to this part of the circuit, but the other end of it is floating when both power buttons are not being pressed, so it doesn’t really affect the circuit until a power button gets pressed.

So in the default state when the system is plugged in but no power button has been pressed:

  • The leftmost HC132 NAND gate has both inputs at 7V. This means its output will be 0V.
  • The output of that first NAND gate is fed into the other two NAND gates.
  • Each of those NAND gates has one input at 0V, and one input directly wired to 7V, so they are both outputting 7V.
  • Since 7V is going into the left side of R14, there is no current between the emitter and base of the transistor. The transistor is turned off.
  • Because the transistor is turned off, the power supply control pin (J18, pin 15) is left floating (remember, the +5V output is not supplying 5V yet, since the power supply is off).
  • Thus, the power supply stays off.

When the keyboard power button is pressed, the left side of R3 is grounded. This pulls pin 2 of the HC132 NAND gate to ground through the 100Ω resistor R3. So:

  • The leftmost HC132 NAND gate has one input at 0V, and one input directly wired to 7V.
  • The output of that NAND gate is thus 7V, which is fed into the other two NAND gates.
  • Each of these NAND gates has 7V going into both input pins, so they are both outputting 0V.
  • Since 0V is going into the left side of R14, current is flowing from the emitter of the transistor to the base of the transistor. The transistor is turned on and allowing current to flow from the emitter to the collector.
  • Because the transistor is turned on, the power supply control pin is driven to 7V or so (just assume that TS1 is a short circuit for now–it will be described in more detail in the power off circuit description)
  • Thus, the power supply turns itself on.

As soon as the power supply turns itself on, it begins outputting 5V. 5V goes through D2 and R10, pulling the power supply control pin up to +5V (minus the diode drop) through the 220Ω resistor. As soon as you release the power button, this mechanism keeps the power supply control pin high. Otherwise, the power supply would immediately turn itself back off.

The rear power switch does essentially the same thing as the keyboard power switch. When you press it:

  • It completes a circuit to allow the battery to begin charging up capacitor C3, which is discharged when this process begins.
  • The left side of R3 starts at 0V and slowly starts raising up as C3 charges.
  • Since C3 starts at 0V, the same steps are followed as when the keyboard power button was pressed, and the power supply turns on.
  • After the power supply turns on, 5V is available through R9, so C3 will charge up to 5V.
  • C3 charging up to 5V is an important detail. If the button is later used for turning the computer off, the rear power switch won’t try to turn on Q3 at the same time it tries to turn the power supply off. It also gives you some time to release the power button before the system tries to turn itself back on again.

Wow! I may try to add some more diagrams in the future to depict it a bit more clearly. It’s a lot to digest.

You may be wondering: what is the point of the three NAND gates? At first glance, all they are effectively doing is repeating the input value that came out of R3. The purpose seems to be to allow better current drive capability for controlling the transistor, and a clean output that ensures that the signal controlling Q3 is either on or off, and not somewhere in between as capacitors charge up.

The three NAND gates essentially combine to create a buffer with double the current drive capability of a single NAND gate’s output. As long as all of the NAND gate pins are connected together and the NAND gates are working, you can assume that a low value at pin 2 of UB2 will turn the transistor on, and a high value at pin 2 will turn the transistor off.

Power Off

PowerOff

Just like the power on part of the circuit, the power off part of the circuit uses NAND gates that make it look way more complicated than it really is. These NAND gates are powered by the power supply’s +5V line. The two leftmost NAND gates form an SR latch. The SR latch’s top “set” input is connected to UB11, pin 13 (and the power button’s second pole). I believe this line is pulled up to +5V by a pull-up resistor inside the 6522 VIA (UB11), keeping it high (deasserted) while the computer is running. UB11 pin 13 is the /POWEROFF signal. At startup, C5 is fully discharged to 0 volts, so the SR latch’s “reset” input is low (asserted), which sets the SR latch’s stored “Q” value to 0. Then, C5 charges up, so the “reset” input goes high (deasserts itself) and stays deasserted while the computer is powered on. So the entire time the computer is running, the SR latch’s “Q” value is 0.

The !Q output of the SR latch is connected to the next two NAND gates, which act as an inverter just like in the power on circuit as I described earlier. So the output of the dual-NAND-gate inverter connected to R15 is really just the noninverted Q output of the SR latch, but with twice the maximum current. Thus, the voltage on the left side of R15 is 0V while the computer is on, keeping the NPN transistor Q2 turned off. Remember that the power supply control pin (J18, pin 15) is being pulled up to +5V in the power on circuit, keeping the power supply turned on.

Let me explain TS1 now. TS1 is a temperature switch. During normal operation, it acts like a short circuit. If the temperature gets too hot, it becomes an open circuit. If it becomes an open circuit, the power supply control pin is no longer pulled up to +5V, so the power supply shuts off (I’m pretty sure the power supply has a weak internal pulldown on that pin). During normal operation, TS1 will act like a wire.

When you tell the computer to shut down, the computer drives UB11 pin 13 (/POWEROFF) low. This causes the SR latch’s “set” input to go low, so it’s now asserted. This makes the !Q output low, which makes the inverter output high, so Q2 turns on, connecting R11 to ground. Now there is a 27Ω pulldown pulling the power supply control pin low, and a 220Ω pullup pulling the power supply control pin high (in the “power on” portion of the circuit). This is a voltage divider, so the voltage at the power supply control pin will be approximately 0.5V. This is low enough to tell the power supply to turn off.

If you press the power button on the back of the computer, the same thing happens — it connects the SR latch’s top “set” input to ground, and the exact same thing happens.

Troubleshooting

OK, so I have tried my best to explain what’s going on in each portion of the circuit. If you have a broken power circuit, how do you fix it?

The system will not power on

First of all, there are a few traces that have a tendency to go bad because they run nearby leaky capacitors:

Of course, make sure you have removed, cleaned around, and replaced all of the electrolytic capacitors on the logic board.

Make sure that all of the connections between R3, UB2, R14, and Q3 are OK. If you can, use a voltmeter to verify that the NAND gates are operating as expected when you press either of the power buttons. You can do this with the logic board out of the computer as long as the batteries are installed.

Check the power supply control pin (J18, pin 15) to see if it’s correctly getting a voltage when you press either of the power buttons. If it is, there’s a chance that the power supply is at fault.

UB2 is somewhat near a capacitor. There’s a good chance that some of its pins have lost connectivity due to corrosion. Check every pin on UB2 and make sure it has connectivity as shown in the schematic.

The keyboard power button works, but the rear power button doesn’t

There are only a few parts of the circuit that could cause this. Make sure R9, C3, the rear power button, R3, and the keyboard power button pin of L2 are all connected as shown in the schematic diagram. For the rear power button to not work when the keyboard does, one of the connections between those components has to be broken.

The system powers on, but it won’t power off

UB1 is extremely close to a capacitor. There’s an extremely good chance that it has lost some connectivity due to corrosion. Verify all connections.

See if the output of UB1 pin 8/11 is correctly going high when you press the rear power button while the system is on. If not, follow the circuit and see where the broken trace is.

Final thoughts

I thought this would be a good idea to explain the circuit in more detail. This has turned into quite a long-winded explanation, and it doesn’t have quite as many pictures as I would have hoped. Maybe this would be better as a video or an animation that shows what the circuit is doing. For now, I hope this is enough to help someone out there troubleshoot their Macintosh II/IIx power circuit.

In my particular case, a trace was broken in the R3-R18-C6-UB2 group of pins that should be connected together, and UB1 was completely covered in capacitor leakage, which had very badly corroded its pins and solder pads. I ended up replacing both UB1 and UB2 to be safe. A trace between a via and C5’s positive end was also broken and needed a patch wire. I had to run several patch wires to repair broken traces:

UB1_UB2_before_after

 

UB1_UB2_after

By the way, this wasn’t the only problem with my IIx. This was actually the last part of the logic board that I fixed. The whole power on circuit can be bypassed by tapping a 3.6V battery between ground and the power supply control pin, so I did that for testing until I repaired this circuit. The power supply would turn on, but nothing would happen. It turns out that there were a ton of other traces that were corroded. The 53C80 SCSI chip (again, near a capacitor) was very badly corroded, so I removed it, cleaned the solder pads, and replaced it with a new one. One of the solder pads had a break from its trace, so I had to run a patch wire. I also found several other broken traces on solder pads near two chips on the board.

Corrosion2

UB8_before_after

It’s been a wild ride, but I now have a working IIx!

When I wrote Parallel Port Tester, I discovered that detecting parallel ports and their I/O addresses in Windows is a pain in the butt. I have received several inquiries asking about how I did it, so it’s obvious that others agree with me. Hopefully, this post will help everyone learn how to automatically detect their parallel port addresses. I certainly don’t claim to be an expert, and there may be better ways to do it, but this method seems to work for me.

Don’t use Win32_ParallelPort

One of the strategies I’ve seen recommended is to do a WMI query on the Win32_ParallelPort class. This is originally how I started implementing Parallel Port Tester, but I quickly discovered that it sucks! Sometimes it gets the I/O address for the wrong port, and sometimes it doesn’t even find your parallel port at all. It seems to depend on the version of Windows and the chipset manufacturer. In particular, I found that it would not detect a parallel port implemented by a Moschip chipset in Windows XP. It also was returning incorrect I/O addresses on a user’s system running Vista. Anyway, I’d highly recommend that you avoid this class. I could not get it to work correctly. Maybe I was doing something wrong…but I’m pretty sure it just sucks. It sucks so bad that I disabled this method of discovering parallel ports in version 1.0.0.1 because of a report that it was returning erroneous information.

Instead, find objects belonging to the Ports class manually

For more reliable results, what I do is search for all devices that are ports (which also include COM ports). The GUID for this class is {4D36E978-E325-11CE-BFC1-08002BE10318}. I only pay attention to returned ports that are LPT ports. On each found LPT port, I request all of the port resources to determine the I/O addresses.

How do you do that? Here’s some sample C# code using .NET. Note that I’m only showing relevant bits and pieces, so you will have to fit them into a class and function as appropriate.

using System.Management;

private static readonly ushort SPPIOLength = 8;
private static readonly ushort ECPIOLength = 4;

SelectQuery portsQuery = new SelectQuery("Win32_PnPEntity",
    "ClassGUID=\"{4D36E978-E325-11CE-BFC1-08002BE10318}\"");
ManagementObjectSearcher portSearcher = new ManagementObjectSearcher(portsQuery);
foreach (ManagementObject port in portSearcher.Get())
{
    // Grab its name. This will be in the form "Blah Blah Parallel Port (LPT1)".
    // We're going to get ugly now...
    string name = (string)port.Properties["Name"].Value;

    // Skip ports that are not parallel ports. This is UGLY...I'd much rather
    // do this a cleaner way, but it seems that different parallel ports do
    // things differently.
    if (!name.Contains("(LPT")) { continue; }

    // and extract the parallel port name by looking for everything inside
    // the parentheses. I don't know how else to do it...
    int beginLoc = name.IndexOf("(LPT") + 1;
    int endLoc = name.IndexOf(')', beginLoc);
    name = name.Substring(beginLoc, endLoc - beginLoc);

    // Grab the device ID so we can find all the associated port resources.
    string deviceID = (string)port.Properties["DeviceID"].Value;
    // Replace single backslashes with double backslashes to be suitable
    // to use in a query. Note that I also had to escape it myself in
    // code...so \\ represents a single backslash.
    // TODO: Any other escaping necessary?
    deviceID = deviceID.Replace("\\", "\\\\");

    // Now search for I/O ranges of this port.
    ObjectQuery resourceQuery =
        new ObjectQuery("ASSOCIATORS OF {Win32_PnPEntity.DeviceID=\"" +
        deviceID + "\"} WHERE ResultClass = Win32_PortResource");
    ManagementObjectSearcher resourceSearcher =
        new ManagementObjectSearcher(resourceQuery);

    // Find the SPP and ECP base addresses
    ushort SPPBase = 0xFFFF;
    ushort ECPBase = 0xFFFF;
    foreach (ManagementObject resource in resourceSearcher.Get())
    {
        // Grab starting & ending addresses
        ushort startAddress =
            (ushort)(UInt64)resource.Properties["StartingAddress"].Value;
        ushort endAddress =
            (ushort)(UInt64)resource.Properties["EndingAddress"].Value);

        ushort rangeLen = (ushort)(endAddress - startAddress + 1);
        // If we haven't yet found the SPP base address, and this range
        // is big enough, use it as the SPP base address.
        if ((SPPBase == 0xFFFF) && (rangeLen >= SPPIOLength))
        {
            SPPBase = startAddress;
        }
        // If we haven't yet found the ECP base address, and this range
        // is big enough, use it as the ECP base address
        else if ((ECPBase == 0xFFFF) && (rangeLen >= ECPIOLength))
        {
            ECPBase = startAddress;
        }
    }
    // Although I didn't include the code here, if I fail to detect an
    // SPP address with the rules above, I grab the first returned
    // address and use it as the SPP base address. Then I set the ECP
    // address to 0xFFFF (which I treat as meaning "unable to find")

    // TODO: If SPPBase != 0xFFFF, add it to the list of discovered
    // ports -- the name of the port is stored in the variable "name"
}

I hope this code is helpful to someone out there! From this point, you can use something like Inpout32.dll to access the port registers directly.

I’m not a huge fan of the hack where I extract the “LPT#” name from the long name by searching for the “(LPT” string, but I couldn’t find a better way to do it. Suggestions would be appreciated! There also may be a better way to use parameters to do the ASSOCIATORS query instead of ugly escaping and string concatenation, but it worked and that was good enough for me on this particular project. I couldn’t find an easy way to set it up for use with parameters or else I would have done it that way.

I bet you thought I gave up on these lessons! In my last post about microcontrollers, I introduced the idea of an interrupt-safe circular buffer. I briefly mentioned that circular buffers are commonly used in software drivers for peripherals that send and/or receive a lot of data. One example in particular that I gave was a UART (universal asynchronous receiver/transmitter). I’d like to expand further on UARTs in this post.

How they work

First of all, what is a UART? Well, let’s break down the acronym:

  • Universal: The device can be configured for different modes/speeds.
  • Asynchronous: The communication does not use a separate clock wire for synchronization; it’s all handled only with a single data wire in each direction.
  • Receiver: It can receive data…
  • Transmitter: …and it can transmit data.

That’s great, but it still doesn’t really explain anything about what the UART does. It’s easier to describe a UART by talking about a computer serial port. If you’re geeky like me, I’m sure you’ve dealt with serial ports before. For example, if you’re a Raspberry Pi aficionado, you might have a USB-to-TTL serial adapter that you hook up to the Pi to access its Linux console. On your computer, you use a program like PuTTY or minicom to open the port for 115,200 baud, 8 data bits, no parity, and 1 stop bit. Then, you can turn on the Pi and see all of the messages it sends at boot. After the operating system is fully booted, you can also enter commands on the console.

If you have done this, you have actually worked with a UART. A UART is this whole interface I just described. You configure it for a baud rate as well as various framing parameters (7-bit data? 8-bit data? 1 stop bit? 2 stop bits? parity bits?). Then you send it bytes. It turns each byte into a sequence of 1s and 0s with a prepended start bit and an appended stop bit or two (there’s also an optional parity bit that can be added to the mix to try to detect errors). Finally, it sends this final sequence of bits onto a transmit pin at the configured bit rate. It also listens on a receive pin for data coming in with that kind of format and parses the serial bits back out into parallel bytes.

On the hardware side of things, let’s go through an example of sending out an uppercase ‘M’ with a UART configured in 8N1 mode. This means there are 8 bits per byte, no parity bit, and 1 stop bit. We start out by telling the UART to send the character ‘M’ which is represented as 0x4D according to an ASCII table. 0x4D in binary is “01001101”. A 1 represents a high state and a 0 represents a low state. The start bit is always a 0 and the stop bit is always a 1.

UART-Diagram

If you transmitted the character ‘M’ and looked at it on an oscilloscope, this is what you’d see. While the UART is inactive, it keeps the pin at a high state. Thus, the start bit is low so you can detect the start of a transmission. After the stop bit, if there is no more data to be sent, the pin stays high until the next start bit begins.

Hopefully this diagram makes sense. You’ll see that the bits are in reverse order because they are sent least-significant first. The horizontal unit on this diagram is time; the vertical lines are each about 8.7 microseconds apart, assuming the baud rate is set to 115200. I got that by calculating the reciprocal of the bit rate. If the bit rate is 115,200 bits per second, then there are (1/115200) seconds per bit.

The receiver looks for this pattern and interprets the bits between the start and stop bit. That’s really all there is to it! UARTs were very common on computers of the past generations. Earlier computers used a separate chip just for the UARTs, like the 8250 or the 16550A. In fact, many UARTs to this day remain software-compatible with the 16550A. Later on, computers began including a Super I/O chip with integrated 16550A-compatible UARTs.

How to use them

In microcontrollers, you typically find a UART (or multiple UARTs) available as memory-mapped peripherals. Sometimes they use the same interface as the 16550A, and sometimes not. For simplicity I’m going to talk about an AVR’s UART in this lesson instead of the 16550A. The 16550A is complicated because it has a couple of built-in 16-byte FIFO buffers for transmitting and receiving along with a fairly strange interrupt setup because of the FIFOs. The FIFOs are important for normal desktop computers because if your computer had to deal with an interrupt on every character, it would get bogged down handling the interrupts. The FIFO allows interrupts to occur less frequently. On a simple microcontroller like an AVR, handling an interrupt on every character is not a huge problem. Because of the lack of a FIFO, the AVR’s software interface will be easier to understand, which makes it appropriate for this introductory lesson.

I’m going to be using the AT90USB646 as an example. This is the same processor that I use on my Mac ROM SIMM programmer. It has a single USART. Notice the extra S? USART stands for Universal Synchronous/Asynchronous Receiver/Transmitter. It just adds the ability to work in a synchronous mode with an external clock pin. For our purposes, we won’t worry about this. We’ll be using it in asynchronous mode.

The AT90USB646 reference manual says that this chip has a single USART, described in chapter 18. It provides some sample code in both assembly and C for talking to the UART. It’s really not that hard to do.

Setting the baud rate

The baud rate is set with two registers: UBRR1L and UBRR1H. They are the low and high bytes of the full register UBRR1 (which you can also access directly as a 16-bit word, at least in AVR-GCC and AVR Libc). The equation, as per the datasheet, is:

UBRR1 = (oscillator_frequency / (16*baud_rate)) – 1;

So let’s use my SIMM programmer board as an example. It has a 16 MHz crystal, and let’s pretend we’re going for a baud rate of 9600. Thus, I should do:

UBRR1 = (16000000UL / (16*9600UL)) - 1;

Note that I used UL at the end of these constants to ensure they’re being treated as unsigned long integers. Otherwise, the intermediate calculations might get messed up due to only being promoted to 16-bit values. Your development environment will probably provide a define for the clock rate. Plus, it’s usually a good idea to create a #define for your baud rate instead of hardcoding a 9600UL randomly in the code. So it’s actually better to do this:

#define MY_BAUD_RATE 9600UL
UBRR1 = (F_CPU / (16*MY_BAUD_RATE)) - 1;

If you do the math (and integer truncation) yourself, you will discover that this will assign UBRR1 a final value of 103. Mathematically, it would be 103.166666666…, but it will get truncated down to an integer. This also means the baud rate won’t be exactly 9600 baud. If we solve the equation above for baud_rate and put our calculated/truncated UBRR1 value of 103 back in, you’ll see that we don’t get exactly 9600 baud:

baud_rate = oscillator_frequency / (16*(UBRR1 + 1));

baud_rate = 16000000 / (16*(103 + 1));

This gives us a baud rate of 9615. The AVR will actually be transmitting and receiving at 9615 baud. That’s an error of about 0.16%, which is definitely close enough to 9600 baud that it won’t be a problem. Usually you’re in good shape if you’re within a percent or two, and we are much closer than that. Keep in mind that oscillators aren’t perfect either, so UARTs are tolerant of baud rates that aren’t exactly correct.

Configuring the framing

We have to make sure that the USART knows what type of data to send and receive. Should there be 5 data bits? 8 data bits? 1 stop bit? 2 stop bits? Parity? No parity? All of this can be configured in some of the registers in the USART. The registers are called UCSR1A, UCSR1B, and UCSR1C. UCSR is short for “USART Control and Status Register”. These registers contain bits for controlling the setup of the USART as well as some bits that signal status of the USART.

The bits that are important for us to configure in order to get basic functionality are the following:

  • U2X1 bit (bit 1) of UCSR1A. This is a double-speed bit that essentially turns the constant “16” in the baud rate calculation formula above into an “8” — we should make sure it’s disabled since we assumed in the baud rate calculation we did earlier that it was in fact 16.
  • RXEN1 bit (bit 4) of UCSR1B. This bit enables the receiver so we can receive incoming bytes.
  • TXEN1 bit (bit 3) of UCSR1B. This bit enables the transmitter so we can send outgoing bytes.
  • UCSZ12 bit (bit 2) of UCSR1B and UCSZ11, UCSZ10 bits (bits 2 and 1) of UCSR1C. These three bits combine together to determine the number of data bits (5 to 9).
    • 0 –> 5 bits
    • 1 –> 6 bits
    • 2 –> 7 bits
    • 3 –> 8 bits (what we want)
    • 4, 5, 6 reserved
    • 7 –> 9 bits
  • UMSEL11, UMSEL10 bits (bits 7 and 6) of UCSR1C. These bits determine the mode (synchronous/asynchronous) of the USART.
    • 0 –> asynchronous (what we want)
    • 1 –> synchronous
    • 2 –> reserved
    • 3 –> master SPI (this USART is capable of being an SPI master; I have discussed SPI in the past)
  • UPM11, UPM10 bits (bits 5 and 4) of UCSR1C. These bits determine the parity mode (none, even, or odd).
  • USBS1 bit (bit 3) of UCSR1C. This bit determines if there is 1 stop bit or 2 stop bits.

So basically, we need to set up these three registers:

UCSR1A = 0; // Not double speed mode, disable address filtering
UCSR1B = 0x18; // Enable RX and TX, high bit of UCSZ1 = 0
UCSR1C = 0x06; // asynchronous, no parity, 1 stop bit, 8 data bits

AVR Libc has some nice macros for the various bit names, so you can also do this with the symbolic names:

UCSR1A = 0;
UCSR1B = (1 << RXEN1) | (1 << TXEN1);
UCSR1C = (3 << UCSZ10);

Transmitting data

After you have configured the USART correctly, it’s very easy to send and receive. To send data, you simply write the character you wish to send into the UDR1 register. Here’s an example of transmitting the character ‘A’:

UDR1 = 'A';

You can only transmit if the hardware has finished transmitting the last character. This is because the USART does not have a transmit FIFO. Because of this, you need to check a status bit to ensure the transmitter is not already busy:

if (UCSR1A & (1 << UDRE1)) {
    UDR1 = 'A';
}

This will only transmit a character if the transmit buffer is empty. Otherwise, it won’t transmit. If you need to make sure the character is transmitted, you can wait until the bit goes high:

while (!(UCSR1A & (1 << UDRE1))) {
}
UDR1 = 'A';

So you do an empty loop until the bit goes high, after which it’s safe to write to the transmit buffer.

Receiving data

Receiving is very, very similar. First, you need to know if a character is ready to be received. The hardware will tell you if a character is waiting to be read by making a bit high in the UCSR1A register. If the bit is high, you can read the character by reading the UDR1 register:

if (UCSR1A & (1 << RXC1)) {
    received_char = UDR1;
    // do something with the received character
}

The receiver actually has a FIFO (first-in, first-out) buffer to hold a couple of received characters in case your program is unable to read the first character received before another one arrives. This is a nice little safety mechanism to have, but you still need to read received characters quickly before the small FIFO fills up. Fancier UARTs have a bigger receive buffer to enable high data rates with less processor utilization.

Example code

Here’s a sample program that will echo back any character that is received. If you type the letter ‘Z’, it will transmit the letter ‘Z’ right back to you.

#include <avr/io.h>

#define MY_BAUD_RATE 9600UL

int main(void) {
    UBRR1 = (F_CPU / (16*MY_BAUD_RATE)) - 1;
    UCSR1A = 0;
    UCSR1B = (1 << RXEN1) | (1 << TXEN1);
    UCSR1C = (3 << UCSZ10);
    
    while (1) {
        while (!(UCSR1A & (1 << RXC1)));
        uint8_t ch = UDR1;
        while (!(UCSR1A & (1 << UDRE1)));
        UDR1 = ch;
    }
}

It’s really not that complicated. It waits until a character is received, and then receives it. Then it waits until the transmitter is empty, and transmits the received character. Note that I put semicolons at the end of the while() loops waiting for the bits. This was done intentionally to save space over giving them an empty loop body, but it can be a little confusing to read, so I wanted to explicitly point that out.

Anyway, this whole thing is easy, right?

Interrupts

There’s a catch. If your program has to do anything other than sit in a loop waiting for a character to arrive over the UART, this whole idea starts to fall apart. If you don’t check for received data often enough, you will run the risk of losing received characters. This could happen if the receive FIFO is filled before you get around to checking the UART again. How do we solve this dilemma?

You may have been wondering why I mentioned circular buffers at the beginning of this post. What do circular buffers have to do with anything I have mentioned so far? Good news: you’ve finally reached the point in this post where everything will start to come together and make sense.

The solution to this problem is to handle the UART with an interrupt. Whenever a character is received, the UART will fire an interrupt. This will cause your main program to temporarily jump to an interrupt handler which reads the received character from the UDR1 register and stores it in a circular buffer. Somewhere in the main loop, you can occasionally check the circular buffer to see if it contains any characters and process them. As long as the buffer is big enough so that you will check it before it fills up, it will solve the problem.

On the other end, you can also create a separate circular buffer for transmitting over the UART. This will allow you to put the data into the circular buffer and then move on to other places in your program while the data is sent in the background by your interrupt handler. Basically, it prevents you from having to sit in a loop and wait for all of your characters to be transmitted before you can move onto other tasks.

This a perfect application for a circular buffer. It’s easy to make a circular buffer interrupt-safe as long as you insert from the main loop and remove from the interrupt handler (or vice versa), and inserting and removing from it are both O(1) operations. This is exactly what we need for both the transmitting and receiving end of things.

The AVR’s USART is really handy because it has what’s known as the UDRE interrupt, which stands for USART data register empty. This interrupt fires anytime the transmit data register is ready for you to write another character. It’s based on the same status bit we checked in the non-interrupt-driven code above. So if the USART is idle, this interrupt will fire. Because of this interrupt, there are no special cases when you first start transmitting characters. When you put a character in the TX buffer, you have to ensure that the UDRE interrupt is enabled. In your UDRE interrupt handler, if the TX buffer is empty, you disable the UDRE interrupt. This logic is very simple compared to other UARTs, which often require special cases when you transmit your first character in order to get the interrupts rolling.

Example code

Here’s some sample code I wrote for interrupt-driven UART functionality. It consists of a header file and source file for the UART driver. You will notice the ring buffer functions which I renamed and borrowed from my post about ring buffers. Many AVR microcontrollers have multiple USARTs. This particular one works with USART1 on the AT90USB646.

uart.h:

#ifndef UART_H_
#define UART_H_

#include <stdbool.h>
#include <stdint.h>

void uart_init(uint32_t baud);
void uart_write_char(char data);
char uart_read_char(void);
bool uart_rx_buffer_empty(void);
bool uart_tx_buffer_empty(void);
bool uart_rx_buffer_full(void);
bool uart_tx_buffer_full(void);

#endif /* UART_H_ */

uart.c:

#include "uart.h"
#include <avr/io.h>
#include <avr/interrupt.h>

#define RING_SIZE   128
typedef uint8_t ring_pos_t;

static volatile ring_pos_t tx_ring_head;
static volatile ring_pos_t tx_ring_tail;
static volatile char tx_ring_data[RING_SIZE];

static volatile ring_pos_t rx_ring_head;
static volatile ring_pos_t rx_ring_tail;
static volatile char rx_ring_data[RING_SIZE];

static int tx_ring_add(char c);
static int tx_ring_remove(void);
static int rx_ring_add(char c);
static int rx_ring_remove(void);

void uart_init(uint32_t baud) {
    // TODO: if you are OK with hardcoding the bit rate,
    // consider using avr-libc's <util/setbaud.h>
    // functionality instead.

    // Set baud rate
    UBRR1 = (F_CPU / (16*baud)) - 1;
    // Enable RX and TX, turn on RX interrupt,
    // turn off data register empty interrupt until needed
    UCSR1B = (1 << RXEN1) | (1 << TXEN1) | (1 << RXCIE1);
    // 8 data bits, 1 stop bit
    UCSR1C = (3 << UCSZ10);

    // Clear out head and tail just in case
    tx_ring_head = 0;
    rx_ring_head = 0;
    tx_ring_tail = 0;
    rx_ring_tail = 0;
}

void uart_write_char(char data) {
    // Wait until there's room in the ring buffer
    while (uart_tx_buffer_full());

    // Add the data to the ring buffer now that there's room
    tx_ring_add(data);

    // Ensure the data register empty interrupt is turned on
    // (it gets turned off automatically when the UART is idle)
    UCSR1B |= (1 << UDRIE1);
}

char uart_read_char(void) {
    // Wait until a character is available to read
    while (uart_rx_buffer_empty());

    // Then return the character
    return (char)rx_ring_remove();
}

bool uart_rx_buffer_empty(void) {
    // If the head and tail are equal, the buffer is empty.
    return (rx_ring_head == rx_ring_tail);
}

bool uart_tx_buffer_empty(void) {
    // If the head and tail are equal, the buffer is empty.
    return (tx_ring_head == tx_ring_tail);
}

bool uart_rx_buffer_full(void) {
    // If the head is one slot behind the tail, the buffer is full.
    return ((rx_ring_head + 1) % RING_SIZE) == rx_ring_tail;
}

bool uart_tx_buffer_full(void) {
    // If the head is one slot behind the tail, the buffer is full.
    return ((tx_ring_head + 1) % RING_SIZE) == tx_ring_tail;
}

static int tx_ring_add(char c) {
    ring_pos_t next_head = (tx_ring_head + 1) % RING_SIZE;
    if (next_head != tx_ring_tail) {
        /* there is room */
        tx_ring_data[tx_ring_head] = c;
        tx_ring_head = next_head;
        return 0;
    } else {
        /* no room left in the buffer */
        return -1;
    }
}

static int tx_ring_remove(void) {
    if (tx_ring_head != tx_ring_tail) {
        int c = tx_ring_data[tx_ring_tail];
        tx_ring_tail = (tx_ring_tail + 1) % RING_SIZE;
        return c;
    } else {
        return -1;
    }
}

static int rx_ring_add(char c) {
    ring_pos_t next_head = (rx_ring_head + 1) % RING_SIZE;
    if (next_head != rx_ring_tail) {
        /* there is room */
        rx_ring_data[rx_ring_head] = c;
        rx_ring_head = next_head;
        return 0;
    } else {
        /* no room left in the buffer */
        return -1;
    }
}

static int rx_ring_remove(void) {
    if (rx_ring_head != rx_ring_tail) {
        int c = rx_ring_data[rx_ring_tail];
        rx_ring_tail = (rx_ring_tail + 1) % RING_SIZE;
        return c;
    } else {
        return -1;
    }
}

ISR(USART1_RX_vect) {
    char data = UDR1;
    rx_ring_add(data);
}

ISR(USART1_UDRE_vect) {
    if (!uart_tx_buffer_empty()) {
        // Send the next character if we have one to send
        UDR1 = (char)tx_ring_remove();
    } else {
        // Turn off the data register empty interrupt if
        // we have nothing left to send
        UCSR1B &= ~(1 << UDRIE1);
    }
}

Finally, here’s a test program that tests out the USART:

#include <avr/interrupt.h>
#include "uart.h"

int main(void) {
    cli(); // disable interrupts
    uart_init(9600);
    sei(); // enable interrupts
    while (1) {
        uart_write_char(uart_read_char());
    }
}

This particular sample program doesn’t really take advantage of the fact that the code is interrupt-driven, but at least it tests the code out. The interrupt-driven aspect of the code shines when you need to send a 50-byte string and don’t want to wait around until it finishes sending. With this code, if you quickly write 50 bytes, the program will continue doing other things while the 50 bytes are sent in the background as interrupts come in. For receiving, you can let your program do many things without worrying about losing a received character. Every time through your main loop, you should process all characters that are waiting in the receive buffer to ensure that it doesn’t fill up.

Other functions

You could easily add other functions for transmitting a string, receiving a full line, etc. It would probably be nice for the transmit and receive functions to have optional parameters to tell them not to block if the buffers aren’t ready. These are all excellent exercises I leave to you, the reader, to implement.

Conclusion

I hope this explains the UART/USART in enough detail to get you started. This is a very important peripheral that is commonly used for interfacing with displays, sensors, and other microcontrollers. If you’re going to use a UART/USART, I would highly recommend that you do it with interrupts as I described above. This is definitely the type of thing you should stick into your toolbox of peripheral drivers and reuse on all of your projects.

Have you ever wanted to be able to call a C++ Qt library from Python? It turns out it’s pretty easy to do. This tutorial explains how to do it in Mac OS X, but it’s slightly out of date. I’d like to provide the world with a way to make it work with Ubuntu 14.04 out of the box. My instructions are based on that tutorial and the PySide Binding Generation Tutorial. Let’s check it out. Note: This mechanism seems to work for wrapping Qt 4 libraries. I don’t think PySide works with Qt 5, at least in Ubuntu 14.04. Keep that in mind. Everything we do will be with Qt 4.

We’re going to create a library called libdougtest. It’s going to contain a QObject subclass called TestClass. We will create a Python wrapper for it called libDougTestPy, which we will eventually be able to refer to as DougTestPy to follow Python’s library naming conventions. Let’s get started.

Install some prerequisites

There are many things you need installed:

sudo apt-get install build-essential qt-sdk python-pyside* libshiboken-dev shiboken pyside-tools libpyside-dev

I got a warning saying I should install a Phonon backend when I did this, so I also installed the package phonon-backend-gstreamer just in case.

Create a project directory

We’re going to start out by creating a directory for this entire project. Let’s call it  dougtest_project:

mkdir dougtest_project
cd dougtest_project

All work we do will be inside of this directory. The reason I’m naming it dougtest_project instead of just dougtest is because I’m going to create another directory inside of it called dougtest for the C++ library itself.

Create libdougtest

Now, let’s create the good old C++ library that we want to wrap. This is pretty easy, since you just follow the standard Qt way of creating a library. For simplicity, I’ve provided a barebones library containing a single class. We are going to create a directory inside of dougtest_project called dougtest which will be the home for this library:

mkdir dougtest

First of all, create the project file:

dougtest/dougtest.pro:

QT += core
TARGET = dougtest
TEMPLATE = lib

HEADERS += testclass.h
SOURCES += testclass.cpp

Now, create the header and source files:

dougtest/testclass.h:

#ifndef TESTCLASS_H
#define TESTCLASS_H

#include <QObject>

class TestClass : public QObject
{
    Q_OBJECT
public:
    TestClass(QObject *parent = 0);
    virtual ~TestClass();
 
    void doSomething();

private:
    int value;
};

#endif

dougtest/testclass.cpp:

#include "testclass.h"

TestClass::TestClass(QObject *parent) :
    QObject(parent),
    value(0)
{
 
}

TestClass::~TestClass()
{
}

void TestClass::doSomething()
{
    value++;
    qDebug("Testing: %d", value);
}

This concludes the creation of the library. Let’s make sure it compiles:

cd dougtest
qmake-qt4
make

If you’re successful, you should see libdougtest.so.1.0.0 with several symlinks. Change back to the main directory:

cd ..

OK, we have successfully created the library that we’re going to wrap. We haven’t seen anything special yet; the fun is coming now.

Create files for shiboken

Next, we’re going to create some files that shiboken will use during the wrapping process. shiboken is a binding generator for creating Python bindings for C++ libraries, which is exactly what we want.

mkdir data

Create the following two files:

data/global.h:

#undef QT_NO_STL
#undef QT_NO_STL_WCHAR

#ifndef NULL
#define NULL 0
#endif

#include <QtCore/QtCore>

#include <pyside_global.h>
#include <testclass.h>

data/typesystem.xml:

<?xml version="1.0"?>
<typesystem package="DougTestPy">
    <load-typesystem name="typesystem_core.xml" generate="no"/>
    <object-type name="TestClass"/>
</typesystem>

These files specify the classes to wrap, and will be passed as parameters to shiboken, as you will see shortly.

Create wrapped library project file

Next, we need to create a Qt project that will generate the wrapper library. Create the directory DougTestPy (inside of the main dougtest_project directory):

mkdir DougTestPy

Create the file DougTestPy/DougTestPy.pro:

TEMPLATE = lib
QT += core

# Need to have access to header files from the library being wrapped.
INCLUDEPATH += ../dougtest

# PySide include paths
QMAKE_CXXFLAGS += $$system(pkg-config --cflags pyside)
INCLUDEPATH += $$system(pkg-config --variable=includedir pyside)/QtCore

# PySide dependencies
LIBS += $$system(pkg-config --libs pyside)

# Link against the library being wrapped
LIBS += -L../bin/ -ldougtest

# Generate the wrapper library in the bin directory
TARGET = ../bin/DougTestPy

# Here are all the sources that go with it
SOURCES += \
    DougTestPy/dougtestpy_module_wrapper.cpp \
    DougTestPy/testclass_wrapper.cpp

This script may be confusing because it refers to source files that we haven’t created yet. This is the naming scheme that shiboken will use when we use it to create the wrapper code.

Create the build script and run it

We are finally ready to create a shell script that will call shiboken and do everything that needs to be done. I hope this isn’t too complicated. It’s very possible that a Makefile would work too, but this is what I have working.

Create the file build.sh inside the main dougtest_project directory:

#!/bin/sh

# Absolute path to this script, e.g. /home/user/bin/foo.sh
SCRIPT=$(readlink -f "$0")
# Absolute path this script is in, e.g. /home/user/bin
SCRIPTPATH=$(dirname "$SCRIPT")

WRAPPER_NAME="DougTestPy"
WRAPPED_NAME="dougtest"

QT_INC=$(pkg-config --variable=includedir QtCore)/..
QTCORE_INC=$(pkg-config --variable=includedir QtCore)
PYSIDE_INC=$(pkg-config --variable=includedir pyside)
QTTYPESYSTEM=$(pkg-config --variable=typesystemdir pyside)

# I'm assuming the wrapped library is built in-place; if not,
# change WRAPPED_BIN_DIR to the location of the final binaries.
WRAPPED_SRC_DIR="${SCRIPTPATH}/${WRAPPED_NAME}"
WRAPPED_BIN_DIR="${SCRIPTPATH}/${WRAPPED_NAME}"

cd $SCRIPTPATH

# Clean up the old build to ensure we start fresh. If we don't do this,
# old stale stuff can be left behind, which can cause crashes in the
# library after you make changes.
rm -rf ./bin ./${WRAPPER_NAME}/Makefile ./${WRAPPER_NAME}/${WRAPPER_NAME}

# Copy the latest C++ library build into the bin directory.
mkdir -p bin
cp -R ${WRAPPED_BIN_DIR}/lib${WRAPPED_NAME}* ./bin/

# Now work inside the wrapper directory
cd $WRAPPER_NAME

# Create the wrappers for the library
shiboken ../data/global.h \
    --include-paths=$WRAPPED_SRC_DIR:$QT_INC:$QTCORE_INC:$PYSIDE_INC \
    --typesystem-paths=../data:$QTTYPESYSTEM \
    --output-directory=. \
    --avoid-protected-hack \
    ../data/typesystem.xml
if [ $? -ne 0 ]; then
    echo "shiboken returned error. Something is shibroken :-("
    exit 1
fi

# Build the python wrapper library
qmake-qt4
if [ $? -ne 0 ]; then
    echo "qmake-qt4 returned error."
    exit 1
fi

make
if [ $? -ne 0 ]; then
    echo "make returned error."
    exit 1
fi

# Back into the directory containing the script
cd ..

# Make meaningful symlink so Python can use the generated library
rm -rf ./bin/${WRAPPER_NAME}.so
ln -s lib${WRAPPER_NAME}.so ./bin/${WRAPPER_NAME}.so

OK, you’re ready to run this build script:

sh build.sh

Assuming everything succeeds, you should end up with a bin directory with the following contents:

  • DougTestPy.so (a symlink to libDougTestPy.so)
  • libDougTestPy.so, libDougTestPy.so.1, libDougTestPy.so.1.0, and libDougTestPy.so.1.0.0
  • libdougtest.so, libdougtest.so.1, libdougtest.so.1.0, and libdougtest.so.1.0.0

The symlink has to be created because the build process generates a library that begins with “lib”, but Python expects the library to not be prefixed with “lib”. Maybe there is a way to fix it so that qmake doesn’t add the “lib” prefix, but I couldn’t figure out how after a quick search, and this did the trick, so I let it go.

Test the library

We’re ready to test it out. Create the file test.py in the main dougtest_project directory:

import sys
sys.path.append("bin")
from PySide.QtCore import *
from DougTestPy import *

# Create a TestClass instance
tc = TestClass()

print "Created test class"

# Test the TestClass
tc.doSomething()

print "Called test class function"

tc.doSomething()

print "Called test class function again"

Now, run it. Note that I had to add the “bin” directory to Python’s path in order to use the library. We’re not done with library trickery yet though: libDougTestPy.so depends on libdougtest.so. So libdougtest.so needs to be in the system library path. The easiest way to do this temporarily is to add the bin directory to LD_LIBRARY_PATH:

export LD_LIBRARY_PATH=bin/

For a more permanent solution, you could put libdougtest.so* into /usr/local/lib and run “sudo ldconfig” to eliminate the need for that hack.

Now, run the test script:

python test.py

You should get this output:

Created test class
Testing: 1
Called test class function
Testing: 2
Called test class function again

Conclusion

I hope this was helpful. It’s really not that bad; it’s just that it requires a lot of manual setup. It would be nice if there was something that could automatically generate all the junk we had to make by hand. Maybe such a tool already exists and I’m wasting my time. Either way, I think this is interesting. Thanks again to the people who wrote the tutorials I linked above.

Note that if you use libraries other than QtCore, you will need to add them into the appropriate places in data/global.h, data/typesystem.xml, DougTestPy/DougTestPy.pro, and build.sh. Just see where the word “core” or “Core” appears, and add your extra libraries such as QtGui and QtXml in the same format.

I’ve already gotten my Willem programmer working in Windows 7 with a custom DLL, so I figured why not do the same in Wine under Linux? I meant to make directions for this earlier, but the last time I looked at this, Wine had a bug that made it really annoying to build 32-bit DLLs on 64-bit Linux. That’s all fixed now, so here we go. These instructions are tested with Ubuntu 14.04 with Wine 1.6. Some older versions of Wine had problems, so this is all I can test with.

Warning: This DLL is slow, and these instructions are fairly complicated. I don’t really want to supply prebuilt binaries, and it requires some special work that might have to be repeated when new versions of Wine are installed due to updates. I’m also too lazy to make it configurable, so you need to change a #define in the code if your parallel port is at an I/O address other than 0x378. If you’re still feeling brave, here’s what I have:

Get your parallel port address

First of all, figure out your parallel port address. Type the following command:

cat /proc/ioports | grep parport

You should see something like this:

  0378-037a : parport0

In this case, the output means my parallel port is at 0x378. Remember this address; we will customize it when we compile the DLL.

Download the DLL source code

Download the DLL source code here:

io_dll_willem_linux.tar.gz

Extract it. If your parallel port is at an address other than 0x378, open up io_main.c and update the LPT_BASE_ADDR define near the top of the code to match your parallel port address.

Install prerequisites

These packages will probably be necessary to make everything work. I hope I got all of them, but I may have forgotten something. If I forgot something, I apologize. Let me know in the comments.

sudo apt-get install build-essential wine1.6-dev gcc-multilib

Build the DLL

Inside the DLL source directory, type:

make

Assuming everything goes without error, you should find the file io.dll.so which is the DLL we will be using.

Install the DLL

Remove any io.dll from your Wine installation. It’s useless inside of Wine anyway, so there’s no need for it. Make sure there’s no io.dll file next to EpromM51.exe either. I’m trying to make sure that Wine doesn’t look anywhere else for this DLL. Now, we’re going to put the io.dll.so file in a place where Wine can find it:

sudo cp io.dll.so /usr/lib/i386-linux-gnu/wine/

I messed around, but couldn’t get this to work without putting it right in Wine’s library directory. There might be a better way to do this, but this was the best I could find.

Even if you’re on a 64-bit system, you’ll find that this DLL gets compiled as a 32-bit DLL. So /usr/lib/i386-linux-gnu/wine is always the correct location for this library, regardless of whether your system is 32-bit or 64-bit.

Set up permissions

Here’s the weird part. Raw port access doesn’t work unless you’re root, or you allow the program to have capabilities to access I/O ports. Running things as root is a bad idea, so let’s give the program capabilities to access the I/O ports. Warning: This could pose a potential security threat to your system. We’re actually going to be giving all Wine programs access to the I/O ports.

Here’s the magic command that allows Wine programs to use raw I/O:

sudo setcap cap_sys_rawio=ep /usr/bin/wine-preloader

You may or may not have to re-run this command every time you update Wine. I just don’t know. Anybody else have a better solution for direct port access? I hate having to mess with wine-preloader, but I don’t want to create a separate daemon or whatever due to the added latency.

All done

If you’re followed all these directions, it’s very possible that your Willem programmer is all ready to go now. Give it a shot. I hope this helps someone out there.

Boring details for developers

For anyone who’s interested in how I did this, I created this DLL by taking the original io.dll, fixing its header file so it doesn’t use weird typedefs that confuse winedump, and then running this command:

winedump spec io.dll -I . -t -C

After that, I ran this command:

winemaker . --nosource-fix --dll --nomfc -D__WINESRC__ --wine32

This gave me everything I needed to get this to work. I also had to fix up the generated code–for example, I commented out the line: #include “config.h” in io_main.c.

I’ve been using /etc/network/interfaces to automatically bring up a Grid Connect PCAN-USB adapter when my computer boots up (or when the adapter is hotplugged). It was working perfectly fine in Ubuntu 13.04 and 13.10.

When I upgraded from Ubuntu 13.10 to 14.04, my previous method of adding the CAN adapter to /etc/network/interfaces caused Ubuntu to take forever to boot with a “Waiting for network configuration…” message (and completely broke all of my network interfaces including both ethernet and CAN). It turns out my previous method was bad because I was using the “inet” address family instead of the “can” address family. See “man 5 interfaces” for details on this address family. I found a much better way to make it work. Here’s my new entry in /etc/network/interfaces:

allow-hotplug can0
iface can0 can static
	bitrate 250000
	up /sbin/ip link set $IFACE down
	up /sbin/ip link set $IFACE up txqueuelen 1000 type can bitrate 250000 sample-point 0.7 triple-sampling off restart-ms 500

Make sure that the three indented lines are indented using tabs. Spaces will cause the file to parse incorrectly, but I had to use spaces to make it format correctly on this blog. (BTW, I believe ifupdown is not supposed to be picky about spaces and tabs, but Network Manager is picky about it). I checked and supposedly even 12.04’s ifupdown has this CAN address family supported, so it should probably work in 12.04 (however, it’s untested by me).

I like to set a custom txqueuelen and restart-ms, neither of which seem to be supported in /etc/network/interfaces, so I added a couple of /sbin/ip commands to do it manually. I first have to bring the interface down, then set the rest of the custom options and bring it back up in the process. These commands seem to be executed after the interface has already been brought up, so that’s why I have to bring it back down. The ip command complains if I try to set options on the interface while it’s already up.

The first indented line that says “bitrate 250000” is required. Even though I end up setting the bit rate again in my manual ip command, the first line has to be there or ifup will complain. You can also set the sample point and triple sampling options with lines that say “triple off” and “samplepoint 0.7”, but since I’m specifying them in the manual command anyway, I figured it would be a waste to repeat them.

Hope this helps someone trying to get their CAN adapter to automatically go up. It’s really quite simple, but I couldn’t find any correct examples online.

Update 8/21/2014: I changed these instructions slightly to use “allow-hotplug can0” instead of “auto can0”. This ensures that it works correctly if the USB CAN adapter is not plugged in when the computer starts up. With “auto can0” you may see a long “Waiting for network configuration…” delay if you boot the system with the adapter unplugged.

Up until recently, I did all of my soldering the “old-fashioned” way with my soldering iron. As I began building up boards for complex projects that use a multitude of surface mount components, I started to realize how inefficient that method is. It’s a lot of tedious work to solder each individual SMD component by hand, especially tons of capacitors and resistors. It’s the same repetitive task over and over again: put solder on one of the pads, pick up the component with tweezers, heat up the solder I just put on the pad, put the component into place, optionally push the component down against the PCB while heating the solder, and finally solder the other pad. When you’re building a board that only has a few parts, it’s no big deal. When you’re building a board that has 20 or 30 capacitors and resistors, it can really get boring and slow. It’s definitely doable, but it’s also exhausting.

I investigated other methods of soldering. This research led me to solder paste. The idea is that you put solder paste onto all of the surface mount pads on your PCB. Next, you place the surface mount parts onto the pads. The paste holds the parts in place so you can move the board around without sending components flying all over. When you have all of the parts in place, you heat up the board to melt the solder and activate the flux. Some people use a hot air rework station to do this; I prefer putting the board into a repurposed toaster oven. I’ll explain in more detail later in this post.

Of course, I still use my iron for a lot of stuff I do: through-hole parts, drag soldering fine-pitch ICs, fixing problems, soldering boards that use a small number of SMD components, and probably other situations I’m not thinking of at the moment. I think there’s a point at which it becomes more efficient to use solder paste instead, particularly if you’re building up multiple boards at once. I’m not sure exactly where that “critical point” is, but you develop a feel for it as you gain more experience. If I’m doing a single board with about ten 0805 components, I’d rather do that by hand with an iron. If I’m doing five of them, or a board with 30 0805 components, I’m going to lean toward using solder paste. The main reason for this is the cost of setup time. It takes longer for me to get set up for building boards with solder paste (not to mention cleaning up when I’m finished). So if I’m going to get set up for a solder paste soldering session, I’m going to prefer if I have a lot of soldering work to do to make the setup time worth it.

There are also situations that basically require using solder paste, such as components with leads that are difficult or impossible to reach by hand. For example, many QFN packages have a large ground connection underneath the chip. Some smaller connectors like micro-USB connectors and microSD sockets are also difficult to do by hand. These can probably be done using normal solder and a hot air rework station (or in some cases, careful soldering with a small iron tip), but it’s usually easier for me to just do the whole board with solder paste in that case despite the setup time.

What I’d like to do in this post is share and describe my soldering setup. With the right tools, solder paste can be pretty fun to work with.

Solder paste

SolderPasteThere are a lot of options out there for solder paste. I can’t speak for all the options because I’m relatively new to the market, but I have had good luck with Chip Quik SMD291AX10 paste and Zephpaste. Solder paste typically has a shelf life of 6 months or less. If you refrigerate the paste it will last longer, but I don’t really care to put the paste in the same location as my food. It lasts long enough without refrigeration for me. If your solder paste dries up and quits working, you can always buy more. Keep this in mind when you buy solder paste–only buy as much as you need.

Not only are there a lot of options for the paste, but there are options out there for putting the paste on the board. Perhaps the best method is to have a stencil made. Many PCB fabrication shops can also make stencils you can use for applying solder paste. You secure the stencil over your board, put some paste on the stencil, use a squeegee to spread the paste over the holes in the stencil, lift the stencil up, and you are left with a board that only has solder paste in the places where you want it.

SolderPasteByHandIf you don’t want to have a stencil made, you can also manually apply the paste onto pads by pushing it through the syringe. If you go this route, you will also want to get a variety of dispensing needles that you can attach to the syringe. You can find these on eBay and many suppliers. I got a variety pack from Zephyrtronics (see near the bottom of the link). In addition, if the syringe does not come with a plunger, you will need to pick one up so you have something to push to make the solder paste come out.

I’m not a big fan of manually dispensing the paste by hand using the syringe. It requires much more force than you might think to push the plunger. I even bought a fancy plunger that is supposed to be more comfortable, but it’s still way too difficult to use. One thing that helps is to cut the needle length down. I use an X-acto knife to cut the needle down so it barely sticks out. You have to be careful because if you push down with too much force while cutting, you’ll squish the needle and it will be bent shut. There are also some tapered plastic tips that are easier to use, but their disadvantage is that it’s easier to accidentally smear the paste when you’re dealing with small pads that are close together.

I found a happy medium between having a stencil made and manually dispensing the paste: a solder paste dispenser! You can attach an air tube to the end of your syringe and use compressed air to push the solder through the syringe. It’s foot-operated so you push a pedal and the paste starts coming out of the needle (you will still need to get the dispensing needles). Because you no longer have to exert force pushing on the syringe plunger, it’s much easier to concentrate on aiming the paste onto the pads.

Solder paste dispenser

DispenserThe dispenser I decided to buy was a cheap (~$100) dispenser on eBay. It doesn’t have a model number on it, but it appears to be a “982” dispenser. I’ve seen this model for sale under the names KLT-982A and SP982. The eBay item didn’t specify a part number. I made sure to buy a version that was advertised as a 110V model compatible with the United States; some of the items on eBay only work on 220V power. I also made sure to buy one that operates on 24V DC power internally because I had heard that some of the older versions actually had 110V power going through the foot pedal–that sounds a little risky!

You can operate it using either a foot pedal or a hand switch. It has a manual mode where as long as you’re pushing down on the pedal, air is pushed through. There is also an adjustable auto mode where it will put out timed pulses of air. Ideally you could use this to dispense exact quantities of solder paste. In practice I’ve found that the auto mode works inconsistently, so I just dispense everything in manual mode. Once you get the hang of it, it’s not too bad. Since I’m only using manual mode, the controller is probably overkill and a clever person could put something together on their own with a valve and foot pedal (and pay a lot less money in the process). I’m too lazy to care; the pre-assembled controller is way more convenient and clean than hacking something together.

I did mention earlier that you need compressed air. This adds more stuff you have to learn about. This is actually the main reason I decided to write this post. Before messing with solder paste I knew nothing about air compressors and the different types of connectors you use to hook things up. I had to go research all of this stuff for myself and I’d like to simplify it for anybody else who buys a solder paste dispenser in the future.

Air compressor and fittings

If you’re going to use a solder paste dispenser, you need a source of compressed air, somewhere in the range of 50 psi or so. I didn’t have an air compressor when I got started, so I did some research to decide on something appropriate for my needs. I didn’t want to get the very smallest compressor available, so I went with the Porter-Cable C2002-WK. It goes up to 150 psi and has a 6-gallon tank. It was still pretty reasonably priced. I was glad I picked this one because it is big enough to handle all of my needs. When I’m going to be applying solder paste, I fill up the tank once, set the regulator to 50 psi, unplug it, and shut it off. It will have plenty of air for the rest of the day. The dispenser barely uses any air at all. The compressor is fairly loud when indoor so I don’t like being near it when it fills up. It’s nice that I can turn it on, walk away until it’s done, and then shut it off knowing that it won’t have to fill back up anymore.

I’m brand new to air compressors, so I’d like to share everything I needed to learn in order to get this system up and running. First of all, the air compressor has this type of connector for hooking something up to it:

FemaleQuickConnect

It also comes with an air hose with an identical connector. The other end of the hose plugs into the compressor’s connector.

QuickConnectHose

This connector is sometimes called a “quick-connect” or something similar. The idea behind these connectors is that when nothing is plugged into a female connector, air is not allowed to flow. As soon as you plug something in, air is allowed to start moving.  To plug it in, you pull back the sleeve on the female connector and push the male connector in. To remove it, you pull back on the sleeve and the male connector pops out. There are many types of quick-connect couplings out there; this one is called an industrial connector.

That explains the air compressor side of things, but what about the receiving end? The connector on the solder paste dispenser is pretty interesting. You see this little blue connector on the back where you feed the air in?

PushConnect

This is what’s called a push-to-connect fitting. You stick an air tube into the connector and it will stay in place. If you try to pull it out it won’t come out. To remove it, you push on the little blue ring that you can see on the connector (it’s kind of like a button). While pushing that blue ring in, you can easily pull the air tube out. It’s really a nifty setup. According to this page, this connector on a similar dispenser accepts tubing with a 6 mm outer diameter. This seems to match what my connector accepts. If you need to get tubing for it, I would definitely recommend to look for 6 mm tubing instead of a similar size in inches just to be safe.

The dispenser comes with an air hose for plugging into this connector. One end is just the bare hose that plugs into this push-to-connect fitting. The other end has a strange quick-connect coupler that seems to baffle many people online:

AsianConnector

This is not the industrial connector that my air compressor expects. I don’t really think I found a definitive answer for what type of connector this is, although I did find a lot of theories. This link was actually very helpful. It seems that this connector is some kind of Asian connector that is not typically found in the United States. After a ton of Googling, I took a gamble and bought a 30E570 coupler. On eBay this goes for about $27 plus shipping. Since the URL might change, I’m not going to link to it; just search for it. It claims to be a “1/4 inch Asian Hi Flow Interchange” adapter. This adapter provides a female connector that mates to the weird male connector on the air tubing that came with the solder paste dispenser. I don’t know if the connector is an exact match, but it seems to fit and it doesn’t leak, so it’s good enough for me!

The other end of this coupler gives you a 1/4″ female NPT. NPT (national pipe thread) is a standard type of threading commonly found in the United States. With NPT stuff, you screw the male and female connectors together, typically wrapping some teflon tape around the threads to ensure a good connection with no leaks. My compressor came with yet another adapter that converts 1/4″ male NPT to the quick-connect needed for plugging into the compressor, so I was able to screw this adapter onto the 30E570 coupler to make something that could finally hook into the compressor.

If you don’t want to buy the weird Asian Hi-Flow to NPT adapter, there is another option: you can buy an adapter that converts 6 mm outer diameter push-to-connect to 1/4″ NPT. There is a male NPT version or female NPT version of this adapter available. Then you just need some bare 6 mm outer diameter tubing and you can connect the tubing between your adapter and the dispenser. Here’s an example of what I mean if you want to try this option:

TubeAndAdapter

You’ll also need a 1/4″ NPT to quick-connect adapter so you can plug it into the compressor. I went ahead and used the Asian Hi-Flow adapter instead though, as you’ll see below.

Air filter

The solder paste dispenser’s documentation says it needs dry, clean air. Air compressors typically put some moisture and oil into the air, so it’s a good idea to add a filter. As far as I can tell, it’s not a huge problem with smaller compressors, but it’s still a good idea just in case. I bought an Ingersoll Rand F35121-400-VS air filter and put it into the chain of connections. It has 1/4″ female NPT connectors for input and output, so with a 1/4″ NPT male-to-male nipple and the 1/4″ male NPT to quick-connect adapter I mentioned earlier (this kit has a nice variety of connectors and this sealing tape worked great), I was able to assemble this final monstrosity that can hook into my air compressor’s hose:

Filter

I read that it’s a good idea to put the filter closer to the dispenser than the compressor, so I hook this assembly into the long hose that came with the compressor to add some distance. Setting up the connection between the compressor and the dispenser was definitely the hardest part to figure out about this whole setup. It would have been nice if either the dispenser had an NPT connection or the included tubing had a standard industrial connector. Either of those two options would have been great. Regardless, it’s not really a concern for me now because I have the required connections. I really hope this information is useful to other people who are considering buying one of these dispensers. This is a huge stumbling block especially for someone like me who is new to pneumatics. I honestly knew nothing about anything I wrote in this section until I researched it online in order to figure out what I needed for this dispenser.

A few more air compressor notes

If you’re going to be a new air compressor owner, I’d like to share a few tidbits. First of all, when you’re done and you release any remaining pressure, it can be extremely loud, especially if you’re indoors. With the tank mostly full, I released the air through the safety valve like the instructions mentioned, and I was very surprised by how loud it was. My ears were bothering me for several days afterward. If you’re going to do that, I would highly recommend earplugs. I prefer to hook up the included air blower tool and release the air through it; it’s much quieter.

After the tank is mostly depressurized, you need to open up the drain valve at the bottom and drain out any liquid. I typically haven’t seen much (if any) moisture come out of my tank when I only fill it up once, but I do see liquid come out if the compressor has to run a lot when I’m using it for something other than solder paste. You should always check to make sure. If you don’t drain the tank it could develop rust inside and that would be a very bad thing.

Connecting to a solder paste syringe

We have covered hooking the dispenser up to the compressor, but what about the solder paste syringe? The air output of the dispenser uses another push-to-connect fitting with the same size as the air input (6 mm outer diameter). The dispenser came with three 30cc syringes and an adapter to connect the dispenser up to a 30cc syringe. It’s basically tubing connected to a cap you can put on the syringe. The cap has an O-ring to keep the syringe sealed tightly.

AirAdapter30cc

The solder paste I bought came in a 10cc syringe, so I didn’t have an appropriate adapter to hook it up. I found some 10cc syringe adapters on eBay and bought them (this adapter on Amazon should work too). The adapters I bought looked exactly the same as the 30cc adapter, except they were smaller and had a green plastic connector instead of bare tubing at the end. I messed around with the green connector and discovered it is a very bad idea to push it directly into a push-to-fit connector. Instead, you should cut off the green connector and just use the bare tubing. Luckily, the one I found seemed to fit perfectly even though the tubing diameter wasn’t specified. The adapter fits tightly onto the solder paste syringe and then twists to lock in place:

AirAdapterOnSyringe

Dispenser needles

As I mentioned earlier, I bought a variety pack of needles. After experimenting with them, I have settled on two favorites: 21-gauge (about 0.22″ diameter) and 23-gauge (about 0.16″ diameter). The 21-gauge tip works well for most purposes. For really fine-pitch stuff such as 0402 capacitors and resistors, the 23-gauge tip is able to dispense a smaller amount of paste. Both of these sizes would take a lot of force to do by hand.

When I first put a needle onto the solder paste syringe, it takes a while for the paste to make its way through. I usually have to push down on the pedal non-stop for 15 seconds or so before the paste starts coming out. Cutting the needle shorter than its original length helps with this problem.

I’ve read that these dispensing needles are not reusable, but in my experience you can reuse them if you clean them out. When I’m finished using them, I put them on an empty syringe and hook the empty syringe up to my dispenser. Then I blow out the rest of the solder paste until only air is coming out of the needle. You can also fill the syringe with water to do a better cleaning job if you want.

Dispensing the paste

I think I’ve shared everything you need to know in order to get everything hooked up. Once it’s all hooked up and you’ve filled up the air compressor, you’re ready to dispense paste. I usually set the air compressor’s pressure regulator to about 60 psi. I’ve never needed more pressure than that. You don’t want to go higher than what the dispenser’s documentation says it can take (100 psi on mine). You can play around with the regulator on the solder paste dispenser to find your sweet spot. I usually set it to about 50 psi or so. You don’t want to go too high, but it has to be high enough to get the solder paste through the thin dispensing needle. You may find that a higher pressure makes the paste come out faster. It’s all up to your experimentation.

Like I said earlier, I don’t trust the auto mode on the dispenser. It’s a really cool idea, but what I’ve noticed is sometimes the needle clogs up and stops dispensing for a while. There are other times when the paste temporarily starts coming out quicker than usual as well. Because both of those situations happen fairly often, I’ve opted to stick with manual mode. When I press down on the foot pedal, the air starts pushing paste out. When I release the foot pedal, the air stops pushing. It may sound annoying to handle it manually, but it actually works pretty well.

I generally put a dab of paste on each SMT pad. When I’m dealing with multiple small pads, such as a QFP or PLCC package, I put a line of solder paste across the pads. This is another good application of the manual mode of the dispenser. You can see an example of solder paste across multiple pads in the picture below (this is one of my Mac ROM SIMM programmer PCBs after I have dispensed paste onto the pads).

PastedBoard

Putting the line of paste across multiple pads is pretty tough. You have to use an appropriate gauge of needle and you have to get just the right amount of paste. If you don’t use enough, some of the chip’s pins will not get enough solder. If you use too much, you’ll end up with pins shorted together. Shorting is especially a problem with fine-pitch components: 0.5 mm and less.

On the pads for the QFP device in the picture above, I got my paste a little thin near the top of the right side and too thick on the bottom of the right side. I luckily didn’t end up with a short on the part where it was too thick, but I didn’t end up with quite enough solder on the part where it was too thin. You’ll see what I mean on the finished board later on.

When I do a board with 0.5-mm-pitch components, I actually solder those components ahead of time with a soldering iron using drag soldering. I’m just better at drag soldering than paste dispensing. After I have the difficult components placed, then I dispense paste for handling the easier components. I would guess that solder paste stencils would work a lot better for getting just the right amount of paste for fine-pitch parts though. Definitely consider having a stencil made if you’re going to be building PCBs in high volumes.

Placing the components

The solder paste holds the parts in place after I drop them onto the PCB. There are many ways to place the parts. The most common way is probably to grab components with fine tweezers and drop them into place. If you’re only placing a few parts, that’s probably the easiest way to do it. At some point, I started realizing that placing a ton of components was annoying, so I searched for a better way to do it. What I’m about to describe is the best solution I came up with. Credit goes to this YouTube video for inspiring me.

The video talks about a vacuum pickup tool. The idea is you have a little pen you hold in your hand. By either pushing a foot pedal or covering a hole on the pen with your finger, you get vacuum suction that is strong enough to pick up a part. The guy in the video recommends something that is foot-operated because otherwise it’s hard to move your finger off the hole without moving the pen, thus jerking the part around that you’re trying to drop. I wholeheartedly agree, but I couldn’t find anything affordable that was foot-operated.

VacuumToolI ended up finding something that isn’t even designed with electronics in mind. It’s called the BeadSmith Pick-It-Up. It’s available from many retailers online (including Amazon). It’s actually meant for people who are doing arts and crafts to assist them with picking up small beads and stones. I’ve found that it works great for my application too! I had a concern about ESD safety since it’s not designed for electronics, but I haven’t had any problems yet.

It comes with a single tip that’s too large for picking up 0805 components, but should be OK for picking up larger things. It also has a little rubber attachment that you can put on the tip to help you get good suction on things with a large surface area. In the picture on the left, I actually have a separate tip attached that came with a different (junky) squeezable vacuum pickup tool I bought. It’s terrible, but the tip it came with is nice. You will also see in the picture that I use one of my solder paste syringe tips for picking fine components up. That blue tip is a 25-gauge needle and it works great for picking up 0805 parts. Just make sure you don’t accidentally suck up any solder paste! 0402 components are so small that I don’t think I could pick them up very reliably. I stick with tweezers for parts that small.

There is a dial on the top that you use to adjust the strength of the suction. You just have to experiment with it until you find what works. Like I said earlier, you put your finger over the hole on the pen, and then the pen should magically pick up the part. I’ve found that when you move the part onto the solder paste, the solder paste will actually grab it with enough force to snatch it from the vacuum tool. This is actually helpful because you don’t have to worry about moving your finger off of the hole when you’re ready to release the part.

I’d like to share one other related piece of advice from the video I linked above. Before I got this vacuum tool, I would take resistors/capacitors I bought from Digi-Key or Mouser or whatever out of the “cut tape” that they typically come in and dump them into little containers. With a vacuum tool, though, it’s actually beneficial to leave the components in the cut tape. With some double-sided sticky tape, you can adhere the cut tape to your workbench. Now every component will be in the exact same orientation with the correct side up. It really speeds up the process of populating the components on your board(s). The downside is that the cut tape takes up a lot of space for storage.

Here’s what the board above looked like after I placed all of the parts:

PopulatedBoard

Cooking the boards

Finally, after you have placed the components, you need some way to heat the solder paste up so it permanently solders everything to the PCB. I suppose you could do it by hand with a soldering iron, but that would be messy and you would move the parts while trying to heat the solder. You could also do it with a hot air rework station. I’ve heard that some people like to use a skillet. I prefer sticking the board into a cheapo toaster oven that I dedicated for soldering. Yeah, smoke and nasty fumes come out. Don’t cook your food in it. Preferably do this outside so the fumes don’t permeate your indoor air.

Using a regular old toaster oven instead of a specialized reflow oven presents a few challenges. Most importantly, you need some way of monitoring the temperature inside the oven to know when it’s time to quit heating. You also have to watch the oven and manually try to follow a heating profile because there’s no controller to do that for you automatically. There are temperature controller kits out there that you can use to modify a toaster oven and turn it into an automatic reflow oven. That would be nice, but it’s overkill for my needs.

To monitor the temperature, I bought a thermometer and thermocouples. The model I bought was a DM6802A+. It’s a no-name brand but it does the job. It came with two K-type thermocouples, one of which I use for soldering. You’ll see in the picture that it’s discolored, probably due to being cooked combined with the nasty crap that comes out when the solder melts and the flux activates. Anyway, I used Kapton tape (which is safe to use with high temperatures) to attach the probe to a spare PCB. I did this to try to make sure I am getting a good idea of the temperature of an actual PCB in the oven.

Thermocouple

The oven I use is a Euro-Pro TO36 convection oven, which as far as I can tell is no longer available. I’m sure you can find a cheap toaster oven somewhere if you want one. Anyway, despite the fact that this model is a convection oven, I’ve found that I have to turn the convection feature off or else the temperature has trouble getting to where I want it (around 220-230° C for solder that has lead, or 250-260° C for lead-free solder). I just use the toast function which doesn’t turn the convection fan on. It has both upper and lower heating elements, although I haven’t figured out which elements turn on in each mode–the documentation that came with the oven doesn’t do a good job of explaining it. Ideally you’d want the convection fan and both the upper and lower elements turned on. You could definitely set it up correctly if you opened it up, stripped out the original timing circuitry, and put in an automatic reflow controller instead. This oven gets the job done for me without a controller and without convection, so I haven’t had the need.

I use some spare PCBs to support the PCB I’ll actually be cooking, and of course I stick the thermocouple probe with its PCB in there too. In a real-world scenario, the red PCB I have on top is the populated PCB that I would be cooking. I would also remove the baking pan that you can see underneath; I just forgot to remove it when I took this picture.

ThermocoupleInOven

When I close the oven, the thermocouple probe sticks out and connects to the thermometer:

OvenClosed

The insulation on the thermocouple wire is good to about 260° C; if I needed to go higher than that, I would need to get a better probe that can withstand higher temperatures.

After everything is in place, I set up the oven controls. I turn the temperature knob all the way up to the maximum temperature. I put the timer knob in the “stay on” position so it doesn’t count down. As I mentioned before, I put it into toast mode. Finally, I plug it into the wall and watch as the temperature increases. I just want to repeat: I do this process outside on a sunny day. The solder paste and flux both make nasty fumes that I don’t want to smell inside.

When the temperature reaches about 10° C below my target temperature, I turn the temperature knob all the way down. It definitely overshoots, so you want to turn it down early. It’s a feel you develop over time. I believe I have discovered through my experience that the amount of overshoot can change based on the temperature outside. Once it has reached my target soldering temperature, I unplug the oven from the wall. It should do a decent job of maintaining the temperature for a while. I try to keep it at my target high temperature for 30 seconds or so. When I’m ready for the temperature to decrease, I carefully open the oven (the solder will be molten and a bump could dislodge components on the PCB) and turn on a fan that I point into it.

This is where the fumes really start coming out of the oven, so I usually walk away and wait for it to cool down.  As the “Will It Blend?” guy would say, “solder smoke, don’t breathe this!” When the thermocouple is reading 80° C or so, it has probably cooled down enough to pull the baked PCB out and inspect it:

FinalBoard

As you may be able to see, a few of the pins in the bottom row of the QFP device (an Atmel AT90USB646 AVR microcontroller, in case you’re curious) didn’t get quite enough solder. That corresponds to places where the line of solder paste was too thin. The leftmost pin in that bottom row has too much solder; that’s the place where I put too much paste. I was lucky enough to not get any bridged pins, but I definitely saw that happen on some of the other boards I built in the same batch. I had to do some touchups with my iron to fix the bridges and add extra solder in places where it looked like there wasn’t enough.

All in all, the solder joints look pretty nice. I built ten of these boards in one sitting and I’m positive that I saved a ton of time by using solder paste instead of soldering each 0805 component by hand. It took some time to bake all of the boards, but I was able to spend that time standing and walking around instead of sitting in front of my soldering workbench. However, this board has a small enough number of components that a single board would probably be easier to just do by hand.

By the way, I have successfully baked two boards at a time in my oven. It still seemed to work fine. I’d be afraid to try too many at once because I don’t know how well the heat spreads throughout the oven. Convection is supposed to help heat the entire oven uniformly, but I have convection turned off.

Cleaning up

When I’m all finished, it takes some time cleaning up and tearing down. Before letting the air out of my compressor, I put the dispensing needles onto an empty syringe and blow any remaining solder paste out of them so I can reuse them. If I really wanted to be frugal, I’d start doing this before I was finished dispensing paste to avoid wasting the paste that’s left in the needle.

Next, I let out the air. I disconnect my weird filter monstrosity from the air line and hook up the little blow gun that came with my compressor. I use this to blow most of the air out. When most of it is gone, I open up the drain valve on the bottom to let any liquid out.

After everything is depressurized, I open up my air filter and make sure to empty out anything in it. I haven’t found anything in my filter yet, and I think this is because the compressor doesn’t typically put much moisture into the air unless you’re using a lot of air and it keeps having to run.

I put away all of the air hoses and junk for connecting the compressor to the dispenser, and that’s about it!

Conclusion

Solder paste is pretty cool. Stencils are probably the way to go, but if you don’t want to use stencils because you do small quantities of many boards, a dispenser works well too. It has a bit of a barrier to entry especially if you don’t know about air compressors, but once you get the hang of it, it’s actually pretty easy. An air compressor is a good tool to have around the house anyway, so keep that in mind if you don’t have one.

Questions? Comments? Suggestions? Please let me know in the comments area below. I hope this information helps someone! Please be careful if you decide to get into this hobby. High pressures and high temperatures can both be very dangerous. Wear eye protection when you’re working with this stuff. Use common sense–don’t stick your hand into the hot oven and burn yourself, don’t blow air into your face, and make sure all air connections are nice and tight. I’m not responsible if you hurt yourself, burn your house down, or whatever else you could possibly imagine; I’m just sharing the process I use when I build PCBs.