USB Proxy Service

Date: 2024/06/28

Context

For basic communication, the Spark 2 and 3 support USB connections. The Spark 4 has phased out USB in favor of ethernet. Future Spark models are not expected to re-introduce USB, as we want to make use of the greater physical flexibility offered by TCP/IP networking.

The Spark Cbox protocol is a message-based protocol. Message separators are defined to support its use in serial streams. While the Spark 2 and 3 support USB connections, this should be thought of as an alternative way to implement a serial stream. The actual USB communication is an implementation detail.

Because it has to support USB connections, the Spark service faces limitations and vulnerabilities. For it to communicate with any USB serial device, the Docker container must run in privileged mode. For it to detect newly connected devices, it must mount the host /dev directory.
The first restriction is unavoidable, the second is unacceptable, as it gives the container unrestricted access to all connected hard drives and devices.
A workaround is to periodically restart the container when it fails to discover the desired Spark controller.

For these reasons, it is desirable to isolate USB connectivity by creating a service that acts as USB-TCP bridge. The Spark service is then no longer burdened by the limitations of legacy controller models, and can leverage TCP/IP networking to enable more flexible container deployment.

Features

A proxy service must implement three required features:

• It must discover Spark USB devices.
• It must create a bridge between a bound TCP port and the Spark USB serial device.
• It must offer a discoverability mechanism so client Spark devices can find and use the correct TCP port.

USB Discovery

On Linux hosts, we can perform serial device discovery by iterating over the /sys/class/tty/ directory. Spark 2 and 3 controllers present themselves to the host as as USB devices that implement the Abstract Control Model (ACM). They will be assigned a ttyACM{0-256} slot. There's a lot of history behind USB and ACM, but the important part is simple: you can open /dev/ttyACM{0-256} as a serial stream.

The directory tree in /sys/class/tty/ involves a lot of symlinks for connected devices. By following these symlinks, we can find the relevant metadata for our devices: USB Vendor ID (vid) and Product ID (pid), and device serial number (used as device ID).

To discover Spark 2 / 3 devices in Python:

from glob import glob
from pathlib import Path

for tty in glob('/sys/class/tty/ttyACM*'):
    tty = Path(tty)

    if (tty / 'device' / 'subsystem').resolve() != Path('/sys/bus/usb'):
        continue

    dev_root = (tty / 'device').resolve() / '..'
    usb_vid = (dev_root / 'idVendor').read_text().strip()
    usb_pid = (dev_root / 'idProduct').read_text().strip()
    usb_device_id = (dev_root / 'serial').read_text().strip()

    # Particle vid = 2b04
    # Photon pid = c006
    # P1 pid = c008
    if usb_vid != '2b04' or usb_pid not in ['c006', 'c008']:
        continue

    print(tty)
    print('\t', f'{usb_vid}:{usb_pid}')
    print('\t', usb_device_id)
    print('\t', '/dev/' + tty.name)

Bridging

The simplest way to implement a USB-TCP bridge is to use socat. It does exactly what we want, and is widely available on Linux. If we found a Spark device on /dev/ttyACM0, and wanted to bind its serial stream to port 9000, we can do so with the following command:

socat \
  tcp-listen:9000,reuseaddr,fork \
  file:/dev/ttyACM0,raw,echo=0,b115200

Now, connecting to localhost:9000 will be the equivalent of opening a serial stream to /dev/ttyACM0 with 115200 baud rate.

Proxy Discovery

If we have a service that bridges USB serial devices to TCP ports, then the natural mechanism for listing these devices and ports is also TCP. The simplest solution is to have a REST server that can be queried for an index.

Implementation

The desired features are not complex, and require little in terms of external libraries. A Python application based on the Brewblox boilerplate would result in a 300MB+ image size. When using C++, we can use a 5MB Alpine base image. Our dependencies are limited to socat and a REST server library of choice.

Crow has the advantages of being small, and being available as a header-only library. The latter is desirable because our use of Alpine means we are building for musl instead of glibc.

With this, we have implementations for all three desired features:

• USB discovery is done using std::filesystem
• Bridging is done using socat subprocesses, started with posix_spawn(3)
• Proxy discovery is done through a REST endpoint, implemented with Crow.

The end result is a single-file application and a 15MB image.

Configuration

The new usb-proxy service must be added as a Compose service to be used. By nature, it is a system service, and should be added to docker-compose.shared.yml if enabled. The proxy service does not require any settings in brewblox.yml apart from the enabled flag.

The default value for enabled will be false. The Spark 2 and 3 are legacy models, and of those who use them, only a subset want or need USB connectivity.

Future Work

The existence of a USB-TCP bridge makes it optional for the usb-proxy service to be on the same host as the Spark service. In the initial implementation, the Spark service handles USB discovery by querying the hardcoded usb-proxy address. If desired, we can extend the Spark service configuration to accept custom addresses for the proxy service, or we can let the proxy service publish its address using MQTT.

For now, this is not implemented, as there does not seem to be any appreciable demand. Physical separation between the Spark controller and service can also be handled by using wifi/ethernet instead of USB. This is one of the reasons why we prefer ethernet over USB for wired connections where possible.