USB Passthrough into Podman Machine (QEMU)

[hftsai256]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind feature

Description

We are deploying cross-platform application which talks to scientific apparatus over USB (to name one, Keysight Technologies: DSOX1202A [2a8d:0387]) to aid our R&D activities. For now I assume I could add a few lines to NewMachine method in pkg/machine/qemu/machine.go in order to pass in additional arguments such as -device usb-host,vendorid=2a8d,productid=0387. However I would like to know if there is a better solution, or would like to kindly ask if adding such a feature to configure podman machine is feasible.

While I'm trying on my Intel Mac, other hosting machines could be MacOS (Intel/M1), Linux, Windows 10/11.

I wouldn't expect passing --device argument through podman run would work, and by doing so it will return Error: stat /dev/<device_name>: no such file or directory.

Output of podman version:

Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.18.8
Built:        Wed Nov  9 15:43:58 2022
OS/Arch:      darwin/amd64

Server:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.19.2
Built:        Fri Nov 11 10:01:27 2022
OS/Arch:      linux/amd64

Output of podman info:

host:
  arch: amd64
  buildahVersion: 1.28.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.5-1.fc37.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.5, commit: '
  cpuUtilization:
    idlePercent: 99.81
    systemPercent: 0.15
    userPercent: 0.04
  cpus: 4
  distribution:
    distribution: fedora
    variant: coreos
    version: "37"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 6.0.9-300.fc37.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 3719778304
  memTotal: 4107595776
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.7-1.fc37.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.7
      commit: 40d996ea8a827981895ce22886a9bac367f87264
      rundir: /run/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-8.fc37.x86_64
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 36m 12.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 2381983744
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 1668178887
  BuiltTime: Fri Nov 11 10:01:27 2022
  GitCommit: ""
  GoVersion: go1.19.2
  Os: linux
  OsArch: linux/amd64
  Version: 4.3.1

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

==> podman: stable 4.3.1 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/4.3.1 (185 files, 48MB) *
  Poured from bottle on 2022-12-01 at 11:56:22
From: /~https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0 and GPL-3.0-or-later
==> Dependencies
Build: go-md2man ✘, go@1.18 ✘
Required: qemu ✔
==> Options
--HEAD
        Install HEAD version
==> Caveats
zsh completions have been installed to:
  /usr/local/share/zsh/site-functions

To restart podman after an upgrade:
  brew services restart podman
Or, if you don't want/need a background service you can just run:
  /usr/local/opt/podman/bin/podman system service --time=0
==> Analytics
install: 30,145 (30 days), 79,965 (90 days), 239,310 (365 days)
install-on-request: 27,626 (30 days), 75,590 (90 days), 233,900 (365 days)
build-error: 20 (30 days)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

[arixmkii]

If you want to just add something to QEMU launch configuration - there is an option to edit machine config (json file) as it gives you to edit the full command line, which start QEMU.

[Luap99]

Possibly duplicate of #14409?

[hftsai256]

In the end of the day they falls into the same category: USB passthrough

After modifying the machine configuration I could have them pop out in lsusb command such as

Bus 002 Device 003: ID 0403:6015 Future Technology Devices International, Ltd Bridge(I2C/SPI/UART/FIFO)
Bus 002 Device 002: ID 0403:6014 Future Technology Devices International, Ltd FT232H Single HS USB-UART/FIFO IC

Kernel dose recognize them when plugged in (observed over udevadm monitor inside the container):

root@7995e227a75f:/etc/udev/rules.d# udevadm monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

KERNEL[604.069710] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1 (usb)
KERNEL[604.083077] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0 (usb)
KERNEL[604.083374] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1 (usb)
KERNEL[604.103966] add      /bus/usb/drivers/ftdi_sio (drivers)
KERNEL[604.104083] add      /bus/usb-serial/drivers/ftdi_sio (drivers)
KERNEL[604.104167] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/ttyUSB0 (usb-serial)
KERNEL[604.112048] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/gpiochip0 (gpio)
KERNEL[604.112195] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/gpiochip0 (gpio)
KERNEL[604.112674] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/ttyUSB0/tty/ttyUSB0 (tty)
KERNEL[604.113093] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0/ttyUSB0 (usb-serial)
KERNEL[604.113587] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1:1.0 (usb)
KERNEL[604.113713] add      /module/ftdi_sio (module)
KERNEL[605.120240] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-2 (usb)
KERNEL[605.138832] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0 (usb)
KERNEL[605.139662] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0/ttyUSB1 (usb-serial)
KERNEL[605.145848] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0/gpiochip1 (gpio)
KERNEL[605.146014] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0/gpiochip1 (gpio)
KERNEL[605.146110] add      /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0/ttyUSB1/tty/ttyUSB1 (tty)
KERNEL[605.146200] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0/ttyUSB1 (usb-serial)
KERNEL[605.146493] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-2/2-2:1.0 (usb)
KERNEL[605.146571] bind     /devices/pci0000:00/0000:00:1d.0/usb2/2-2 (usb)

or dmesg in podman machine ssh:

[ 1694.331194] usb 2-1: new full-speed USB device number 4 using uhci_hcd
[ 1694.475802] usb 2-1: not running at top speed; connect to a high speed hub
[ 1694.495279] usb 2-1: New USB device found, idVendor=0403, idProduct=6014, bcdDevice= 9.00
[ 1694.495284] usb 2-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 1694.495286] usb 2-1: Product: Single RS232-HS
[ 1694.495287] usb 2-1: Manufacturer: FTDI
[ 1694.510236] ftdi_sio 2-1:1.0: FTDI USB Serial Device converter detected
[ 1694.510259] usb 2-1: Detected FT232H
[ 1694.518140] usb 2-1: FTDI USB Serial Device converter now attached to ttyUSB0
[ 1695.519127] usb 2-2: new full-speed USB device number 5 using uhci_hcd
[ 1695.701858] usb 2-2: New USB device found, idVendor=0403, idProduct=6015, bcdDevice=10.00
[ 1695.701866] usb 2-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1695.701869] usb 2-2: Product: FT230X Basic UART
[ 1695.701871] usb 2-2: Manufacturer: FTDI
[ 1695.701873] usb 2-2: SerialNumber: D3089623
[ 1695.717742] ftdi_sio 2-2:1.0: FTDI USB Serial Device converter detected

However I couldn't set up those udev rules and talk to them over serial or GBIP/IEEE 488 inside the container. (I assume I could do this directly in the podman machine, because /dev/ttyUSB{0,1} exist)

SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="601?", GROUP="dialout", MODE="0660"
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="0957", ATTRS{idProduct}=="17??", GROUP="dialout", MODE="0660"
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="2391", ATTRS{idProduct}=="????", GROUP="dialout", MODE="0660"
SUBSYSTEMS=="usb", ACTION=="add", ATTRS{idVendor}=="2A8D", ATTRS{idProduct}=="????", GROUP="dialout", MODE="0660"

Probably some more work to be done inside the podman machine?

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@baude PTAL

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@baude PTAL

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@ashley-cui any chance you can look at this?

[beriberikix]

Has anyone taken another look at this? I've been doing my own experiments here, still no luck.

[hftsai256]

I haven't actually tried, but I wonder if usbip is the way to go in such situations.

[beriberikix]

USB/IP could work for some cases (that's how WSL does it) but not in the general case. And USB passthrough has been well supported in QEMU for ages!

[victortoso]

Hi, I have been doing similar work to KubeVirt kubevirt/kubevirt#10015. I could help out here as well later. Is anyone working on this already?

[beriberikix]

I've paused experiments in this area but would love to test if you work on it!

[baude]

i would think long-term it would be best to allow an option for usb passthrough; however, we support WSL, HyperV, and Apple's HV so the option would ideally work across the board (though I wouldn't mind if initially it was a qemu only option).

[beriberikix]

In general that makes sense. However, having looked a bit into those various targets, the implementation details and even deployment (ex. Apple code signing) vary a lot. Starting with QEMU and figuring out how to generalize does make sense, though.

[beriberikix]

@victortoso did you end up looking at this?

[victortoso]

@beriberikix thanks for the ping and sorry, I haven't had the time yet but it is on my plan for the next two weeks. Cheers!

[beriberikix]

Thanks, no worries at all! Still happy to test when you need testers!

[beriberikix]

@victortoso hope all is well! Did this make it into your priority queue? I'm sure you are busy, I'd love to use this feature as part of an upcoming project! 🙏

[ashley-cui]

I'm taking a look at this now, but if @victortoso you have any progress or would like to work on it together, please let me know!

[victortoso]

Hi @ashley-cui and @beriberikix , sorry, I've been lagging behind due other work I've been doing in qemu/kubevirt.
If you are already working on it, feel free to take it, otherwise I'd work on it Today & Tomorrow.

[mheon]

Can we close this given #20540 is merged?

[rhatdan]

Closing, reopen if this was a mistake.