linux-pipewire: Add explicit sync support

[dkorkmazturk]

Description

This change leverages PipeWire's ability to share DRM syncobj fds to implement an explicit synchronization solution.

Motivation and Context

It is possible for OBS to receive incomplete frames from the compositor if the graphics drivers in use do not support implicit synchronization (e.g. NVIDIA drivers). The compositors usually try to workaround this issue by waiting all render operations to complete before sharing frames with OBS. However, this has performance implications for the compositors. Similarly, OBS should also make sure that all the rendering operations sourcing the received buffer are complete before returning the respective pw_buffer back to the compositor, or it would risk the image it is working on getting overwritten without some type of synchronization.

How Has This Been Tested?

• Recorded the screen using the "Screen Capture (PipeWire)" option as the source on a Wayland session on both GNOME 47.3 and KDE 6.3 Beta, which support explicit sync for screencasting. Verified that explicit sync paths were in use by attaching a debugger to OBS.
• Recorded the screen using the "Screen Capture (PipeWire)" option as the source on Sway 1.10 to make sure that this change does not introduce regressions for the compositors that do not support PipeWire explicit sync.

Types of changes

• Bug fix (non-breaking change which fixes an issue)

• Performance enhancement (non-breaking change which improves efficiency)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[kkartaltepe]

Please work with upstream pipewire and compositors to document the pipewire protocol and how to correctly handle explicit sync. (google doesnt bring up anything for SPA_META_SyncTimeline)

First two commits seem ok for adding fence support at first glance.

[dkorkmazturk]

I believe it has already been documented by upstream here: https://docs.pipewire.org/page_dma_buf.html

[kkartaltepe]

I believe it has already been documented by upstream here: https://docs.pipewire.org/page_dma_buf.html

Thanks.

What compositor other than gnome was involved, do they have a tentative implementation this can be tested against? Are any other clients like chromium already onboard with this protocol? Basically just like when we originally implemented pipewire we dont want to include something that will only work for one compositor or risk changing dramatically when other users more important than us request changes to the protocol that break our implementation.

Also, how does cross device dmabuf sharing work in this protocol, right now it seems the fences are all assumed to be on a single device if I understand the drm calls correctly (or can the fences be imported across device?). Cross device issues are already coming up a fair bit in the current pipewire dmabuf protocol unfortunately...

[dkorkmazturk]

What compositor other than gnome was involved, do they have a tentative implementation this can be tested against?

KWin has an implementation that should be released with KWin 6.3: https://invent.kde.org/plasma/kwin/-/merge_requests/6204

Are any other clients like chromium already onboard with this protocol? Basically just like when we originally implemented pipewire we dont want to include something that will only work for one compositor or risk changing dramatically when other users more important than us request changes to the protocol that break our implementation.

I believe GNOME and KDE implementations should cover the majority of the Linux users.

So far, the only client that I know that takes advantage of explicit sync on PipeWire is NvFBC (NVIDIA's proprietary screen recording solution).

I am not sure if the explicit sync logic can really dramatically change. The only change required on the PipeWire side was the addition of SPA_META_SyncTimeline meta type. The DRM syncobj mechanism used for implementing this feature is also used by Wayland's linux-drm-syncobj-v1 protocol, which is implemented by both Mesa and NVIDIA drivers.

Also, how does cross device dmabuf sharing work in this protocol, right now it seems the fences are all assumed to be on a single device if I understand the drm calls correctly (or can the fences be imported across device?). Cross device issues are already coming up a fair bit in the current pipewire dmabuf protocol unfortunately...

DRM syncobjs should be importable across devices and it does not change how dmabufs are shared. I may be able to test these changes in a multiple device configuration later.

[dkorkmazturk]

I did some more testing on KDE 6.3 Beta and Sway 1.10. Updated the description with the details.

[dkorkmazturk]

Fixed a bug that was causing OBS to signal the wrong syncobj. This was not caught during testing because both GNOME and KDE use one syncobj per image for both acquire and release tracking with duplicated syncobj file descriptors.

[kkartaltepe]

flush will happen once all the drawing is queued anyway, can we avoid introducing this in the middle of the frame?

[dkorkmazturk]

I don't think that it is avoidable with the current design of libobs. The EGL spec says:

... the next Flush() operation performed by the current client API causes a new native fence object to be created, and the EGL_SYNC_NATIVE_FENCE_FD_ANDROID attribute of the EGL native fence object is set to a file descriptor that refers to the new native fence object.

And that file descriptor is accessed in gs_sync_export_syncobj_timeline_point() when the eglDupNativeFenceFDANDROID() function is called.

[kkartaltepe]

It looks like the syncobj work in gnome is now in fedora 41 so I tested this a little bit on my low spec machine. The stalls from waiting for the acquire fence and creating the release fence during drawing are kind of brutal, and it seems to tickle gnome+intel causing significantly more stalls in other parts of the wsi than when disabling explicit sync during negotiation. Below is a particularly bad frame, but basically p90 cpu time is up 10x on my machine compared to implicit sync.

Ill try and test with gnome+amd later, which is more of a typical machine.