Add ability to switch between voice & video in native matrix 1:1 calls

[nadonomy]

Your use case

What would you like to do?

When you make a call in Element, if it's a call between 2 people we use native Matrix calls. However, once the place is called, if you want to switch between voice or video you need to hang up and place a new call. It should be possible to switch between voice & video after placing a call.

How would you like to achieve it?

@SimonBrandner has offered interest in developing this on Web, pending us working on designs and gaining confidence we want the feature.

@jakewbate it'd be great if you can figure out the details on this. The things which spring to me are:

1. Any solution shouldn't break compatibility with other clients, as we can't guarantee concurrent resources on iOS & Android. Worth leaning on expertise from @dbkr to see the feasibility of this, or if we need to take a cross-platform approach..
2. We need to figure out when you can toggle between voice/video. Is it while the call is ringing? Or only after the other participant has answered?
3. We need to figure out call tiles. Could they update in real-time in the timeline if they change? Once a call finishes, should they just persist as their last known state?

Have you considered any alternatives?

No response

Additional context

No response

[SimonBrandner]

@jakewbate, feel free to DM me directly (@simon.brandner:envs.net)! I can provide any necessary info about this from the code point of view

[jakewb-b]

Based on my discussion with @SimonBrandner and @dbkr, we propose the following solution:

• A video call can already be 'downgraded' to a voice call essentially just by turning off camera, so there is no urgent requirement for any change. (This does not force the other user to also turn their camera off, but I don't think this is a greatly needed use-case, and arguably not even a desirable one)
• A voice call is essentially just a video call just with both cameras turned off. Given that, this means that it can be easily 'upgraded' to a video call by simply turning the camera on. This requires very little extra functionality or UI.
• Therefore, on Element web, using the 'voice call' button (telephone icon) would (from the user's POV) be the same as starting a video call but simply with both cameras defaulting to off.
• As the user placing a voice call, I can choose to turn my camera on while the call is ringing, but the other party's camera will still default to off. They can turn it on as soon as the call starts if they wish.
• The video call (camera icon) remains in place, and starts a video call with both cameras defaulting to on.
• Notifications will remain specific to the type of call, so the recipient knows what to expect when they answer.
• The current tiles will also remain the same, and simply reflect the type of call that was originally placed, regardless of whether cameras were turned on or off part-way through.
• For cross-client compatibility, we can simply disable the 'turn camera on' icon when receiving a voice call from a client where this is not implemented.

Ideally, this same solution will be adopted on other platforms, but this is not critical.

I will work with @gaelledel next week to review the UX and confirm any UI changes. They should be very minimal.

[SimonBrandner]

Therefore, on Element web, using the 'voice call' button (telephone icon) would in fact start a video call but simply with both cameras defaulting to off.

To be clear, from the WebRTC perspective we will still start a voice call - audio track only. But that doesn't concern the UI. From the UI perspective it will appear as what is suggested above

[nadonomy]

Therefore, on Element web, using the 'voice call' button (telephone icon) would in fact start a video call but simply with both cameras defaulting to off.

The tile could change to a more generic icon and just the text 'Call', rather than 'video' or 'voice' call removing the need to update them mid-call. However keeping the current tiles and simply leaving them in the state that the call was originally placed in might be preferable, particularly so the call recipient knows whether their camera will be on or off when they accept the call. <-- @gaelledel to input.

Interested to see how this proposal would impact push notifications too.

Anecdotally I really like to know if someone is placing an explicit voice or video call before answering. I might be somewhere inappropriate to answer with video, for instance.

[SimonBrandner]

Anecdotally I really like to know if someone is placing an explicit voice or video call before answering. I might be somewhere inappropriate to answer with video, for instance.

The difference should definitely stay visible in the notification 👍

[jakewb-b]

Therefore, on Element web, using the 'voice call' button (telephone icon) would in fact start a video call but simply with both cameras defaulting to off.
The tile could change to a more generic icon and just the text 'Call', rather than 'video' or 'voice' call removing the need to update them mid-call. However keeping the current tiles and simply leaving them in the state that the call was originally placed in might be preferable, particularly so the call recipient knows whether their camera will be on or off when they accept the call. <-- @gaelledel to input.

Interested to see how this proposal would impact push notifications too.

Anecdotally I really like to know if someone is placing an explicit voice or video call before answering. I might be somewhere inappropriate to answer with video, for instance.

Thanks. The more I think about it, the more I'm leaning towards keeping the current tiles and notifications as they are, so you know what is incoming and what will happen when you press accept.

With this, there is the potential that I start a voice call, then switch my camera on while it is ringing, so the notification for you is still 'voice call' and when you answer you might be surprised to see my video on. However, your video is still off, so I think while this may be a little unexpected it shouldn't be a great concern. If it is, we could just prevent the user from enabling video while the call is ringing.

To be clear, from the WebRTC perspective we will still start a voice call - audio track only. But that doesn't concern the UI

@SimonBrandner - thanks! I updated that line just to make it more clear.