Multiple issues when trying to get CGImage from video file

[tipa]

My goal is to get a thumbnail image from a video file and I was using AVAssetImageGenerator.CopyCGImageAtTime for this.
Now I experienced crashes coming from that method and while investigating, I noticed that the method deprecated.

Problem 1: This method is not marked deprecated in the bindings, see: /~https://github.com/xamarin/xamarin-macios/blob/2761123a6bf6dd8eed5dd91ac85f969711f38114/tests/xtro-sharpie/iOS-AVFoundation.todo#L9

So according to Apple one should use the image(at:) method (which is Swift only so there are no .Net bindings for it) or the generateCGImagesAsynchronouslyForTimes:completionHandler: method.

Problem 2: This method is not bound at all, see: /~https://github.com/xamarin/xamarin-macios/blob/2761123a6bf6dd8eed5dd91ac85f969711f38114/tests/xtro-sharpie/iOS-AVFoundation.todo#L72

Ok, then I try the generateCGImagesAsynchronouslyForTimes:completionHandler: method instead, which is bound.

Problem 3: This method seems to be bound twice, which is rather confusing:

Ok, but I try it, like this:

imgGenerator.GenerateCGImagesAsynchronously(new[] { NSValue.FromCMTime(new CMTime(0, 1)) }, (requestedTime, imageRef, actualTime, result, error) =>
{
    CGImage c = imageRef;
});

Problem 4: When I run this method with a bunch of files (in rather quick succession), it crashes quite often:
Visual Studio logs this crash report in the Output window:

2023-06-15 12:58:28.080 MyApp[40044:959378] Microsoft.macOS: Aborting due to trying to marshal managed exception:
Could not initialize an instance of the type 'CoreGraphics.CGImage': handle is null.
It is possible to ignore this condition by setting ObjCRuntime.Class.ThrowOnInitFailure to false. (System.Exception)
   at ObjCRuntime.DisposableObject.InitializeHandle(NativeHandle handle, Boolean verify)
   at CoreFoundation.NativeObject..ctor(NativeHandle handle, Boolean owns, Boolean verify)
   at ObjCRuntime.Trampolines.SDAVAssetImageGeneratorCompletionHandler2.Invoke(IntPtr block, CMTime requestedTime, NativeHandle imageRef, CMTime actualTime, IntPtr result, NativeHandle error)

It looks like this crash happens when the thumbnail retrieval failed and IntPtr imageRef==0 (and the error parameter contains the retrieval error).
I seem to be able to workaround the crash by using the other binding method that returns the IntPtr instead of the CGImage in the callback, but I am not sure if to specify "true" or "false" for the owns parameter when constructing the CGImage object:

if (imageRef != 0) { var cgImg = ObjCRuntime.Runtime.GetINativeObject<CGImage>(imageRef, *true or false?*); }

Environment

Version information

Visual Studio Community 2022 for Mac Preview
Version 17.6 Preview (17.6 build 1569)
Installation UUID: 04789853-9f3e-4547-ab61-5d65b01a0fa4

Runtime
.NET 7.0.3 (64-bit)
Architecture: Arm64
Microsoft.macOS.Sdk 13.1.1007; git-rev-head:8afca776a0a96613dfb7200e0917bb57f9ed5583; git-branch:release/7.0.1xx-xcode14.2

Roslyn (Language Service)
4.6.0-3.23180.6+99e956e42697a6dd886d1e12478ea2b27cceacfa

NuGet
Version: 6.4.0.117

.NET SDK (Arm64)
SDK: /usr/local/share/dotnet/sdk/7.0.304/Sdks
SDK Versions:
	7.0.304
	7.0.302
	7.0.203
	7.0.202
	7.0.201
MSBuild SDKs: /Applications/Visual Studio (Preview).app/Contents/MonoBundle/MSBuild/Current/bin/Sdks

.NET Runtime (Arm64)
Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	7.0.7
	7.0.5
	7.0.4
	7.0.3

Xamarin.Profiler
'/Applications/Xamarin Profiler.app' not found

Updater
Version: 11

Apple Developer Tools
Xcode: 14.3.1 21815
Build: 14E300c

Xamarin.Mac
Version: 9.3.0.6 Visual Studio Community
Hash: 97731c92c
Branch: xcode14.3
Build date: 2023-04-11 22:38:35-0400

Xamarin.iOS
Version: 16.4.0.6 Visual Studio Community
Hash: 97731c92c
Branch: xcode14.3
Build date: 2023-04-11 22:38:36-0400

Xamarin Designer
Version: 17.6.3.9
Hash: 2648399ae8
Branch: remotes/origin/d17-6
Build date: 2023-06-07 19:09:46 UTC

Xamarin.Android
Not Installed

Microsoft Build of OpenJDK
Java SDK: Not Found

Eclipse Temurin JDK
Java SDK: Not Found

Android SDK Manager
Version: 17.6.0.50
Hash: a715dca
Branch: HEAD
Build date: 2023-06-07 19:09:51 UTC

Android Device Manager
Version: 0.0.0.1309
Hash: 06e3e77
Branch: HEAD
Build date: 2023-06-07 19:09:51 UTC

Build Information
Release ID: 1706001569
Git revision: c51ed514fc64c7e298fdb6e199ffec2133ae500c
Build date: 2023-06-07 19:07:58+00
Build branch: release-17.6
Build lane: release-17.6

Operating System
Mac OS X 13.4.0
Darwin 22.5.0 Darwin Kernel Version 22.5.0
    Mon Apr 24 20:53:44 PDT 2023
    root:xnu-8796.121.2~5/RELEASE_ARM64_T8103 arm64

[rolfbjarne]

Problem 1: This method is not marked deprecated in the bindings

Correct, I'll fix this.

So according to Apple one should use the image(at:) method (which is Swift only so there are no .Net bindings for it) or the generateCGImagesAsynchronouslyForTimes:completionHandler: method.

Problem 2: This method is not bound at all

Not quite correct, but it's confusing.

We've bound generateCGImagesAsynchronouslyForTimes:completionHandler: - and that's what Apple recommends to use.

We haven't bound generateCGImageAsynchronouslyForTime:completionHandler: - which is not what Apple is recommending.

In any case, I'm binding the latter anyways now.

Problem 3: This method seems to be bound twice, which is rather confusing:

Yes, that was to fix #16314 to make the API easier to use. We'll remove one of those overloads the next time we can do breaking changes.

Problem 4: When I run this method with a bunch of files (in rather quick succession), it crashes quite often

It looks like this crash happens when the thumbnail retrieval failed and IntPtr imageRef==0 (and the error parameter contains the retrieval error).

I think I see the problem here. I'll have a look.

I seem to be able to workaround the crash by using the other binding method that returns the IntPtr instead of the CGImage in the callback, but I am not sure if to specify "true" or "false" for the owns parameter when constructing the CGImage object:

Use false in this case.

[rolfbjarne]

Closing, since all the relevant fixes have been merged.