net7.0-ios binding nuget can't be consumed from Windows

[LeadAssimilator]

VS 2022 17.7.4 w/ 16.4.7098 workload

When adding a reference to a net7.0-ios binding nuget to a maui project targeting net7.0-ios on Windows, the Assembly.resources.zip attempts to be unpacked on Windows via /usr/bin/unzip which of course will never work unless Windows magically turns itself into a Mac.

Even if it Windows could unzip it, it shouldn't because if the package contains frameworks, the paths can be too long or contain symlinks - so it needs to remain a zip that gets unpacked on the Mac only.

The nuget binding project was built on a Mac with TargetFramework net7.0-ios with a NativeReference to either an xcframework or a framework and it always ends up being packed with an Assembly.resources.zip regardless of NoBindingEmbedding or CompressBindingResourcePackage. If I switch to net6.0-ios and to the latest 6.0.414 sdk and corresponding ios workloads, the properties are respected again - consuming the package from Windows still fails however if there is an Assembly.resources.zip (NoBindingEmbedding=true).

Either the way to make and consume ios binding nugets changed that I don't see documented somewhere or net7 broke everything.

[rolfbjarne]

net7 broke

It seems like this is what happens, because nothing should have changed.

Are you connected to a Mac, or are you using Hot Restart?

In either case, could you get a binlog for the failed build?

[microsoft-github-policy-service]

Hi @LeadAssimilator. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[LeadAssimilator]

Paired Mac. Hot Restart is disabled.

Here is the buildlog with the failed unzip on Windows TestMauiApp_Debug_AnyCPU_Build_2023-09-14T11_49_42.7933929-04_00.zip

I also tried to get a build log from the Mac showing the NoBindingEmbedding not being respected when building the binding project nuget, but it started working after having installed latest net6 sdk/workloads despite still using the net7 toolchain. That doesn't sound like it should have fixed anything, but I'll see if I can trash all nets and workloads at some point and try again. I'll report that as a separate issue if I can repro it.

[LeadAssimilator]

Even with NoBindingEmbedding=false, the build of the consuming project fails with undefined symbols in registrar.o. For whatever reason some symbols from the binding project are ending up in registrar.o and the compile cmdline including registrar.o is missing the framework link flags. If I add the framework link flags to the clang++ cmdline as logged by the build, it works, which confirms there are no problems with ApiDefinitions mismatching the framework and no problems with the framework itself missing other frameworks or symbols.

TestMauiApp_Debug_AnyCPU_Build_2023-09-15T01_17_18.0555775-04_00.zip

Not sure if you want that as a separate issue, but both are similar results - build from windows to a paired mac of ios targets fails when trying to consume a nuget binding package from a maui app.

[rolfbjarne]

Is the native library in question a .framework or an .xcframework?

In any case I think I see what's happening, it's a code change in one place uncovered a latent bug somewhere else :/

[LeadAssimilator]

.xcframework

I tried .framework with just the device bits and it appears to work, but that isn't a viable workaround unless there is some way to get both the arm64 device and simulator bits to coexist outside of an xcframework that I'm not aware of.

[rolfbjarne]

I can reproduce this, it happens when the binding resource package (the *.resources[.zip] next to the binding assembly) is compressed (i.e. a. *.resources.zip file).

We automatically create a zipped version of the *.resources directory if there's a symlink in there, which typically only happens for macOS or Mac Catalyst, not iOS nor tvOS.

So if the NuGet only targets iOS (or tvOS), then a potential workaround would be to remove any macOS/Mac Catalyst-specific files from the xcframework (thus removing the symlinks).

Alternatively it's possible to keep the *.resources directory by adding this to the binding project:

<CompressBindingResourcePackage>false</CompressBindingResourcePackage>

I'm not sure what will happen when the build tries to restore/extract a nupkg with symlinks inside on Windows though.

[LeadAssimilator]

Ok so that explains the behavior I was seeing when trying to make the binding nugets. Unfortunately we need support for ios with simulators on all archs and maccatalyst which has symlinks, and be able to use them from Windows to build on a paired mac. I'm not sure about the latest versions yet, I will have to test them, but it used to be that if Windows had the symlinks, they would not be preserved properly when copying the files to the mac (I'm assuming Ditto or whatever does the copy was handling them as normal files rather than symlinks) which leads to code signing failures on the mac during build. It would be great if that could be fixed so the symlinks would be properly replicated on the mac, but it doesn't fix the path too long problems that are also typical with extracted xcframeworks on Windows - so it is kind of best they remain zipped or embedded for maximum compatibility for package consumers.

The biggest issue at the moment is still the fact that binding nugets built with either NoBindingEmbedding true or false (when excluding maccatalyst) both fail with build errors from Windows to a paired mac with either the failed unzip or the missing symbols per the attached binlogs above. I'm assuming some things got broken in the msbuild targets, so if you have any edit/overrides/workarounds there to try, I'd be happy to give them a go.

[rolfbjarne]

Note that we've never supported NoBindingEmbedding=false with xcframeworks (this has never worked), I believe that's the problem you're running into with the build errors about missing symbols.

A potential workaround could be to create two separate NuGets, one for iOS and one for Mac Catalyst, and put the .xcframework without the native entries for the platform that doesn't apply in each case in each.

This should make the iOS build work from Windows, and Mac Catalyst doesn't build on Windows anyway, so that shoulnd't be a problem.

[LeadAssimilator]

I'm surprised that still isn't supported. It shouldn't be too difficult to make it work. I only tried NoBindingEmbedding=false because true wasn't working though.

I don't see how your proposed workaround would help, because we need both arm64 dev and sim bits to exist together for ios and pushing maccatalyst into a separate package won't affect that. That means we need to use an .xcframework, but when NoBindingEmbedding=true, build fails from Windows to paired Macs with the unzip error, and =false yields the symbol error, which you said isn't supported.

And maccatalyst does build from Windows normally. You just can't run/debug...

[rolfbjarne]

I don't see how your proposed workaround would help

<ItemGroup>
    <PackageReference Include="MyNuGet.iOS" Condition="$(TargetFramework.EndsWith('ios'))" Version="1.0.0" />
    <PackageReference Include="MyNuGet.MacCatalyst" Condition="$(TargetFramework.EndsWith('maccatalyst'))" Version="1.0.0" />
</ItemGroup>

Then in MyNuGet.iOS you add an .xcframework with only the iOS bits (which doesn't have any symlinks, and as such won't be compressed), and in MyNuGet.MacCatalyst you add an .xcframework with only the Mac Catalyst bits (which may contain symlinks).

The build will work from Windows when targeting iOS, because you avoid the unzip problem. It will also work from Mac when targeting Mac Catalyst.

And maccatalyst does build from Windows normally. You just can't run/debug...

It's not a complete build, because it will only compile the managed code. Anything that needs a Mac (such as creating a native executable, or linking native libraries) won't be done.

[LeadAssimilator]

Right ok, I misread, sorry. That does indeed avoid the unzip problem, but then we run into path too long.

[rolfbjarne]

Right ok, I misread, sorry. That does indeed avoid the unzip problem, but then we run into path too long.

My next idea is even uglier... have separate NuGets per runtime identifier, and put the corresponding .framework (.not .xcframework) in the NuGet. This should save quite a few characters off the path length.

<ItemGroup>
    <PackageReference Include="MyNuGet.iOSSimulator" Condition="$(RuntimeIdentifier.StartsWith('iossimulator-'))" Version="1.0.0" />
    <PackageReference Include="MyNuGet.iOSDevice" Condition="$(RuntimeIdentifier.StartsWith('ios-'))" Version="1.0.0" />
</ItemGroup>

In any case I'm fixing this, and we'll hopefully get the fix in the next service release.

[LeadAssimilator]

Interesting...uglier yes, but it might reduce bloat. Or I have to do something similar to what I'm doing for legacy xamarin.ios10 to include of a zip of the xcframeworks myself with nuget build targets that ditto to mac, unzip and select the right arch for linking - it's pretty ugly too.

Thanks for fixing this!