Global tools targeting netcoreapp2.0 do not work with preview2 SDK

[spboyer]

Steps to reproduce

• Create dotnet global tool dotnet new global-tool -n my-tool --command-name go
• dotnet pack --ouput ./
• install dotnet tool install -g my-tool --source ./
• run go

Error in latest SDK (see --version below)
A fatal error was encountered. The library 'hostpolicy.dll' required to execute the application was not found in 'C:\Program Files\dotnet'.

same on macOS
A fatal error was encountered. The library 'libhostpolicy.dylib' required to execute the application was not found in '/usr/local/share/dotnet'.

Environment data

dotnet --info output:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.300-preview2-008530
 Commit:    822ae6d43a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.17133
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\2.1.300-preview2-008530\

Host (useful for support):
  Version: 2.1.0-preview2-26406-04
  Commit:  6833f3026b

[livarcocc]

@spboyer I am not familiar with this global-tool template. Can you share its contents? Also, in preview2, if you are not using a default location (one set through a native installer), you need to set DOTNET_ROOT, pointing to the root of your dotnet installation. This is so that the tool shim can find a shared framework to run against.

[spboyer]

This is the template from @natemcmaster - dotnet new --install "McMaster.DotNet.GlobalTool.Templates::2.1.300-preview1"
/~https://github.com/natemcmaster/DotNetGlobalTool

I did't purposely set an installation location outside of the default for dotnet (clicked next, next, next :-))

[livarcocc]

@wli3 @peterhuene can you guys check the template above and make sure it is correct for preview2?

[wli3]

It is not correct for preview 2 anymore. It is targeting netcoreapp2.0 which does not have new AppHost support(able to find runtime on programfiles).

[natemcmaster]

So, are you saying that global tools don't work for console apps that target netcoreapp2.0?
If so, that seems like either a bug or a serious design flaw.

FYI the templates should produce a project that effectively looks like this.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <PackAsTool>true</PackAsTool>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.0</TargetFramework>
  </PropertyGroup>
</Project>

[natemcmaster]

I played with netcoreapp2.0, and it seems this is completely busted. I'm re-opening as it seems like either:

• the SDK should fail when TargetFramework != netcoreapp2.1 and PackAsTool == true
• fix global tools so they work for all versions of netcoreapp.

Repro:

dotnet new console -o testclitool
cd testclitool
dotnet pack -o pkgs -p:PackAsTool=true -p:TargetFramework=netcoreapp2.0
dotnet tool install --tool-path $(pwd)/.tools --source-feed $(pwd)/pkgs testclitool

Error

Failed to locate managed application [/private/tmp/testclitool/.tools/testclitool.dll]

[livarcocc]

I think an error on Pack when PackAsTool is set and TFM is less than netcoreapp2.1 seems reasonable. Your second option is not really feasible as we depend on the framework exe feature built in the runtime in netcoreapp2.1.

[livarcocc]

@wli3 @peterhuene can you take a look?

[peterhuene]

I think an error on pack is reasonable as well.

[wli3]

@livarcocc I can add check on Pack to prevent TFM is less than netcoreapp2.1 . So i will target this work for release 2.1.3xx.

@KathleenDollard I think we have a communication issue on the breaking of netcoreapp2.0.

[wli3]

So the action item is to add warning for PackAsTool

[KathleenDollard]

I agree that we should add this warning on pack

[seesharper]

How come this worked in preview 1?

The following package installs and runs fine on preview 1

https://www.nuget.org/packages/dotnet-script/

[filipw]

I would like to +1 this - this issue has broken /~https://github.com/filipw/dotnet-script/ as well, which is targeting netcoreapp2.0 as well.

With preview1, it was generating dotnet-script.exe.config in the .dotnet/tools folder upon installation.
Now with preview2, a dotnet-script.startupconfig.json is being generated.

It seems that the problem is with that appRoot path found in that startup JSON file, as it is not being respected by the generated shim EXE.
Instead of using the path to find dependency DLLs, it searches for them in the current folder, hence the error mentioning a DLL under the current path:

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'C:\Users\xxx\.dotnet\tools\dotnet-script.dll'. The system cannot find the file specified.

If I copy the generated dotnet-script.exe into the tool package folder (so C:\Users\xxx\.dotnet\tools\.store\dotnet-script\0.21.0\dotnet-script\0.21.0\tools\netcoreapp2.0\any, then the shim works.

If possible, it would be great if this could be fixed rather than simply communicated as a breaking change.

[KathleenDollard]

@seesharper , @filipw
There were several changes between Preview 1 and Preview 2 with how Global Tools start. This included a change to the host which removed a hack for Windows and scripts for the other operating system and replaced that with a special kind of executable. The result is a cleaner experience, including skipping an extra process.

Also, for security reasons, the location of the executable is hardcoded when the shim is created.

If you recompile, publish, and re-install your application, does it work?

Also, because of changes between Preview 1 and Preview 2, you must manually uninstall Preview 1 Global Tools by removing the directory and the shim.

[spboyer]

@KathleenDollard if you target netcoreapp2.1 and recompile, it works.

[filipw]

if you target netcoreapp2.1 and recompile, it works.
but if you target netcoreapp2.0 and recompile (using the new SDK), it doesn't work.

Not every tool can afford to retarget itself into netcoreapp2.1 straight away, as there could be other considerations and dependencies.
I think it would be a shame if global tools were only allowed if one targets netcoreapp2.1, which seems to be the case now.

[livarcocc]

That's the plan. For global tools, you will need to target netcoreapp2.1.

Can you share more details on what are the considerations and dependencies that would block you from moving your global tool (the first version should have been out with preview1 only) to netcoreapp2.1. netcoreapp2.1 should be able to reference anything that netcoreapp2.0 did. So, dependencies should not be a problem.

[filipw]

So, dependencies should not be a problem.

In the case of dotnet-script project, it is a C# script runner, and it compiles your scripts by combining your own script-level references and some implicit references inherited from the runner itself. Long story short, if the script wants to be executed as netcoreapp2.0 it cannot be fed the implicit inherited dependencies from the host runner which is built against netcoreapp2.1.

I am sure I do not see the full picture here, but it sounds like you could have made it work with netcoreapp2.0 (after all, it worked already) but it was consciously decided to take advantage of a the runtime feature of netcoreapp2.1? Is this driven by an expectation that people would not use netcoreapp2.0 once netcoreapp2.1 comes out?
Also, aren't the global tools part of SDK, which should be kind of independent of the runtime? 🙂

Is there any workaround for that? Could I for example bring my own shim instead of rely on the generated one?

[wli3]

@filipw

Is there any workaround for that? Could I for example bring my own shim instead of rely on the generated one?

It is possible. If you follow the convention tools/TFM/RID/shims/SHIM-RID
you can reference the test project dotnet/cli@b0ee5db

Your own shim will be used

However, you need to cover shims for every different OSs you support(rid win and unix). Also, you need to pre compute the dll path on consumer's machine. Which is .store/$(PackageId)/$(PackageVersion)/$(PackageId)/$(PackageVersion)/tools/$(TargetFramework)/any/$(yourdllname)

and shim name need to be the same as your command name and .exe is needed on Windows.

[filipw]

thanks a lot, looks promising - we will have to explore this.

On a related note - if a netcoreapp2.0 based tool won't work anymore, what is the purpose of the framework option during the tool installation?
For example, I can invoke the following: dotnet tool install dotnet-script -g -f netcoreapp2.0.
This generates the assets file relevant for .NET Core 2.0, but obviously doesn't work.

Is this intended for the future versions only? i.e. 2.2, 2.3, 3.0 etc? If so, then maybe it would be better to remove the option for now to not cause confusion for now.

[wli3]

@filipw more importantly it is to fit dotnet core TFM model. And currently it is for future proof, to have an option for multi target 2.1 and next rumtime major bump --a breaking runtime version will cause existing 2.1 build fail to run. But producer can multi target both TFM to meet all audience

[KathleenDollard]

@filipw

Regarding the decision to have Global Tools support only 2.1 and forward.

The previous shim was a hack, particularly on Windows. The new executable approach allows a single process, which means it will show up as the exe name not "dotnet' in your process explorer, be immensely more debuggable, and other benefits. Some amazing work by the runtime team made this possible and we believe it is the right experience for Global Tools. Sadly it did break the netcore20 compatibility.

We decided to allow BYOS (bring your own shim) to handle some internal scenarios, and rather late. So, sadly, we have zero support for building packages with custom shims. We think for almost everyone, the new standard shim strategy is a really good one, and more secure as we make it difficult to hijack the DLL call.

You're in a special position, and I hope the custom shim works for you.

As a side note: why did you call your tool dotnet script instead of dotnet csscript as there are a number of languages supported by dotnet?

[filipw]

@KathleenDollard thanks for the info. We managed to solve it for the time being by using cross compilation. So if you are on 2.1 preview SDK, you'd run as netcoreapp2.1, if you are on earlier version you'd run as netcoreapp2.0. The shim is something we will explore down the road.

Regarding your naming comment, it's simple - there has been a vision since the beginning that in the future dotnet script foo.csx would still compile and run C#, while, for example, dotnet script bar.vbx would compile and run VB.

[KathleenDollard]

yes, I answered to quickly without realizing this was an existing tool.