-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] Using LeMP and .ecs as T4 replacement #112
Comments
I think you are asking "what is the LeMP alternative to |
yes, kind of |
Well... give me the weekend and I'll have a proper answer for you. For a long time, I've been meaning to add a way to run arbitrary C# code at compile-time, but no one has actually asked for it. So, currently, the only way to run arbitrary code at compile-time is to create a separate C# project to define macros, compile them as normal, then import them into LeMP. I think I can add a more convenient feature quickly, but first I have to produce an initial 2.8.0.0 (semver 28.0.0) release, which I'll try to do immediately. After that I will add a macro that allows you to run code at compile time via Roslyn (Microsoft.CodeAnalysis.CSharp.Scripting). Here's my current thinking:
For example, this code: compileTime {
using System;
using System.Collections.Generic;
using System.Linq;
using Loyc.Ecs;
static List<string> GetFunctionNames(LNodeList input) {
// Get list of function names
var functionNames = new List<string>();
foreach (var part in input) {
// Check if it's a method
if (Loyc.Ecs.EcsValidators.MethodDefinitionKind(part, out _, out LNode name, out _, out _) == CodeSymbols.Fn) {
functionNames.Add(name.ToString());
}
}
return functionNames;
}
static LNodeList WithMetadata(LNode input) {
// quote { } produces a single node if the input was a list. Convert input back to a list.
var nameList = GetFunctionNames(input.AsList(CodeSymbols.Splice));
return quote {
static string[] metadata = new[] { $(..result.Select(name => LNode.Literal(name))) };
$input;
}
}
}
precompute(WithMetadata(quote {
static int square(int x) => x*x;
static int cube(int x) => x*x*x;
})); ...should produce output like this: static string[] metadata = new[] { "square", "cube" };
static int square(int x) => x*x;
static int cube(int x) => x*x*x; The part that says
If I do this in a quick-and-dirty way based on Roslyn's interactive C# mode, the behavior of the macro will be a bit counterintuitive. For example, the input
...would work: it would write a temp.txt file to my "Documents" folder, even though the definitions What do you think of this proposal? |
Thanks for detailed answer and plan, I will digest them slowly.. My general thoughts on code-gen in C#: the valid C# gen with the output available to user inspection currently boils down to T4. Wich is practical (has a small API surface) and tool-friendly (intellisence, coloring, debugging). But MS does not love it (is not fixing the existing problems) and stopped the dev for newer .Net Core targets. Which opens a space of possibility for alternative solutions, given that some features will cover for absense of some T4 features. Btw, I don't consider Roslyn for the described task (overly complex and lower level) and upcoming source-generators (incomplete moving target). Also I still see a big demand in the working solution. The latest addition of T4 support in Rider IDE and activity in mono\t4 repo, questions on StackOverflow. My dream is important as well :) for compile-time DI. As a PL lover I also see a value in C# macro-system and extending the language via user customization not relying on MS offerings. Looking at C#8, C#9 and beyond - IMHO MS does not have a vision and drived by comittee producing all-over-the-place features. The progression is not fast either. Regarding your points
Yes, this is an important feature to not jump over the hoops. Also having the syntax matching the already existing solutions is good for familiarity.
This is perfectly fine, and the code sharing is not the first order feature anyway.
I am not yet figured out the what's is what here, feeling a bit dumb.. What I want is something like this in my .ecs: //auto-generated
namespace Blah
{
partial public class X
{
compileTime {
foreach (var expr in GetExpressions())
{
var methodName = "Get" + expr.Type.Name;
generate {
public ${methodName}() =>
${exp.Body}
}
}
}
}
}
Looks interesting but I wonder the use cases for it.
Simple and useful. As I understood this is the closest to T4
Honestly I did not get all the details. Let it seat with me for a couple of days, maybe I'll come with more thoughts. |
In my current plan, In LeMP it's not obvious how to support something like So in my current plan you can't write this...
...but you could write this:
...or this:
|
@qwertie |
Success! 😁 Input: compileTime {
#r "C:\Dev\ecsharp\Bin\Debug\Loyc.Utilities.dll"
using System.Collections.Generic;
var d = new Dictionary<int,string> { [2] = "Two" };
var stat = new Loyc.Utilities.Statistic();
stat.Add(5);
stat.Add(7);
}
public class Class
{
// Yeah baby.
const string Two = precompute(d[2]);
const int Six = precompute((int) stat.Avg());
double Talk() {
precompute(new List<LNode> {
quote(Console.WriteLine("hello")),
quote { return $(LNode.Literal(Math.PI)); }
});
}
} Output: public class Class
{
// Yeah baby.
const string Two = "Two";
const int Six = 6;
double Talk() {
Console.WriteLine("hello");
return 3.14159265358979;
}
} |
This is cool and congrats!
This is must have and the next immediate step is resolving the path from the "known stable" location, either from the current directory or project directory or something. Actually it can be anything fixed but not an absolute path. |
I've just added relative-path resolution (relative to One significant issue with this new feature is that you'll be able to crash Visual Studio by writing code like this:
This is because So of course, you might write some code that is unfinished. If it's an infinite loop, not a huge deal, VS will freeze up for 10 seconds and then LeMP will automatically terminate the macro. But in case of stack overflow, or if it's an infinite loop that allocates more and more memory, VS will crash. So I was thinking about how to fix this - it's not easy to set up a separate process with the necessary interprocess communication just for In fact, T4 is worse. LeMP lets you run arbitrarily evil code at compile time (e.g. "reformat drive D:"). Maybe this could be avoided with a tool like Sandboxie but again, T4 has the same problem. |
Cool :) I wonder what happens when you Link ecs file to project instead of adding it. This is a common thing for content files NuGet packages and a major PITA. Many questions arise: where transform will put the output file, what to consider a containing folder.
I am planning to do that too as a part of build process outside of VS - in VSCode and on CI.
Exactly!!!
I think educating the user via docs and readme will be enough and already far exceeding current T4 experience. Update:Couple of links with T4 struggles on the topic: |
I don't know what you mean by "a common thing for content files NuGet packages". If I choose "Add as Link" when adding an existing file from a different folder in Visual Studio, then... let's see...
In this case, the input folder is still the folder that contains the ecs file, not the project folder. And Visual Studio puts the transformed file into the input folder. Please tell me more about this NuGet scenario you're thinking about. I don't use the command-line |
About // You might have `class Order` in a Order.cs file that is part of your project
// and use compileTime { includeFile("Order.cs"); } to use it at compile time,
// but if it's a small class (or if it uses EC# features) you might want to use
// compileTimeAndRuntime instead:
compileTimeAndRuntime {
namespace Company {
public partial class Order {
public string ProductCode { get; set; }
public string ProductName { get; set; }
}
}
}
compileTime {
using System.Linq;
// (This is already provided as an extension method of Loyc.Essentials btw)
public static string WithoutPrefix(string s, string prefix) =>
s.StartsWith(prefix) ? s.Substring(prefix.Length) : s;
// In real life you might read a file with includeFileText("FileName.csv")
// and parse it at compile time, to produce a list or dictionary of objects.
Order[] CannedOrders = new[] {
new Order { ProductName = "Tire", ProductCode = "#1234" },
new Order { ProductName = "XL Tire", ProductCode = "#1236" },
new Order { ProductName = "Black Rim", ProductCode = "#1238" },
new Order { ProductName = "Red Rim", ProductCode = "#1240" },
};
}
namespace Company {
public partial class Order {
precompute(CannedOrders
.Select(o => quote {
public static Order $(LNode.Id("New" + o.ProductName.Replace(" ", "")))() =>
new Order {
ProductName = $(LNode.Literal(o.ProductName)),
ProductCode = $(LNode.Literal(WithoutPrefix(o.ProductCode, "#"))),
};
}));
}
} Output: namespace Company {
public partial class Order {
public string ProductCode { get; set; }
public string ProductName { get; set; }
}
}
namespace Company {
public partial class Order {
public static Order NewTire() => new Order {
ProductName = "Tire", ProductCode = "1234"
};
public static Order NewXLTire() => new Order {
ProductName = "XL Tire", ProductCode = "1236"
};
public static Order NewBlackRim() => new Order {
ProductName = "Black Rim", ProductCode = "1238"
};
public static Order NewRedRim() => new Order {
ProductName = "Red Rim", ProductCode = "1240"
};
}
} But while testing this, I discovered a couple of significant limitations of the C# scripting engine:
So I've added code to strip out namespaces and variable declarations marked with |
When installed the content of NuGet package is put into "%USERPROFILE%\.nuget\libX\versionY\contentFiles\cs\netstandard\MyFile.tt" Then it is included into the project via link. That's means the output will also go to the machine wide cache, plus it is a pain to include the ".ttinclude" files. Ideally it is something for user to add, so include files should be part of the project in version control, etc.
I will try to create a reference minimal example with T4 to illustrate the mess. |
So as I understood |
Hmm, when I install NuGet packages I normally get a "packages" folder associated with the solution, instead of (or in addition to) the packages being put in %USERPROFILE%.nuget. Unfortunately, the Visual Studio extension is using the ancient "Single-File Generator" a.k.a. "Custom Tool" APIs, which do not provide access to information about the current project (such as its location). No doubt there is a way to upgrade it with more modern amenities, but ... ugh, doing anything with VSIXs is horrible. Case in point, yesterday and today I spent several hours figuring out how to force Visual Studio to include DLLs in the VSIX that it was randomly excluding for no apparent reason so that the new But... VS/MSBuild variable expansions don't seem to work in the "Custom Tool Namespace" field where I have been shoehorning command-line arguments until now. For example |
I just published a new version here. The source code of the feature is CompileTimeMacro.cs. |
Great, that's something to experiment with. Regarding the NuGet, since v4.0 and for modern sdk-based projects the packages are installed into the Windows user ".nuget" folder. Maybe some other things can influence this behavior (like legacy solution or existence of packages folder) but if you try to scaffold the new .NET Core 3.1 or .NET Standard project you will get what I get. |
I have started to put together a sample here /~https://github.com/dadhi/LempTest and stuck with exception while using any of
I tried to install The |
Confirmed. So I guess this is why Visual Studio has been adding "app.config" files in my projects without explanation. In the folder with LeMP.exe, create a file called LeMP.exe.config with the following code (courtesy VS): <?xml version="1.0" encoding="utf-8"?>
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<dependentAssembly>
<assemblyIdentity name="System.Runtime.CompilerServices.Unsafe" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-4.0.6.0" newVersion="4.0.6.0" />
</dependentAssembly>
</assemblyBinding>
</runtime>
</configuration> It's bizarre: my own code doesn't use this DLL, which implies Roslyn is explicitly depending on the wrong version and expecting every app that uses Roslyn to have a bindingRedirect to fix it. |
Thanks for the tip with the binding redirects, I did not think about adding them to LeMP installation, just tried on my project :/ Regarding Roslyn seems like the common problem, the Span and ValueTuple where added later and somehow MS decided to conform them for the older .net targets - who knew. |
I have updated the zip file with the necessary .config files. |
I will be putting found issues and/or questions here while exploring things. Small things first:
|
Could you suppress the warning CS7021 regarding the |
I don't want to overload this thread; please file separate bug(s) for these issues. |
I am stuck on the next code where the method body is generated from the string in var ecs = Loyc.Ecs.EcsLanguageService.Value;
var getMethods = factories.Select(f => quote {
object $(LNode.Id("Get_" + f.Key.Name))(IResolver r) =>
$(..ecs.Parse(f.Value.Body.ToString()));
}); I want: object Get_A(IResolver r) => new A(); But getting object Get_A(IResolver r) => _numresult(new A()); Could you please help? |
Use expression-parsing mode:
Explanation of what happens: I originally designed Enhanced C# to support a Rust-like syntax where you'd return stuff by leaving off the semicolon, e.g.
I never actually supported the feature in any way, except that it is still supported in the parser, where an statement expression without a semicolon at the end is encoded in the Loyc tree by |
It would be nice to record Q&As like this on StackOverflow, so if you ask the question again on SO - I'm subscribed to the |
Cool, will post next questions there and thanks for the answer. |
Regarding "The C# scripting engine does not support namespaces.", you can't suppress that because it comes from the macro, not from the scripting engine (the latter produces error 7021 which of course can't be suppressed either, so the macro removes the namespace directive entirely) and LeMP currently has no system for suppressing warnings. So you could make a feature request about that... but I have no idea how warning suppression should be designed... by substring match, perhaps? |
may be somehow propagate |
@qwertie |
Hi there and thanks for the interesting project(s).
I have read the documentation regarding using LeMP as C# generator and installed VS 2019 extension (btw, thanks for still supporting it). But maybe I am lacking the essential understanding of how things should work or am I chasing something else completely.
What I want is mostly C# code as the output of my
.ecs
file sprinkled with the dynamically generated methods and fields. Similar to this: /~https://github.com/jbogard/MediatR/blob/d916697159a1f1270d704c6fca95dab7dc274efa/samples/MediatR.Examples.DryIocZero/Container.Generated.tt#L131Does it mean I need to construct
LNode
for the output methods and print it as a plain CSharp,or should I do an intermediate
quote
step in some other place (what place?) and then use itLNode
output for my purposes?Basically is the LeMP alternative to T4
<# /* insert generated C# */ #>
?The text was updated successfully, but these errors were encountered: