Investigate replacing Scripty with Source Generators

The introduction of C# 9 Source Generators should theoretically be able to replace the current generation done by Scripty. This issue will track the results of investigating this proposal.

Positives

• This is pretty straight forward: A vanilla compiler mechanism would replace a third-party compile-time dependency.
• Source Generators support intellisense, unlike Scripty script files.

Negatives

Due to Source Generators being a new compiler feature, the concept is not fully implemented and some of the problem areas should be smoothed out as the dotnet team expands the feature.

• Source Generator projects MUST target netstandard2.0. To enabled later language features, use the LangVersion element in the csproj file, eg <LangVersion>9.0</LangVersion> to enabled language features up to C# 9.
• Visual Studio must be closed, reopened, and the project rebuilt to get intelliscence and other features to notice changes in the generated source. Based on videos, it seems they had this working correctly at some point, but regardless this is something the dotnet team will eventually fix and isn't a big deal in the long term.
• This is the biggest one. Despite the name, Source Generators aren't actually meant to generate source code for their containing compilation; they are meant to generate syntax trees that are added to the referencing project's final compilation. In other words, as the feature is designed, CommonCore project wouldn't include a NumericGenerator class that generates a NumericExtensions.cs file. Instead, a CommonCoreNumericGenerator library project would contain the NumericGenerator class. The CommonCore project would then reference the CommonCoreNumericGenerator library, causing the built CommonCore dll file to contain the code generated by the NumericGenerator class, with no NumericExtensions.cs file ever being created.
  • I both like and dislike this design decision. First, it seems to ignore the basic use case of generating code as part of a library or application. Namely this use case where I simply want to generate a series of methods as part of the CommonCore library itself. I can see some benefit to organizing generation code into its own library, ie separation of concerns and all that, but it still seems like unneeded library bloat to me.
  • I only dislike the decision of not generating source files. Yes, adding the EmitCompilerGeneratedFiles bool element to the referencing project's csproj file will emit the generated source files (sometimes), but it puts them in the a sub directory of the obj directory, as they are meant to be emitted only for manual inspection by the developer, not as source files to be included in the project. This seems to me like one of those ideas that sounds cool when you are spitballing a system, but is pretty boneheaded from a practicality perspective. Code review, feature discovery, debugging, repo versioning, visual debugging, enforcement of coding standards, and just doing what it says on the tin, generating source code, everything in the coding world revolves around having code in files. Supposedly there will be a way to control where emitted files are written, but that is just a band-aid on what is ultimate a design mistake. Once again, WinForms did it right by creating the .designer.cs files, because, say it with me, source generation is about generating source code.

Work-Arounds

We are stuck using generation libraries to generate CommonCore code. There seems to be no way around this; source generators don't generate code for themselves, they generate code for their referencing compilation.

However, the proper generation of source files in the referencing project is doable, though with a few minor downsides.

The first step is determining the location of the project directory. There is no direct way to get this information and, as I understand it, this is due to source generators being analyzers (code that inspects other code) and analyzers are run on a layer below the workspace layer (the part that knows the what and where of your solutions and projects) and former can't access the latter.

A solution is to add a file to the AdditionalFiles element of the referencing project's csproj file. The file must be in the project's root directory, so for convenience we can use the csproj file itself, eg <AdditionalFiles Include="$(ProjectPath)" />. This then provides the referencing project directory through something like Path.GetDirectoryName(context.AdditionalFiles[0].Path). With that directory in hand, we simply write whatever code string to a file in that directory, eg File.WriteAllText(Path.Combine(dir, "Foo.cs"), code). Since the file is being written into the project, we don't want it duplicated in-memory by the source generator, so no call should be made to context.AddSource for the related code string.

Another solution would be to use build properties, whether global or on some kind of individual additional files basis. This way allows the files to be written anywhere the referencing project wishes and would support the current CommonCore convention of nesting extension files in the _Extensions directory.

Downsides

Currently, Scripty runs pre-build, meaning errors in the generated files don't stop those files from being regenerated. Source Generators effectively run post-build, meaning errors in generated files stop the regeneration of those files.

In practice, generated files occasionally need manual repair before they can be regenerated. On the small scale this is mostly just annoying and could be solved by simply deleting the contents of the file. However, on the larger scale, where deleting the code could itself cause errors... it could be a problem. This needs further investigation, but a naive solution is allowing a test generation mechanism that adds some kind of test suffix to the file name; this test file could then be deleted if it causes problems and the real file only regenerated when the developer is confident it will work.