Skip to content

[For Review/Testing] Feature: High level Variant Records

Frederic Kehreinrequested to merge
Warfley/fpcsource:managedvariants into main

Hello everyone,

with this MR I'd like to introduce a set of new features to the FPC, which for a lack of a better name I just called High level Variant Records. This MR builds on top of !498, as some of the features introduced there (namely the ability to have reference symbols) was required to make this possible (these features were actually the driver behind !498 to begin with). To see the changes of only this MR, you can do a direct branch compare

About

Variant records in Pascal are quite a low level feature most often just used as equivalent to C's union types. But unlike C's union types, variants do encode some form of semantic in them through the usage of selector fields and association of values to branches. While there have been some approaches in the past to utilize this more, like ISO pascal, which allows setting the selector chains in the new construction, it has generally been very underutilized, and any programmer who wants to use it, has to build all the semantics on top themselves.

With these features I tried to change that. To do so, I changed the internal representation of variants. But because Variants are so important for ABI compatibility, especially with C APIs, it is all behind the VARIANTRTTI directive, which as a local switch can be set and unset any time. If unset (default), the internal representation will be exactly like before, ensuring ABI compatibility.

Features

This MR consists of 4 main features, each enabled by their own switch

Variant RTTI

The first feature which is also required for the other features, enabled by the VARIANTRTTI switch, does most of the ground work. Previously symbols of the variant branches where directly added to the record, just with resetted offsets. Which is great for low level ABI compatibility, but any information about the branches is lost. Especially nested variants get flattend and lose most of their hierachical information. Instead with this switch enabled, each branch is added as a hidden field and the member fields are just forwarded (similar to how record composition adds the fields, which is why it builds on top of that MR).

This change now enables to embedd RTTI information, as each record can have at most one variant part (as the nested variants are part of their own records), making it possible to traverse the variant through RTTI information.

This feature adds a new field to the Records (full) RTTI table, containing information about the switching field, as well as for each branch, which values are associated with that branch, and which fields belong to that branch. Also a helper function was added that allows to check which branch is currently selected by the selector field.

The newly introduced hidden fields will not come up in the RTTI tables ManagedFields list, but instead transparently the forwarded fields are added, to ensure compatibility with existing code. This now allows to traverse the variant parts of the record e.g. for serialization algorithms, reflection, or other RTTI techniques.

{$VariantRTTI On}

uses typinfo;

type
  TVarRec = record
  case sel:(Enum1, Enum2, Enum3) of
  Enum1: (A, B: LongInt);
  Enum2: (C, D, E, F: Word);
  Enum3: (G: Double);
  end;
var
  vr: TVarRec;
  rd: PRecordData;
begin
  rd := PRecordData(GetTypeData(TypeInfo(TVarRec)));
  vr.sel := Enum2;
  WriteLn(rd^.VariantBranch[@vr]); // Prints 2 because Enum2 selects the second branch

Strict Variants

The next feature does not necessarily require the new internal representation or RTTI functionality, but it is useful in combination with the others. Through the switch STRICTVARIANTS additional compiler checks are added, to ensure that branches don't overlap, and that a variant must always cover a branch for the 0 value, as this is set by using the Default directive.

Strict variants can be enabled per variant branch, ensuring that the branches for which it is enabled do not overlap.

{$StrictVariants On}

type
  TVarRec = record
  case Integer of
  0..10: (A, B: LongInt);
  11..20, 5: (C, D, E, F: Word); // Error because 5 overlaps with branch 1 (0..10)
  end;

Variant Access Checks

With the compiler switch CHECKVARIANTACCESS new access checks are added. Whenever a field of a variant branch is accessed, new runtime code is inserted to check if the selector field of that record selects the branch the accessed field is on. If not a runtime error/exception is fired, similar to Range or Overflow checks

To perform this runtime check VariantRTTI must be enabled.

{$VariantRTTI On}
{$CheckVariantAccess On}

type
  TVarRec = record
  case sel:Integer of
  0..10: (A, B: LongInt);
  11..20: (C, D, E, F: Word);
  21..30: (G: Double);
  end;
var
  vr: TVarRec;
begin
  vr.sel:=5;
  vr.G := 42; // Error because branch 1 is selected and G is on branch 3
end.

Managed Variants

Finally the reason why I did all of that (and the previous MR), with the ManagedVariants switch, managed types can be used in the variant part of the variant record. For this the compiler will only execute the management operators on the fields of the currently selected branch. Additionally all assignments to the selector field will be augmented with code that when switching branch, will finalize the old branch and initialize the new branch.

While this can be circumvented with direct pointer access to the selector field (or RTTI), the compiler will throw at least some warning when trying to do so.

{$Mode ObjFPC}{$H+}
{$VariantRTTI On}
{$ManagedVariants On}
uses heaptrc;
type
  TVarRec = record
  case sel:Boolean of
  True: (s:String); // Managed field
  False: (I: Integer);
  end;

var
  vr: TVarRec;
begin
  vr.sel:=True; // Branch switch -> will finalize vr.i and initialize vr.s
  vr.s:='Hello World';
  UniqueString(vr.s); // ensure dynamic memory allocation so any errors are visible in heaptrc
  vr.sel:=False; // Branch switch -> will finalize vr.s and initialize vr.i
end.

Final Words

This feature is not yet finished. First things first !498 needs to be finished anyhow. Also while writing this text a few more things came to mind that would still needed to be added. But because it is now in a more-or-less usable state (in that all my tests seem to work fine), I wanted to share to get feedback, first from the maintainers if a feature like this is wanted anyway (I don't want to waste my time with something thats not gonna make it anyway), but also potentially from other users what they think.

Merge request reports