Reflection

I think the standard choice for a collider shape of a player controller nowadays remains firmly in the court of a capsule. It can represent a vertical body with rotational symmetry, quite like a cylinder, but without involving much more complicated math. The most important advantage over other shapes with flat bottom walls is that a capsule can scale slopes and obstacles much more smoothly than its counterparts.

Those were the reasons I chose it when prototyping Hosper in Godot and it served me well as during testing of different forms of geometry for maps. I decided to support capsule collisions in Chavelleh as well, but despite already having dealt with spheres and arbitrary hulls, this exercise still took me some time and left me unconvinced with regards to the reliability of my implementation. Wanting to have more insight into collision data, I quickly put together some visualizers and started testing scenarios involving different sets of bodies. While this immediately helped me iron out some bugs, the approach of coding every scenario in a single scene, in C++, also quickly revealed itself to be hard to scale - it took a lot of boilerplate and time to construct and compile.

The problem would be easily solved by physics-sandbox, my physics testing tool with a capability to define basic scenes using JSON, if it wasn’t for the fact that I wanted the bodies to perform arbitrary motions in space in the scope of the testing scenarios and that could not be easily represented just by a set of parameters in a JSON file. There was little doubt in my head that this called for finally attacking the problem of scripting in the engine - custom logic interpreted at runtime fits two major use cases perfectly, those being testing and mapping/modding.

There was a major barrier on the horizon, however. In order to fully integrate an interpreted language, something like Lua for example, the interfaces of the engine that I use in C++ would have to be exposed to the interpreter in their majority; otherwise the usefulness of the scripts would be vastly limited. I very much so didn’t want to introduce a ton of glue code binding the two languages together, especially considering that it’d amount to a ton of redundancy. Thank the heavens, I didn’t spend another couple weeks thinking of how to solve this issue, but simply downloaded the trunk of the GCC repository, enabled the newest standard, added the -freflection flag and went to town.

The following is a very rough insight into how I managed to get automatic C++-to-Lua binding generation using the new and shiny reflection, coming along with support for the C++26 standard. The two fundamental references behind this endeavour will be the original proposal, P2996, found here, and the user manual for sol2, a C++ library implementing sensible interfaces on top of the Lua interpreter (either PUC Lua - the original - up to version 5.4, with trivial changes necessary for supporting 5.5, or LuaJIT 2.1), which can be found here.

I’m not sure exactly how to format this knowledge dump, so let me introduce you to basic concepts of the proposal in a form of a list of bullets:

• Most, if not all, things related to reflection, i.e. stuff found in the std::meta namespace, is consteval - this means they must be known at compile time; this stands in contrast with constexpr, which allows a function to be used both at runtime and at compile time. The two can be easily mixed up here, as in some cases values of some arguments may only be known at runtime (which they can’t be required to, with consteval), or some values will have to have be established strictly at compile time. C++26 also standardized the concept of “poisoning” statements to become consteval (those involving expressions known as immediate-escalating expressions). I don’t fully understand the nature of some of the errors I have encountered along the way, but resolving them involved a slight bit of refactoring and thinking about when each piece of information is available or needed exactly.
• The pain points stemming from the aforementioned distinction become especially prevalent when constructing wordy operations on reflected types, as, for a reason that is unknown to me, you can’t explicitly require function arguments to be consteval - this means that you can’t pass intermediate processing results between different helper functions easily. At best, you can use (capturing) lambdas.
• The proposal introduces a new operator, ^^, replacing ^ and reflexpr from previous experiments, as well as the splicing operator [: X :]. The reflection operator is used on types to be reflected as arguments to std::meta functions. For example, if one were to want to obtain a list of all members of a particular class or structure, one would invoke std::meta::members_of(^^T, std::meta::access_context::current()).
• The proposal also defines a structure, std::meta::info, that to my knowledge is implementation-specific and I’m not sure it can be inspected. Most std::meta functions return either one instance of this structure or a vector of them, for iteration. Each such structure refers to, or serves as a descriptor of a specific language construct, e.g. a value, a variable, a function, a type, etc.
• In order to refer to this construct as a {l,r}value, one can use the splicing operator, [:x:], which the compiler will then substitute for the construct being referred to.
• While vectors of std::meta aren’t themselves consteval, their contents need to be known at compile time in order to be able to generate code. For this purpose, another proposal introduced a mechanism of define_static_array, which, when passed such a compile-time vector, creates an actual array that then can be iterated over using a new language syntax, template for. It intuitively works like a regular for loop, except it generates the body of the loop for each instance of the std::meta structure being iterated over (and only permits the container interation syntax).
• Some of the std::meta query methods also allow specifying the access context in which the query should take place, meaning that one can limit the retrieved list of constructs to only those accessible from the call site (e.g. only public ones if invoked from outside of a structure/class/method definition), or all of them. Those are outlined in the proposal.
• The proposal in general is written in a half-referential, half-insightful manner, so there is no reason to be afraid of any serious language legalese in there. Go and read it yourself.

All these in tandem form a surprisingly expressive, if somewhat verbose, mechanism for generating code targetting specific parts of a particular type, based on just its identifier. For example, one can now obtain a list of fields and methods of a structure, iterate over them, obtain their addresses or offsets within said structure and create bindings based off that.

Sounds exactly like my use case.

(Notabene: One major caveat that was explicitly not addressed in the original proposal is reflection of constructors and destructors. There’s more “magic” involved here than with regular methods, so its authors have decided to work on resolving their remaining issues in scope of another proposal - meaning, most likely, C++29 at the earliest.)
(Another notabene: If you get decide to use the trunk before GCC 16 is properly released and encounter linking errors to the tune of <insert_math_func_name_here>@GLIBC<version> being undefined, liberally sprinkle -lm in linking arguments for your project and dependencies. Do make sure that -lstdc++ is added too and that the exact right collection of tools is used for compilation and linking - making sure that this is correct for C++ tools outside of /usr can be a bit of a pain.)

A small snippet of the method I cooked up to perform the bindings follows. It uses sol2/sol3 for the purpose of interfacing with the Lua interpreter itself.

template<typename T, typename... Ctrs>
void RegisterType()
{
    // Create an instance of sol::constructors using the list of constructors 
    // from the template.
    constexpr auto solConstructors = sol::constructors<Ctrs...>();

    // Create a new usertype for T.
    constexpr std::string_view typeName = std::meta::identifier_of(^^T);
    sol::usertype<T>& newType = _lua.new_usertype<T>(typeName, solConstructors));

    // Map members of T to the new usertype.
    template for (constexpr auto member :
                  std::define_static_array(std::meta::members_of(
                      ^^T, std::meta::access_context::current())))
    {
        // Methods and non-static fields.
        if constexpr (std::meta::has_identifier(member)
                      && !std::meta::is_static_member(member)
                      && !std::meta::is_constructor(member)
                      && !std::meta::is_type(member)
                      && (std::meta::is_function(member)
                          || std::meta::is_nonstatic_data_member(
                              member)))
        {
            constexpr std::string_view memberName
                = std::meta::identifier_of(member);
            newType[memberName] = &[:member:];
        }

        // Anonymous unions.
        else if constexpr (std::meta::is_type(member)
                           && std::meta::is_union_type(member)
                           && !std::meta::has_identifier(member))
        {
            template for (constexpr auto datum :
                          std::define_static_array(
                              std::meta::nonstatic_data_members_of(
                                  member,
                                  std::meta::access_context::
                                      current())))
            {
                if constexpr (std::meta::has_identifier(datum)) {
                    constexpr std::string_view datumName
                        = std::meta::identifier_of(datum);
                    newType[datumName] = &[:datum:];
                }
            }
        }
    }
    
    // [...]
}

The snippet leaves out some bits pertaining to template discovery and binding operator overloads, which I think is a good exercise to leave for the reader to solve. In any case, it presents the basic mechanisms used for the purpose - the two new operators, the whole shebang with template for and define_static_array and exactly how much of the code needs to be marked constexpr - it appears to me that certain if constexprs actually modify some internal compiler assumptions related to the constructs being reflected on and matching those assumptions directly is crucial to successful compilation (e.g. in the // Anonymous unions condition, try replacing is_union_type with is_complete_type - in the current trunk (as of the time of writing), this will trigger compilation errors complaining about reflected types not being complete classes, which makes no sense on the face of it). There are some what I believe to be temporary quirks in GCC’s implementation of the new features, but all in all everything is usable, if requiring a bit of elbow grease.

I think Herb Sutter referred to reflection as the engine of the language that will be used to propel it into the next decade. After the last couple days of experimentation, I can’t agree more - while still far from the usability that e.g. Jai will or Zig offers now, if you are the kind of a person who does this for the love of the struggle, this is indeed a feature to beat.