Standardizing Ghidra's C++ Class Representation

[astrelsky]

@ghizard @dev747368 @ghidra007 I'm pinging the three of you since, afaik, you maintain the debug parsers and the C++ RTTI scripts.

This discussion will omit virtual base classes since Ghidra's datatype model is currently incompatible with it.

It would be a good idea to standardize the way c++ classes are represented in Ghidra so that the debug parsers and RTTI analysis can construct them in the same way. This will lead to less confusion and make reversing programs written in c++ easier.

My suggestion would be the following:

struct MyBaseClass;

// naming the virtual function table structs like this makes it easier to use through the ui.
struct MyBaseClass_vftable {
    void (*~MyBaseClass)(struct MyBaseClass *this);
    void (*~MyBaseClass)(struct MyBaseClass *this);
    int (*start)(struct MyBaseClass *this);
};

struct MyBaseClass {
    struct MyBaseClass_vftable *_vptr;
    int iVar1;
};

struct MyClass_vftable {
    void (*~MyClass)(struct MyClass *this);
    void (*~MyClass)(struct MyClass *this);
    int (*start)(struct MyClass *this);
    int (*stop)(struct MyClass *this);
};

struct MyClass {
    // embedded unnamed composites would work best here
    // the main benefit would be that the unnamed composite may not be shown as a type in the decompiler as a cast or variable
    union {
        // this should be the decompilers first choice when the union is dereferenced as a pointer.
        // if it's not the user can always force it (when said feature wants to work)
        struct MyClass_vftable *_vptr;

        // the "super_" prefix can get tedious and cumbersome to look at after a while, I find this style simpler.
        struct MyBaseClass MyBaseClass;
    } _; // full unnamed composite member support would be nice for this
    int iVar2;
};

[astrelsky]

I know having structures fields with duplicate names was discussed in the past, but I see no reasonable way around it for virtual function tables. I think it should be allowed and maybe the decompiler could emit a comment above the access giving the offset of the field?

[ghizard]

@astrelsky
Thank you for your suggestion. You can see there has been prototype script work that is trying to force-fit the C++ language items into our mostly C-aligned framework, but we are envisioning a more sturdy, well-thought-out design to cope with C++ and OO in general. We ran into the same issues in the initial PDB Universal delivery which contains a lot of class work that was eventually exposed with a launch property; there is no good way to store the information and trying to shoe-horn the very complicated class structures into the C-aligned framework is messy and would cause issues with upgrades when we have the better solution.

Anonymous unions had been tried and are not a good solution, as the Decompiler doesn't emit what one would use in source code to refer to members of the anonymous unions, and they would only be pursuing the shoe-horn solution that we are trying to avoid.

We are striving to make sure that what we design will be consistently used whether by users and analyzers alike to receive input from users, debug information, or program information. There is a lot to be done including reworking the Symbol/DataType systems, Decompiler, Debugger, PDB/DWARF, and other analyzers.

The rough prototypes are trying to work out algorithms while still providing some useful tools for the users. The same is true for the PDB capability. These prototypes will likely be thrown away when the final designs are surfaced.

[astrelsky]

@ghizard understood. I was under the impression that there was no intention of changing the way data types were modeled to be able to properly handle this.

However, could the debug parsers at least create the virtual function table structures in the meantime? That information is incredibly valuable and there is no other easy way to get it especially if the only debug info you have would be for the types. The function table structures help fill things missed from fidb and let you know what override functions should be for the implementations. Having to fill them out manually is a lot of work especially for large frameworks. Even if they are not added to the class structures, it still saves a ton of work if the user wants to add it to the type themselves.

[ghizard]

@astrelsky We are looking into a couple of options. Might have some questions for you if we need clarification.

[ghizard]

I'm speaking directly about PDB, but DWARF (Itanium ABI) could have similar situation.

Lets say you have class A with virtual functions, and B with virtual function, and C : A, B that has virtual functions.

When C gets laid out, you will find an occurence of class A inside of C and an occurrence of B inside C. The occurrence of B inside of C has its own vftptr that is used if you cast C to B. There is also a vftptr that was initially laid out for class A within C that C will share (in the circumstance of this simple example).

For PDB, this would look roughly like:

+--- Class C
| +--- (occurrence of A)
| | {vfptr of A shared by C}
| | members of A
| +---
| +--- (occurrence of B)
| | {vfptr of B}
| | members of B
| +---
| members of C
+---

The entries for methods of class A will be in the table first, followed by the entries for class C which includes (maybe first) the entries for class B. When class C is cast to class B in this case, the this pointer is moved. When class C is cast to class A, the this pointer (since this example is simple) is not moved, so the location of the vftptr being shared with C points to the same table, but the table that it points to is only consulted for the entries that pertain to class A.

Let's say that class A has 2 entries in its vftable, class B has 1, and class C has 4. Then class A and class C share the same table, but class A knows its table size is 2, but class C knows the same table size to be 4. MSFT just lays things out such that the shared entries are the first two, and thus the same table (after the class has been constructed).

PDB knows the shapes of the three tables such that it could create pointers for the vftptrs, but we have the same conflict as with constructing the class in general (why the work had not continued in the current fashion); do we use unions or something elset to allow this shared vftptr?

I view this as the vftable of A owned by C, and thus could construct it with 4 entries, but if the class C is cast to class A, then it is only legitimate for class A to consider that it has 2 entries belonging to it. A native class A would have the vftable only having 2 entries.

The prototype PDB work currently only sets the shape to be that of class A; this is a known issue that I intend to fix. But how to present this is still a challenge and would be a challenge for what you want. What are your thoughts?

Ignoring this issue at the moment, I'd like to better understand what you would like to "edit." If a class structure had a single vftptr with datatype vtshape_C *, it would not know where the table is located in memory. Moreover, the vftptr values only set dynamically during class Construction so we would now know to lay down the vtshape_C at the memory location. I changed MSFT demangler within the last couple of years so I'll be able to programatically get some of the heritage of the tables from the mangled string so that I could eventually try to match that with each vtshape from each vftptr within a class. That has not been done yet, but that would be one solutions that I've planned to try out.

Back to the question... do you plan to edit the structure of vtshape_C to set the types on the elements, such as pointer to member function with one float argument and integer return type? Or would you just leave the elements as undefineds or code pointers? Reason why I ask is that with PDB, a vtshape can be shared by totally unrelated classes. If class C needs a vtshape of {near32, near32, near32, near32} (pointers), you might also find class monkey using the same shape record, but these shapes don't get into the detail of the function definitions and so could collide unless each class gets its own "copy" to modify. Maybe, since the PDB knows that shape, you would not need to perform any additional modifications. Can you elaborate on your intended use/edits?

[astrelsky]

At the moment I would only be looking for the virtual function table structures which contain the function signatures. If I want to analyze a class function for C, I would edit C's datatype and unpack all the base classes and replace the appropriate virtual function table pointer with the one for C. This would at least allow me to see how the function should flow with respect to virtual function calls. If I need to see the calls as they would be at runtime, I would just manually apply call overrides where necessary. Having the table structure with the signatures would also allow me to fill in unidentified functions in the table. I would be able to apply the appropriate function signature from there.

When I have to manually deal with virtual functions where everything is unknown, except for the table location, I'll usually create a structure from the table and name the fields with the default function names. Then where they are used I just override the call site by copying the field name and removing the FUN_ prefix after pasting it in the reference editor. This is done so I can traverse the references and figure out the types to create the appropriate function signature if there are other implementation of the base class I need to look at.

I know that the end goal would be to handle every possible situation, but right now good enough is good enough. If I at least have a struct of the appropriate function signatures I can manage much better. It would save hours of work in some cases with large frameworks like MFC.

A bit off topic but for the Itanium ABI it is a bit simpler. Each function table in a implemented class (class C) has individual function tables for each implementation except the first base class where they are merged. Then above each function table is the typeinfo pointer or NULL and a sorted array of virtual base offsets accessed through a negative index.