C++ calling convention for not-trivial-return-values

[gipi]

As indicated here

Non-static member functions, including constructors and destructors,
take an implicit this parameter of pointer type. It is passed as if it were
the first parameter in the function prototype, except as modified for
non-trivial return values.

Indeed in some C++ binaries that I'm reversing the __thiscall erroneously set the first argument as a pointer to the class instance but in reality should use the second parameter for that and uses the first for the return value.

Is there any way of describing a call fixup for this case? moreover, what switch8_r0 and similar that I see for ARM32 are used for? I didn't find any information.

[astrelsky]

There is an attribute for prototype in the cspec called retbeforethis that should be set to true. I think there is an example of it being used in the x86 cspecs somewhere.

The callfixups are for things like compiler generated functions that have side effects the decompiler needs to know about that it can't see. An example is alloca or chkstack which alter the stack pointer. You can add a program specific specification extension for ones not provided with Ghidra. I personally use these all the time for functions such ass __allmul on 32 bit x86 to have the decompiler replace the function call with multiplication.

ghidra/Ghidra/Processors/x86/data/languages/x86win.cspec

Lines 155 to 157 in c6cb8a0

	<prototype name="__thiscall" extrapop="unknown" stackshift="4">
	<input thisbeforeretpointer="true">
	<pentry minsize="1" maxsize="4">

[gipi]

I'm sorry but I don't understand: is it the attribute retbeforethis or thisbeforeretpointer? because the first doesn't exist anywhere in the ghidra source code. Moreover thisbeforeretpointer is the opposite of what I want. In theory from the return type ghidra should understand that the object returned should go as a first argument but I assumed that the decompiler is not able to do that automatically (too complicated?) this is why I talked about call fixups.

I'm working on an ARM32 binary so I don't know if it works for x86.

[astrelsky]

I'm sorry but I don't understand: is it the attribute retbeforethis or thisbeforeretpointer? because the first doesn't exist anywhere in the ghidra source code. Moreover thisbeforeretpointer is the opposite of what I want. In theory from the return type ghidra should understand that the object returned should go as a first argument but I assumed that the decompiler is not able to do that automatically (too complicated?) this is why I talked about call fixups.

I'm working on an ARM32 binary so I don't know if it works for x86.

Oh wow, I didn't even notice. I will confirm later but I think it is backwards. I think I have always gotten the behavior you are looking for where the return pointer came before the this pointer.

The name is accurate so setting it to false should do what you're looking for.

[ddkwork]

There is an attribute for prototype in the cspec called retbeforethis that should be set to true. I think there is an example of it being used in the x86 cspecs somewhere.

The callfixups are for things like compiler generated functions that have side effects the decompiler needs to know about that it can't see. An example is alloca or chkstack which alter the stack pointer. You can add a program specific specification extension for ones not provided with Ghidra. I personally use these all the time for functions such ass __allmul on 32 bit x86 to have the decompiler replace the function call with multiplication.

ghidra/Ghidra/Processors/x86/data/languages/x86win.cspec

Lines 155 to 157 in c6cb8a0

	<prototype name="__thiscall" extrapop="unknown" stackshift="4">
	<input thisbeforeretpointer="true">
	<pentry minsize="1" maxsize="4">

Hi, not working

[astrelsky]

You can't have both thisbeforeret and retbeforethis set to true.

[ddkwork]

You can't have both and set to true.thisbeforeret``retbeforethis

thx,i will try. and i maked a rep for test this:
https://github.com/ddkwork/asm

[ddkwork]

You can't have both and set to true.thisbeforeret``retbeforethis

this label seems not a right label or node title ?

[Wall-AF]

The attribute you're looking for is thisbeforeretpointer and appears in:

D:\...\Ghidra\ghidra>findstr /s /n  thisbeforeretpointer *.java *.h *.hh *.hp* *.inc *.inl *.c *.cc *.cp* *.asm *.sinc *.*spec *.opinion *.sla *.rxg *.xml  | more
Ghidra\Features\Decompiler\src\decompile\cpp\fspec.cc:35:AttributeId ATTRIB_THISBEFORERETPOINTER = AttributeId("thisbeforeretpointer",128);
Ghidra\Features\Decompiler\src\decompile\cpp\fspec.hh:44:extern AttributeId ATTRIB_THISBEFORERETPOINTER;        ///< Marshaling attribute "thisbeforeretpointer"
Ghidra\Features\Decompiler\src\main\doc\cspec.xml:1428:  <td><code>thisbeforeretpointer</code></td>
Ghidra\Features\Decompiler\src\main\doc\cspec.xml:1540:The <emphasis>thisbeforeretpointer</emphasis> indicates how the two hidden parameters, the
Ghidra\Framework\SoftwareModeling\data\languages\compiler_spec.rxg:458:        <optional> <attribute name="thisbeforeretpointer"/> </optional>
Ghidra\Framework\SoftwareModeling\src\main\java\ghidra\program\model\lang\ParamListStandard.java:344:           attribute = mainel.getAttribute("thisbeforeretpointer");
Ghidra\Framework\SoftwareModeling\src\main\java\ghidra\program\model\pcode\AttributeId.java:209:                new AttributeId("thisbeforeretpointer", 128);
Ghidra\Processors\x86\data\languages\x86-64-win.cspec:126:    <input pointermax="8" thisbeforeretpointer="true">
Ghidra\Processors\x86\data\languages\x86win.cspec:157:    <input thisbeforeretpointer="true">

[ddkwork]

The attribute you're looking for is thisbeforeretpointer and appears in:

D:\...\Ghidra\ghidra>findstr /s /n  thisbeforeretpointer *.java *.h *.hh *.hp* *.inc *.inl *.c *.cc *.cp* *.asm *.sinc *.*spec *.opinion *.sla *.rxg *.xml  | more
Ghidra\Features\Decompiler\src\decompile\cpp\fspec.cc:35:AttributeId ATTRIB_THISBEFORERETPOINTER = AttributeId("thisbeforeretpointer",128);
Ghidra\Features\Decompiler\src\decompile\cpp\fspec.hh:44:extern AttributeId ATTRIB_THISBEFORERETPOINTER;        ///< Marshaling attribute "thisbeforeretpointer"
Ghidra\Features\Decompiler\src\main\doc\cspec.xml:1428:  <td><code>thisbeforeretpointer</code></td>
Ghidra\Features\Decompiler\src\main\doc\cspec.xml:1540:The <emphasis>thisbeforeretpointer</emphasis> indicates how the two hidden parameters, the
Ghidra\Framework\SoftwareModeling\data\languages\compiler_spec.rxg:458:        <optional> <attribute name="thisbeforeretpointer"/> </optional>
Ghidra\Framework\SoftwareModeling\src\main\java\ghidra\program\model\lang\ParamListStandard.java:344:           attribute = mainel.getAttribute("thisbeforeretpointer");
Ghidra\Framework\SoftwareModeling\src\main\java\ghidra\program\model\pcode\AttributeId.java:209:                new AttributeId("thisbeforeretpointer", 128);
Ghidra\Processors\x86\data\languages\x86-64-win.cspec:126:    <input pointermax="8" thisbeforeretpointer="true">
Ghidra\Processors\x86\data\languages\x86win.cspec:157:    <input thisbeforeretpointer="true">

To be honest I'm not sure this attribute would allow the decompiler to replace x86 multiplication and division with 64-bit /* / % and so on and so forth for these math symbols. As I understand it, removing the implementation of mul and div is just one step, it's more about combining the two high and low 32-bit integers of the passed formal parameter into a single 64-bit one, so ideally there should be a lot less upper level code to see calls to decompile mul and div.

Also, it would be great if you could make a unit test handy, thanks.

[Wall-AF]

Sorry, but I now don't understand what it is you're attempting. I thought/understood that you were looking for the correct mechanism to place a pointer before the this pointer. You would only do this if you'd already established that the code you're re'ing came from a compiler that specified that "for returned objects larger than the standard return size, this compiler creates an object of the appropriate size (somewhere) and uses a hidden pointer placed on the stack to allow function calls to return them" or words to that effect.

If you're specifically asking how to replace function calls the compiler uses for mathematical operations, I'd also like to see an example please @astrelsky.

[astrelsky]

Sorry there is no retbeforethis you just do thisbeforeretpointer=false

As for the callfixup for replacing function calls with mathematical operations, it's pretty simple. This one was for arm so you'll need to change the register names and adjust the stack if applicable.

NOTE: it's an xml file but GitHub doesn't like that. Just add it in the Specification Extensions for in the "Options for {program}" window.
__sdivsi3.xml.txt

[ddkwork]

Sorry, but I now don't understand what it is you're attempting. I thought/understood that you were looking for the correct mechanism to place a pointer before the this pointer. You would only do this if you'd already established that the code you're re'ing came from a compiler that specified that "for returned objects larger than the standard return size, this compiler creates an object of the appropriate size (somewhere) and uses a hidden pointer placed on the stack to allow function calls to return them" or words to that effect.

If you're specifically asking how to replace function calls the compiler uses for mathematical operations, I'd also like to see an example please @astrelsky.

I am doing test this logic:
https://github.com/ddkwork/asm/blob/master/main.c#L44-L473

[ddkwork]

Sorry there is no retbeforethis you just do thisbeforeretpointer=false

As for the callfixup for replacing function calls with mathematical operations, it's pretty simple. This one was for arm so you'll need to change the register names and adjust the stack if applicable.

NOTE: it's an xml file but GitHub doesn't like that. Just add it in the Specification Extensions for in the "Options for {program}" window.
__sdivsi3.xml.txt

Not working, could you please test it? Thx

hi, it still not working,step here:
1. msbuild x86 build ../ get ./untitled4.exe
2. fix x86win.cspec
   1.<input thisbeforeretpointer="false">
   2.
      <callfixup name="__sdivsi3">
      <target name="__sdivsi3"/>
      <pcode>
      <body> r0 = r0 s/ r1; </body>
      </pcode>
      </callfixup>
3. drop ./untitled4.exe into ghidra got this:

we can see the mul and div function is not replaced as * and / or % operator.

///////////////////////////////////
 lVar11 += CONCAT44(local_10f,local_13f);
  lVar11 = mul((uint)lVar11,(uint)((ulonglong)lVar11 >> 0x20),2,0);
  lVar12 = mul(uVar9 + 0xf366,0,0x1634,0);
  lVar12 = mul(uVar5 + 1,local_13f + CARRY4((uint)lVar12,uVar9) + (uint)(0xfffffffe < uVar5),
               (uint)local_127,local_127._4_4_);
...

  lVar11 = div((uint)local_127,local_127._4_4_,0x6381be9a,0);
///////////////////////////////////

https://github.com/ddkwork/asm/blob/master/bigNumber%2FbigNumber_test.go#L12-L36

[ddkwork]

This is the expected unit test data for this behavior

func Test_bigNumber(t *testing.T) {
        data := []byte{0x9, 0x99, 0x8a, 0x7b, 0xfe, 0x46, 0xc2, 0xf0}
        b := bigNumber(data)
        mylog.Struct(b)
        assert.Equal(t, uint64(0xCD2F21A91272EE20), b.QuadPart())
        assert.Equal(t, mylog.Check2(hex.DecodeString("CD2F21A91272EE20")), b.Bytes())
}

https://github.com/ddkwork/asm/blob/master/demo%2Fdemo_test.go#L11-L17

It is a validation of x86 inline assembly, which gets its data from the results of c code inline asm inputs and outputs, asm is here, thanks.

https://github.com/ddkwork/asm/blob/master/main.c#L44-L473

[astrelsky]

Sorry there is no retbeforethis you just do thisbeforeretpointer=false

As for the callfixup for replacing function calls with mathematical operations, it's pretty simple. This one was for arm so you'll need to change the register names and adjust the stack if applicable.

NOTE: it's an xml file but GitHub doesn't like that. Just add it in the Specification Extensions for in the "Options for {program}" window.
__sdivsi3.xml.txt

Not working, could you please test it? Thx

hi, it still not working,step here:
1. msbuild x86 build ../ get ./untitled4.exe
2. fix x86win.cspec
   1.<input thisbeforeretpointer="false">
   2.
      <callfixup name="__sdivsi3">
      <target name="__sdivsi3"/>
      <pcode>
      <body> r0 = r0 s/ r1; </body>
      </pcode>
      </callfixup>
3. drop ./untitled4.exe into ghidra got this:

we can see the mul and div function is not replaced as * and / or % operator.

///////////////////////////////////
 lVar11 += CONCAT44(local_10f,local_13f);
  lVar11 = mul((uint)lVar11,(uint)((ulonglong)lVar11 >> 0x20),2,0);
  lVar12 = mul(uVar9 + 0xf366,0,0x1634,0);
  lVar12 = mul(uVar5 + 1,local_13f + CARRY4((uint)lVar12,uVar9) + (uint)(0xfffffffe < uVar5),
               (uint)local_127,local_127._4_4_);
...

  lVar11 = div((uint)local_127,local_127._4_4_,0x6381be9a,0);
///////////////////////////////////

https://github.com/ddkwork/asm/blob/master/bigNumber%2FbigNumber_test.go#L12-L36

1. thisbeforeret is not applicable here since it is not a thiscall function.
2. User error. I explicitly specified that that was for arm. Arm is not x86. Change the register names.

[ddkwork]

Hi, I didn't understand you correctly the other day, are you saying that this can be achieved by modifying the registers? I can't select the callfixup I just added in the function signature edit window because it's not in the list.

<callfixup name="__sdivsi3">
      <target name="__sdivsi3"/>
      <pcode>
      <body> eax = eax s/ ecx; </body>
      </pcode>
</callfixup>

<callfixup name="__mul">
      <target name="__mul"/>
      <pcode>
      <body> eax = eax s* ecx; </body>
      </pcode>
</callfixup>

[astrelsky]

Hi, I didn't understand you correctly the other day, are you saying that this can be achieved by modifying the registers? I can't select the callfixup I just added in the function signature edit window because it's not in the list.

<callfixup name="__sdivsi3">
      <target name="__sdivsi3"/>
      <pcode>
      <body> eax = eax s/ ecx; </body>
      </pcode>
</callfixup>

<callfixup name="__mul">
      <target name="__mul"/>
      <pcode>
      <body> eax = eax s* ecx; </body>
      </pcode>
</callfixup>

Did you add it to the program in edit->Options for 'program name'->Specification Extensions in the code browser?

[astrelsky]

I think the x86 registers need to be uppercase. You might need to have each extension in its own file too.

[ddkwork]

The arithmetic should be this logic, I don't know how to populate this correctly :)

Eax*ecx. return uint64(eax)<<32|edx

/
Eax/ecx. Return uint64(eax)<<32|edx

[astrelsky]

If you are working with something written in go, you should be using the golang compiler specification.