Phases of translation

[BengtGustafsson]

This issue is for discussing where in the phases of translation f/x-literal handling happens.

My personal idea is that in phase 3 where string literals are parsed a f/x literal is parsed to something that syntactically resembles a function call with the excavated literal and expression-fields as parameters, for example: f"Value: {val}" translates to __builtin_fliteral("Value: {}", val).

These function-call like token sequences pass through phase 4, the actual preprocessing macro replacement step and phase 5 which just translates characters to the execution character set.

In phase 6 which is all about string literal concatenation the function-call like tokens are treated as literals during concatenation and all their expression-fields are added after the complete concatenated string literal. For f-literals all of this is enclosed in a function call, which can be to std::format directly or a helper function depending on whether there exist x-literals. For instance: __builtin_fliteral("Value: {}", val) "Just a literal" __builtin_fliteral("Other {}", o) is translated to std::format("Value: {}Just a literalOther {}", val, o).

If the preprocessor actually works with just token streams phase 6 would have to match the parentheses after __builtin_fliteral including ignoring right parens inside nested string literals. In reality I think that it would be possible in most implementations to stick some additional information (actually a token count would suffice) on to the __builtin_fliteral token to avoid having to do this re-parsing.

[BengtGustafsson]

Unfortunately it would not be enough that the extraction performed in phase 3 sticks a token count to the _builtin_fliteral token as this count can be changed by the macro replacement in phase 4. From the experience with Clang it may not be easy to add functionality in phase 4 to update this token count. I will look into this soon.