Integrate LLVM Propeller

[sunnyflunk]

The content is following the build process from https://github.com/google/llvm-propeller using upstream clang

LLVM Propeller is an alternative to the LLVM BOLT approach which can minimize some of the negative aspects, particularly the large increase in file size! This makes it unattractive to apply BOLT outside of larger programs like clang or firefox (though these will receive the greatest benefit). As this requires LBR from perf, it currently only works well on an Intel machine.

This should be fairly straight forward to integrate and would look much the same as CSPGO (but with different flags). There's probably a couple of issues to address through testing to make sure it is integrated correctly.

Currently waiting on the LLVM 16 release (but may test with a git release), it was tested with the git version and suggested a fraction better performance with minimal increase in file size. It also used perf for BOLT which we typically don't.

Some testing needs to occur to ensure the best performance:

• Investigate the use of -funique-internal-linkage-names and -Wl,-z,keep-text-section-prefix and whether to use it with BOLT as well
  • Does it increase binary size?
    • Even after stripping?
  • Does it increase performance when used with BOLT?
• Does creating the perf profile with a binary without -Wl,-q matter? (this is what occurred in the test suggesting propeller was better and not what we do)

To integrate propeller we need:

• To add create_llvm_prof from https://github.com/google/autofdo
• Plumb in an additional set of stages for Propeller instrumentation
  • Add these flags to the generate stage:
    • C/CXXFLAGS: -funique-internal-linkage-names -fbasic-block-sections=labels
    • LDFLAGS: -fuse-ld=lld -Wl,--lto-basic-block-sections=labels
  • Add in macros to generate perf profile and converting to the propeller format
  • Add these flags to the final stage:
    • C/CXXFLAGS: -funique-internal-linkage-names -fbasic-block-sections=list=${PATH_TO_PROFILES}/cluster.txt
    • LDFLAGS: -fuse-ld=lld -Wl,--lto-basic-block-sections=${PATH_TO_PROFILES}/cluster.txt -Wl,--symbol-ordering-file=${PATH_TO_PROFILES}/symorder.txt -Wl,--no-warn-symbol-ordering -Wl,-gc-sections -Wl,-z,keep-text-section-prefix
• Test performance for use in clang
• Test the impacts on performance and file size on some smaller programs we use with PGO already