How do you develop a new language module for Ghidra

[ghidra1]

We are in the process of reviewing when and how processor language specification files get compiled from a slaspec to a sla file. In doing so, we would like to understand how users go about the following tasks and what steps/methods they rely on to compile a language specification.

1. When developing a new language module do you work from an installation of Ghidra or a development pull from GitHub?
2. If developing a new language module using a Ghidra installation, do you simply create a new Ghidra module under the Ghidra/Processors directory or use Eclipse GhidraDev in some fashion? What steps do you follow?
3. After making changes to the spec file how do you recompile the language? Do you restart Ghidra after every language modification?
4. Do you utilize the Eclipse GhidraSleighEditor when modifying a sleigh language file?

Thank you for you input.

[astrelsky]

I'll mention this to a few people who have built some language modules to bring some more input.

As for myself, I don't think I'd be much help because I don't use Eclipse at all. I use vscode and basically dump all of the Ghidra modules into the classpath. As long as the data folder is included as an entry in the language server classpath it works fine (assuming I remember correctly, I haven't done it in a while now).

As for 3, I will always just run sleigh directly and restart Ghidra. I do it this way because I don't want to have everything disassembled again if it isn't necessary.

[thixotropist]

1. I work from a development pull from Github, with commits pushed to a Github fork
2. Eclipse is never used. New languages (CPU variants) are added in files like RISCV/data/languages/riscv.ldefs with supporting additions to data/languages *.sinc files. mscode is the preferred editor.
3. A full gradle build from source after each change to any Ghidra sinc file
4. The Eclipse GhidraSleighEditor is never used, nor is support/sleigh ever called directly

If you choose to update the way sla files are generated, would you please consider adding hooks to support:

• Better integration between processor-specific extensions to ElfExtension and sinc files, so that architecture (or language) attributes of the imported binary can appear as run-time @ifdef variables within the sinc files. You might be able to generate (and cache) the sla files at runtime rather than at Ghidra compile-time, reducing the size of the Ghidra distribution tarballs.
• Allow for processor-specific Java code to generate disassembly listings and pcode for selected instruction encodings. It's much easier to translate binutils' objdump disassembly logic from C to Java than to sleigh.

This last one is a bit of a stretch goal:

• User defined pcodeops currently generate CALLOTHER semantics. Can we mix-in regular function call semantics so the decompiler view can assign or override parameter types to user-defined pcodeops ?

[Zinfidel]

I want to preface my answers by saying that I am very new to writing language modules and modifying Ghidra in general, and that the tooling that is provided (Eclipse integration, GhidraDev, etc.) is absolutely invaluable to me. If the tooling was not available and I had to do it manually, I likely would never have gotten very far and given up a long time ago. I am not very familiar with Java and its build tools, and the tooling takes some of the agony out of that learning process.

1. I work from a release build most of the time because the target audience for my module are people who will only be downloading a release version of Ghidra to use (unfamiliar with or uninterested in dealing with git, basically).
2. I use GhidraDev. My module is disconnected from the Ghidra installation file tree, and I use the GhidraDev "Link to Ghidra" feature. The module I'm working on was actually started by someone else though, so I do not know how to create a module from scratch. I pulled the latest changes of the extant module, link to Ghidra in Eclipse, then I went from there.
3. I have fumbled around a lot to find out how to do this, but this is what I settled on:
   1. In Eclipse, I run the compile task from the ant build file after changes until I get it to compile.
   2. I click the debug button, choose to start with Ghidra, and GhidraDev/Eclipse does whatever magic is necessary to start Ghidra with a debug version of my module installed and ready to go.
   3. I load up a test program and test my changes
   4. I will make changes to the definition while debugging, and compile with the ant build task to verify that it compiles.
   5. I run the script in script manager to "hot" reload the language.
   6. After I do enough damage to my test programs to merit closing Ghidra, GOTO 1.
4. I think I do -- I installed the extension and what I got out of it was syntax/semantics/parsing via textmate (or was it xtext?). I open sinc files in Eclipse and get the colors/hover info. It is a bit slow but invaluable to me as a newbie.

[mumbel]

• When developing a new language module do you work from an installation of Ghidra or a development pull from GitHub?

For initial development with slaspec/sinc I have always developed within an unzipped release making the minimal directory structure manually. Easy access to processors for reference and support/sleigh. Once its mostly coming together and it makes sense to work on ldef/cspec/pspec I eventually add to a git branch and work out of eclipse relying on loading a binary to compile or running support/sleigh, rarely/never do I run a sleigh configuration from eclipse.

My biggest deciding factor is when is src/ included. Once I get far enough along for ELF or needing an analyzer is when I move from the zip to eclipse (full tree). I wouldn't consider data/ ever being a reason for me to really choose one or the other except just generally saving my work in github to avoid any data loss and not wanting to make a new repo.

• If developing a new language module using a Ghidra installation, do you simply create a new Ghidra module under the Ghidra/Processors directory or use Eclipse GhidraDev in some fashion? What steps do you follow?

I have not used GhidraDev for a processor only feature plugins like loaders/filesystems/analyzers. For initial dev there is just too much work to get the SLEIGH correct/functional enough to worry about launching ghidra where GhidraDev is useful. I think I have always started with mkdir -p Ghidra/Processors/<thing>/data/languages. Once its further along either fill in the blanks for in tree or copy the skeleton and rename things and remove cruft. For RISC-V and i960, I created a PR branch and a module repo.

• After making changes to the spec file how do you recompile the language? Do you restart Ghidra after every language modification?

If I'm fixing compilation errors, I'll just run support/sleigh and won't run ghidra until it does compile. Typically this is starting with instructions, tokens, display, patterns and unimpl or { }. Slowly adding in tables and implementations ill support/sleigh and then launch ghidra to test disassembly or view pcode.

If bugfixing, I still support/sleigh to make sure no compilation errors and validate an instruction/pcode in ghidra afterwards if the fix does not look super straight forward.

• Do you utilize the Eclipse GhidraSleighEditor when modifying a sleigh language file?

I have not. I'm typically do not develop in Eclipse and I'm not a power user. Java development sure, but I'd rather just use my editor of choice for SLEIGH (day to day coding is C/python/asm in text editor, not really any IDE).