Kex Integration #431
Open
Kex Integration #431
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Needs debugging and testing * load kex-runner jar from github (build.gradle.kts toplevel) * setup code for kex properties * KexErrorManager based on LLMErrorManager * KexProcessManager based on EvoSuiteProcessManager * Basic UI element (button for running kex) * kex works only for the class codeType (todo funciton and line if possible) * read resource files kex.policy and modules.info
use a provided kex path for now instead of downloading jar and adding dependency use ProcessBuilder (jdk) instead of OSProcessHandler (IJ sdk) use kex.py instead of building java command directly generate Report objects by reading generated java classes
Deleted stuff: running Kex through through OSProcessHandler (IJ sdk) running Kex with kex.py downloading kex from github in build.gradle.kts
arguably overkill solution of modify PsiMethodWrapper class added parameter names, types and return tyepes explicitly updated implementation for java and kotlin
For multimodule projects using TestSpark the correct module's build directory path is passed based on the location of code for which tests are generated
also fixed a bug with the way options were passed to kex subprocess made them a list of strings instead of a single big space separated string
This also includes a simple string based mapping to jvm types through type erasure
It no longer depends on the order of the methods generated
Show a warning if LLM returns a response that cannot be parsed into a test case (e.g., an explanation of this test case rather than a modification).
…or a line (JetBrains-Research#344) * fix update function * create availableForGeneration * ktlint * feat: add javadoc for `JavaPsiHelper.availableForGeneration` * feat: check for nullness of a PSI file in `TestSparkAction.update` * feat: update javadocs in `PsiComponents.kt` * feat: check for a class or method/func in `KotlinPsiHelper.availableForGeneration` * feat: add TODO to `ToolUtils` about a potential bug The bug is reflected in the issue JetBrains-Research#375. * feat: make `PsiHelper.getSurroundingLineNumber` return 1-based line numbers Before, the `KotlinPsiHelper` returned a 0-based line number which caused an issue with line-based test generation. The generated prompt contained a line above the selected one. * feat: implement line-based test generation with CUT as a context When there is no surrounding method about the selected line, we use the CUT as a context for this line. The CUT must always be present. Otherwise, the generation action should have been disabled for this line. * refactor: apply ktlint * feat: add `See` in TODO * feat: add TODO and surround $NAME in backticks in `linePrompt` template * feat: collect class constructor signatures in `PsiClassWrapper` * feat: remove backticks from `linePrompt` * feat: fill line-based test generation with additional context The line-based test generation that has a method as a context of the line now also accepts constructors of the containing class. * refactor: use `firstOrNull` for `cut` extraction * refactor: apply ktlint * fix: add required parameter to `ClassRepresentation` in tests * publish: core module version `4.0.0` The major version increased due to the change of the public API of `PromptGenerator.generatePromptForLine` method. --------- Co-authored-by: Vladislav Artiukhov <[email protected]>
arksap2002
force-pushed
the
edwin1729/improvement/kex-integration
branch
from
3435206 to
a919e65
Compare
|
I have a weird issue in Kex test generation. The indicator starts with |
pderakhshanfar
requested changes
Jan 15, 2025
src/main/kotlin/org/jetbrains/research/testspark/appstarter/TestSparkStarter.kt
Outdated
Show resolved
Hide resolved
|
Also another issue, When trying to generate tests by Kex on Kotlin, it generates some random Java code |
arksap2002
added
new feature
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Integrating Kex (Symbolic Execution) into TestSpark
Generates test cases using Kex and displays them in the TestSpark tool window
Summary of changes
The integration of kex follows the example of EvoSuite and LLM for teh most part, so I highlight just the kex specific parts.
Automatic folding (hiding behind
...) of all code other than test annotated methods in the tool window is implemented. This is using thePsiFilefor each test casePsiMethodWrapper(and implementations) because kex requires specification of the method for which tests are generated in a specific format. Namely it involves FQNs and an example is given here:org.example.Matrix::product(org.example.Matrix,java.util.function.BinaryOperator,java.util.function.BinaryOperator):org.example.MatrixNote also that types have been erased. Kex expects JVM types which is easy to get from Java types but not from kotlin. For this reason kotlin is not yet supported by kex.