Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(xrpl): custom definitions support #2683

Open
wants to merge 20 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
23411ac
feat: support for custom definitions on xrpl methods: sign, encode, e…
elmurci Apr 21, 2024
9249824
chore: tests
elmurci Apr 23, 2024
b529149
docs(history): xrpl history update
elmurci Apr 23, 2024
a618fb9
chore: maintain linebreak
elmurci Apr 23, 2024
b283fc3
chore: keep source.fixAll.eslint value
elmurci Apr 23, 2024
4689b02
chore: typo
elmurci Apr 23, 2024
59c2074
Update packages/xrpl/src/models/transactions/transaction.ts
elmurci May 7, 2024
6089035
Update packages/xrpl/src/models/transactions/transaction.ts
elmurci Aug 19, 2024
7021e63
Support custom definitions for `client.submit()`, `client.submitAndWa…
tequdev Aug 23, 2024
02a8503
fix some lint error
tequdev Aug 23, 2024
3153659
Merge pull request #1 from tequdev/xrpl_binary_codec_custom_definitio…
elmurci Aug 25, 2024
b047f04
feat: addresses https://github.com/XRPLF/xrpl.js/pull/2683#discussion…
elmurci Aug 26, 2024
eeb9f43
Merge branch 'main' into xrpl_binary_codec_custom_definition_support
elmurci Aug 26, 2024
2900ab3
Update packages/xrpl/src/client/index.ts
mvadari Sep 5, 2024
2c60a1e
fix: address lint errors
elmurci Sep 13, 2024
071beb5
docs: update history on xrpl package
elmurci Nov 21, 2024
04446ff
chore: typo validateTxAgainstCustomDefintions
elmurci Nov 21, 2024
2452093
chore: remove getDefinitins function, https://github.com/XRPLF/xrpl.j…
elmurci Jan 9, 2025
4731f60
docs: add information about the new definition parameter, https://git…
elmurci Jan 9, 2025
5db85fe
docs: typo, https://github.com/XRPLF/xrpl.js/pull/2683#discussion_r18…
elmurci Jan 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions packages/ripple-binary-codec/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Functions to encode/decode to/from the ripple [binary serialization format](http
```


### decode(binary: string): object
### decode(binary: string, definitions?: XrplDefinitionsBase): object
elmurci marked this conversation as resolved.
Show resolved Hide resolved
Decode a hex-string into a transaction object.
```js
> api.decode('1100612200000000240000000125000000072D0000000055DF530FB14C5304852F20080B0A8EEF3A6BDD044F41F4EBBD68B8B321145FE4FF6240000002540BE4008114D0F5430B66E06498D4CEEC816C7B3337F9982337')
Expand All @@ -26,7 +26,7 @@ Decode a hex-string into a transaction object.
}
```

### encode(json: object): string
### encode(json: object, definitions?: XrplDefinitionsBase): string
Encode a transaction object into a hex-string. Note that encode filters out fields with undefined values.
```js
> api.encode({
Expand All @@ -37,12 +37,12 @@ Encode a transaction object into a hex-string. Note that encode filters out fiel
OwnerCount: 0,
PreviousTxnID: 'DF530FB14C5304852F20080B0A8EEF3A6BDD044F41F4EBBD68B8B321145FE4FF',
Balance: '10000000000',
Account: 'rLs1MzkFWCxTbuAHgjeTZK4fcCDDnf2KRv'
Account: 'rLs1MzkFWCxTbuAHgjeTZK4fcCDDnf2KRv'
})
'1100612200000000240000000125000000072D0000000055DF530FB14C5304852F20080B0A8EEF3A6BDD044F41F4EBBD68B8B321145FE4FF6240000002540BE4008114D0F5430B66E06498D4CEEC816C7B3337F9982337'
```

#### X-Address Compatibility
#### X-Address Compatibility
* ripple-binary-codec handles X-addresses by looking for a few specific files (Account/SourceTag, Destination/DestinationTag).
* If other fields (in the future) must to support X-addresses with tags, this library will need to be updated.
* When decoding rippled binary, the output will always output classic address + tag, with no X-addresses. X-address support only applies when encoding to binary.
Expand All @@ -54,15 +54,15 @@ Encode a transaction object into a hex-string. Note that encode filters out fiel
* When _decoding_, if a currency code is three uppercase letters or numbers (`/^[A-Z0-9]{3}$/`), then it will be decoded into that string. For example,`0000000000000000000000004142430000000000` decodes as `ABC`.
* When decoding, if a currency code is does not match the regex, then it is not considered to be an ISO 4217 or pseudo-ISO currency. ripple-binary-codec will return a 160-bit hex-string (40 hex characters). For example, `0000000000000000000000006142430000000000` (`aBC`) decodes as `0000000000000000000000006142430000000000` because it contains a lowercase letter.

### encodeForSigning(json: object): string
### encodeForSigning(json: object, definitions?: XrplDefinitionsBase): string

Encode the transaction object for signing.

### encodeForSigningClaim(json: object): string

Encode the transaction object for payment channel claim.

### encodeForMultisigning(json: object, signer: string): string
### encodeForMultisigning(json: object, signer: string, definitions?: XrplDefinitionsBase): string

Encode the transaction object for multi-signing.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello,
For the sake of completeness, aren't you interested in modifying the signatures of the below two functions?

decodeLedgerData(binary: string): object
encodeForSigningClaim(json: object): string

Aren't custom definitions useful for these methods?


Expand All @@ -72,7 +72,7 @@ Encode the transaction object for multi-signing.
'5D06F4C3362FE1D0'
```

### decodeQuality(value: string): string
### decodeQuality(value: string): string
```js
> api.decodeQuality('5D06F4C3362FE1D0')
'195796912.5171664'
Expand Down
3 changes: 3 additions & 0 deletions packages/xrpl/HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@ Subscribe to [the **xrpl-announce** mailing list](https://groups.google.com/g/xr
* Add missing `lsfAMMNode` flag to `RippleState` ledger object
* Add `PreviousFields` to `DeletedNode` metadata type

### Added
elmurci marked this conversation as resolved.
Show resolved Hide resolved
* Custom definitions support for `util.encode`, `util.decode`, `util.encodeForSignning` and `Wallet.sign`.
elmurci marked this conversation as resolved.
Show resolved Hide resolved

## 3.0.0 (2024-02-01)

### BREAKING CHANGES
Expand Down
25 changes: 19 additions & 6 deletions packages/xrpl/src/Wallet/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ import {
encodeForSigning,
encodeForMultisigning,
encode,
XrplDefinitionsBase,
} from 'ripple-binary-codec'
import {
deriveAddress,
Expand Down Expand Up @@ -367,15 +368,17 @@ export class Wallet {
* @param this - Wallet instance.
* @param transaction - A transaction to be signed offline.
* @param multisign - Specify true/false to use multisign or actual address (classic/x-address) to make multisign tx request.
* @param definitions Custom rippled types to use instead of the default. Used for sidechains and amendments.
* @returns A signed transaction.
* @throws ValidationError if the transaction is already signed or does not encode/decode to same result.
* @throws XrplError if the issued currency being signed is XRP ignoring case.
*/
// eslint-disable-next-line max-lines-per-function -- introduced more checks to support both string and boolean inputs.
// eslint-disable-next-line max-lines-per-function, max-params -- introduced more checks to support string and boolean inputs.
public sign(
this: Wallet,
transaction: Transaction,
multisign?: boolean | string,
definitions?: XrplDefinitionsBase,
): {
tx_blob: string
hash: string
Expand Down Expand Up @@ -406,7 +409,7 @@ export class Wallet {
* This will throw a more clear error for JS users if the supplied transaction has incorrect formatting
*/
// eslint-disable-next-line @typescript-eslint/consistent-type-assertions -- validate does not accept Transaction type
validate(tx as unknown as Record<string, unknown>)
validate(tx as unknown as Record<string, unknown>, definitions)

const txToSignAndEncode = { ...tx }

Expand All @@ -420,20 +423,24 @@ export class Wallet {
txToSignAndEncode,
this.privateKey,
multisignAddress,
definitions,
),
}
txToSignAndEncode.Signers = [{ Signer: signer }]
} else {
txToSignAndEncode.TxnSignature = computeSignature(
txToSignAndEncode,
this.privateKey,
undefined,
definitions,
)
}

const serialized = encode(txToSignAndEncode)
const serialized = encode(txToSignAndEncode, definitions)

return {
tx_blob: serialized,
hash: hashSignedTx(serialized),
hash: hashSignedTx(serialized, definitions),
}
}

Expand Down Expand Up @@ -466,22 +473,28 @@ export class Wallet {
* @param tx - A transaction to sign.
* @param privateKey - A key to sign the transaction with.
* @param signAs - Multisign only. An account address to include in the Signer field.
* @param definitions Custom rippled types to use instead of the default. Used for sidechains and amendments.
* Can be either a classic address or an XAddress.
* @returns A signed transaction in the proper format.
*/
// eslint-disable-next-line max-params -- Needs 4 params
function computeSignature(
tx: Transaction,
privateKey: string,
signAs?: string,
definitions?: XrplDefinitionsBase,
): string {
if (signAs) {
const classicAddress = isValidXAddress(signAs)
? xAddressToClassicAddress(signAs).classicAddress
: signAs

return sign(encodeForMultisigning(tx, classicAddress), privateKey)
return sign(
encodeForMultisigning(tx, classicAddress, definitions),
privateKey,
)
}
return sign(encodeForSigning(tx), privateKey)
return sign(encodeForSigning(tx, definitions), privateKey)
}

/**
Expand Down
44 changes: 42 additions & 2 deletions packages/xrpl/src/client/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

/* eslint-disable max-lines -- Client is a large file w/ lots of imports/exports */
import { EventEmitter } from 'eventemitter3'
import { XrplDefinitionsBase } from 'ripple-binary-codec'

import {
RippledError,
Expand Down Expand Up @@ -218,6 +219,12 @@ class Client extends EventEmitter<EventTypes> {
*/
public buildVersion: string | undefined

/**
* Custom rippled types to use instead of the default. Used for sidechains and amendments.
*
*/
public definitions: XrplDefinitionsBase | undefined

/**
* API Version used by the server this client is connected to
*
Expand Down Expand Up @@ -526,6 +533,33 @@ class Client extends EventEmitter<EventTypes> {
}
}

/**
* Get Definitions from server_definitions
*
* @returns void
* @example
* ```ts
* const { Client } = require('xrpl')
* const client = new Client('wss://s.altnet.rippletest.net:51233')
* await client.getDefinitions()
* console.log(client.definitions)
* ```
*/
public async getDefinitions(): Promise<void> {
elmurci marked this conversation as resolved.
Show resolved Hide resolved
try {
const response = await this.request({
command: 'server_definitions',
})
this.definitions = new XrplDefinitionsBase(
JSON.parse(JSON.stringify(response.result)),
{},
)
} catch (error) {
// eslint-disable-next-line no-console -- Print the error to console but allows client to be connected.
console.error(error)
}
}
elmurci marked this conversation as resolved.
Show resolved Hide resolved

/**
* Tells the Client instance to connect to its rippled server.
*
Expand Down Expand Up @@ -760,7 +794,10 @@ class Client extends EventEmitter<EventTypes> {
wallet?: Wallet
},
): Promise<SubmitResponse> {
const signedTx = await getSignedTx(this, transaction, opts)
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: this.definitions,
})
Comment on lines +770 to +773
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Avoid overwriting user-provided definitions in submit

By spreading ...opts and then setting definitions: this.definitions, any definitions provided by the user in opts will be overwritten by this.definitions. To respect user-supplied definitions, consider prioritizing opts.definitions:

 const signedTx = await getSignedTx(this, transaction, {
   ...opts,
-  definitions: this.definitions,
+  definitions: opts?.definitions ?? this.definitions,
 })

This change ensures that if the user provides definitions in opts, it will be used; otherwise, this.definitions will be applied.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: this.definitions,
})
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: opts?.definitions ?? this.definitions,
})

return submitRequest(this, signedTx, opts?.failHard)
}

Expand Down Expand Up @@ -834,7 +871,10 @@ class Client extends EventEmitter<EventTypes> {
wallet?: Wallet
},
): Promise<TxResponse<T>> {
const signedTx = await getSignedTx(this, transaction, opts)
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: this.definitions,
})
Comment on lines +847 to +850
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consistent handling of definitions in submitAndWait

Similar to the submit method, the submitAndWait method overwrites any user-provided definitions in opts. Apply the same fix to ensure consistency and respect for user input:

 const signedTx = await getSignedTx(this, transaction, {
   ...opts,
-  definitions: this.definitions,
+  definitions: opts?.definitions ?? this.definitions,
 })
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: this.definitions,
})
const signedTx = await getSignedTx(this, transaction, {
...opts,
definitions: opts?.definitions ?? this.definitions,
})


const lastLedger = getLastLedgerSequence(signedTx)
if (lastLedger == null) {
Expand Down
26 changes: 25 additions & 1 deletion packages/xrpl/src/models/transactions/common.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { isValidClassicAddress, isValidXAddress } from 'ripple-address-codec'
import { TRANSACTION_TYPES } from 'ripple-binary-codec'
import { TRANSACTION_TYPES, XrplDefinitionsBase } from 'ripple-binary-codec'

import { ValidationError } from '../../errors'
import {
Expand Down Expand Up @@ -351,6 +351,30 @@ export function validateBaseTransaction(common: Record<string, unknown>): void {
validateOptionalField(common, 'NetworkID', isNumber)
}

/**
* Validate that the passed transaction is a valid type against the types provided by the custom definitions.
*
* @param tx - A Transaction.
* @param definitions - Custom definitions
* @throws When the passed transaction type is not found in the definitions.
*/
export function validateTxAgainstCustomDefintions(
intelliot marked this conversation as resolved.
Show resolved Hide resolved
tx: Record<string, unknown>,
definitions: XrplDefinitionsBase,
): void {
// Validate just transaction type for now, leaving it open for further validations against the custom definition spec.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you explain what else is remaining as far as validation is concerned? Are you referring to validation of the SFields used inside a transaction ? (This is referred to as FIELDS in the definitions.json file)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that was the idea. You could introduce further validations in the future, like the one you mentioned.

const txType = tx.TransactionType
if (typeof txType !== 'string') {
throw new ValidationError(
'TransactionType field is not specified or not a string',
)
}

if (!definitions.transactionType[txType]) {
throw new ValidationError(`Invalid transaction type: ${txType}`)
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Missing test coverage for validateTxAgainstCustomDefintions function

The function validateTxAgainstCustomDefintions is used in the transaction validation flow but lacks dedicated test coverage. Specific test cases needed:

  • No tests found for validating transactions against custom definitions
  • The function is used as a fallback validation in transaction.ts when custom definitions are provided
  • Current implementation validates only the transaction type, but lacks tests for both valid and invalid scenarios
🔗 Analysis chain

Verify test coverage for the new validation function

Please ensure that unit tests cover:

  1. Valid transaction types
  2. Invalid transaction types
  3. Invalid input types (non-string TransactionType)
  4. Edge cases with custom definitions

Let's verify the test coverage:

Additionally, consider these architectural improvements:

  1. Validate the structure of custom definitions when they're provided
  2. Add validation for required fields specified in the custom definition
  3. Consider adding type validation for fields specified in the custom definition

Would you like assistance in implementing these improvements or creating the test cases?

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check test coverage for the new validation function

# Look for test files that might contain tests for this function
rg -l "validateTxAgainstCustom" test/

Length of output: 83


Script:

#!/bin/bash
# Let's first find the test directory location
fd -t d "test" .

# Then search for any test files containing the function name
rg -l "validateTxAgainstCustom" .

# Also search for any test files related to transaction validation
rg -l "transaction.*validation" .

Length of output: 511


Script:

#!/bin/bash
# Let's check the content of test files in xrpl package for any related tests
rg -A 5 -B 5 "validateTxAgainstCustom" ./packages/xrpl/test/

# Let's also check how this function is used in transaction.ts
rg -A 5 -B 5 "validateTxAgainstCustom" ./packages/xrpl/src/models/transactions/transaction.ts

# Check for any existing transaction validation tests
rg -A 5 -B 5 "describe.*transaction.*validation" ./packages/xrpl/test/

Length of output: 982


/**
* Parse the value of an amount, expressed either in XRP or as an Issued Currency, into a number.
*
Expand Down
24 changes: 19 additions & 5 deletions packages/xrpl/src/models/transactions/transaction.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
/* eslint-disable max-lines -- need to work with a lot of transactions in a switch statement */
/* eslint-disable max-lines-per-function -- need to work with a lot of Tx verifications */

import { XrplDefinitionsBase } from 'ripple-binary-codec'

import { ValidationError } from '../../errors'
import { IssuedCurrencyAmount, Memo } from '../common'
import { isHex } from '../utils'
Expand All @@ -18,7 +20,11 @@ import { CheckCancel, validateCheckCancel } from './checkCancel'
import { CheckCash, validateCheckCash } from './checkCash'
import { CheckCreate, validateCheckCreate } from './checkCreate'
import { Clawback, validateClawback } from './clawback'
import { BaseTransaction, isIssuedCurrency } from './common'
import {
BaseTransaction,
isIssuedCurrency,
validateTxAgainstCustomDefintions,
} from './common'
intelliot marked this conversation as resolved.
Show resolved Hide resolved
import { DepositPreauth, validateDepositPreauth } from './depositPreauth'
import { DIDDelete, validateDIDDelete } from './DIDDelete'
import { DIDSet, validateDIDSet } from './DIDSet'
Expand Down Expand Up @@ -170,10 +176,14 @@ export interface TransactionAndMetadata<
* Encode/decode and individual type validation.
*
* @param transaction - A Transaction.
* @param customDefinitions - Optional parameter to validate against a custom definition.
* @throws ValidationError When the Transaction is malformed.
* @category Utilities
*/
export function validate(transaction: Record<string, unknown>): void {
export function validate(
transaction: Record<string, unknown>,
customDefinitions?: XrplDefinitionsBase,
): void {
const tx = { ...transaction }
if (tx.TransactionType == null) {
throw new ValidationError('Object does not have a `TransactionType`')
Expand Down Expand Up @@ -407,8 +417,12 @@ export function validate(transaction: Record<string, unknown>): void {
break

default:
throw new ValidationError(
`Invalid field TransactionType: ${tx.TransactionType}`,
)
if (customDefinitions) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I happen to modify the behavior of an existing transaction (say TicketCreate), the custom-definitions will not help me. Since this transaction-name already exists in the codebase, the old transaction validation is used.

The existing validate_<transaction_name> methods are functionally different from validateTxAgainstCustomDefinitions method. The former enforces the TxFormat whereas the latter concerns itself with serialization issues.

Is my understanding correct ?

validateTxAgainstCustomDefintions(tx, customDefinitions)
elmurci marked this conversation as resolved.
Show resolved Hide resolved
} else {
throw new ValidationError(
elmurci marked this conversation as resolved.
Show resolved Hide resolved
`Invalid field TransactionType: ${tx.TransactionType}`,
)
}
}
}
Loading