Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix parsing foreign currency strings #1074

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

itingliu
Copy link
Contributor

@itingliu itingliu commented Dec 7, 2024

Currently parsing a currency string would fail if the currency code does not match FormatStyle's locale region.

For example,

let style = Decimal.FormatStyle.Currency(code: "GBP", locale: .init(identifier: "en_US")).presentation(.isoCode)

This formats 3.14 into "GBP\u{0xa0}3.14", but parsing such string fails.

Fix this by always set the ICU formatter's currency code.

Resolves rdar://138179737

@itingliu
Copy link
Contributor Author

itingliu commented Dec 7, 2024

@swift-ci please test

@itingliu itingliu requested a review from theMomax December 7, 2024 01:13
@itingliu itingliu marked this pull request as draft December 7, 2024 01:22
@itingliu
Copy link
Contributor Author

itingliu commented Dec 7, 2024

@swift-ci please test


let frenchPrice = "57 379 €"
_verifyMatching(frenchPrice, formatStyle: frenchStyle, expectedUpperBound: frenchPrice.endIndex, expectedValue: 57379)
_verifyMatching(frenchPrice, formatStyle: floatStyle.locale(frFR), expectedUpperBound: frenchPrice.endIndex, expectedValue: 57379)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests incorrectly assumed that a currency style with "USD" code but "fr_FR" locale can correctly match a currency string with EUR symbol ("57 379,00 €"). This isn't correct, and is exactly what this bug is about: we only respect the locale regional currency, but ignore the currency code.

Copy link
Contributor

@theMomax theMomax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good! I think we just need to make sure the serialization format is backwards compatible, then this should be good to go!

Currently parsing a currency string would fail if the currency code does not match `FormatStyle`'s locale region. For example,

```swift
let style = Decimal.FormatStyle.Currency(code: "GBP", locale: .init(identifier: "en_US")).presentation(.isoCode)
```

This formats 3.14 into "GBP\u{0xa0}3.14", but parsing such string fails.

Fix this by always set the ICU formatter's currency code.

Resolves rdar://138179737
Copy link
Contributor Author

@itingliu itingliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach better than the previous one. We already have the currency code stored in the FormatStyle as a top-level property. There's no need to store it again in the nested configuration.

Also this should address @theMomax your concern about changing Codable representation. This should not change the serialized output anymore since we're not touching any Codable type in public types.

@itingliu
Copy link
Contributor Author

@swift-ci please test

@itingliu
Copy link
Contributor Author

@swift-ci please test

@itingliu itingliu marked this pull request as ready for review December 17, 2024 05:25
Copy link
Contributor

@theMomax theMomax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new approach is definitely better because I think people serializing FormatStyles is much more common than serializing ParseStrategys. But I think we should still make it backwards compatible in any scenario. Could we make currencyCode an optional property on CurrencyFormatStyleConfiguration.Collection? That way you could still avoid writing out a custom codable conformance.

Comment on lines 123 to 126
enum NumberFormatType : Hashable, Codable {
case number(NumberFormatStyleConfiguration.Collection)
case percent(NumberFormatStyleConfiguration.Collection)
case currency(CurrencyFormatStyleConfiguration.Collection)
case currency(CurrencyFormatStyleConfiguration.Collection, /*currency code*/ String)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type is part of the Codable format of a number of public types conforming to ParseStrategy, e.g. IntegerParseStrategy. So while this solution means the Codable conformance for Decimal.FormatStyle.Currency is backwards compatible, your decoding test would still fail for IntegerParseStrategy.

Copy link
Contributor Author

@itingliu itingliu Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you meant this is a var in IntegerParseStrategy and FloatingPointParseStrategy and friends ... OK we need tests for that too. Thanks

Copy link
Contributor Author

@itingliu itingliu Dec 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Add codable test for ParseStrategy

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another attempt: In 9a1e617 I removed the stored property numberFormatType from IntegerParseStrategy and FloatingPointParseStrategy. That means the we no longer preserve numberFormatType in serialization, and numberFormatType in previous Codable representation will be ignored from now on.

This is ok because the information is already provided by the other variable, formatStyle. In other words, numberFormatType has always been redundant. I also liked this because the previous serialization output feels like we're leaking internal information.

There is indeed a slight chance of regression -- if you manually craft an archive where numberFormatType represents different configurations from that of formatStyle, previously we'd favor what's represented by numberFormatType, but now we'd favor formatStyle. I think this risk is low enough though. Besides, I don't think we ever guarantee compatibility for manually crafted archives.

@itingliu
Copy link
Contributor Author

Could we make currencyCode an optional property on CurrencyFormatStyleConfiguration.Collection? That way you could still avoid writing out a custom codable conformance.

I'm flipping back and forth. A custom Codable conformance feels brittle as well because anyone could easily change the struct without realizing the Codable implementation also needs to be updated. But on the other hand with a Codable implementation it at least makes it clear what properties are expected in the serialization

…eStrategy` and `FloatingPointParseStrategy`

These properties are redundant as the information is already available through the public variable `formatStyle`.
@itingliu
Copy link
Contributor Author

@swift-ci please test

Copy link
Contributor

@theMomax theMomax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a good solution, thank you!

@itingliu
Copy link
Contributor Author

@swift-ci please test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants