perl:data-munge : v0.10 is sunk by error #697
Comments
|
This is upstream problem. Versions are not numbers and |
|
can you please explain this clearly? It's not "0.97", it is "0.097". You are off by a facter of 10. what do you mean "0.097" equals "0.97"? that makes zero sense. What do you mean "versions are not numbers" ? Ever single character in the version is a digit or a period. It's a number Can you please explain what the actual problem is? |
|
multiply both versions by 1000. It's basic math. |
|
I'm starting to figure out what you are saying. |
|
I suggest repology software be improved to either
data::munge has a perfectly valid version scheme This shortcoming is clearly on repology's side. |
No I am not. Versions are neither text nor numbers, versions are versions and are compared in a special way. For repology, dedicated library was written which can handle the most cases meaningfully, and most package managers implement their own version comparison code. For all of these So this problem has nothing to do with Repology: data::munge versioning is not compatible with any mechanisms which use its version, so it is broken and should be fixed upstream. Repology does not fix stuff which needs to be fixed elsewhere. |
|
That is a compelling answer. It's a shame it took about 6 attempts to get it. edit: I emailed Lukas Mai. Hopefully he agrees to update the version number. |
Have fun telling every single Perl module author and the language itself that they need to change what they have been doing since about 1990. That is, old-style Perl version numbers are (fractional) numbers and need to be compared numerically. See also:
I don't think a change there is going to happen anytime soon. You'd have to patch every little deployed piece of code and utility that compares version numbers using
... except for the Perl ecosystem that it was developed for. PS: I like how your website says that CPAN has an "outdated version" when that is upstream (CPAN is the original release channel). |
|
I checked all four of these examples with "pkg version -t". All four show the wrong comparison on at least one part of the example. It's a pretty good rebuttal. |
|
That being said -- if all these package managers ave the same behavior, does that mean perl modules have been problematic for years? is this a long-standing well-known issue that package system users simple live with, knowing that their system will not properly detect when newer versons are available? |
There's no need to, as overwhelming majority of authors are using compatible versions (1). In fact, this is the first report of perl module version problem I've got in 5 years of repology and ~7k other reports.
There's no need to change any code or any rules, as just that compatible versions can/should be used. In this case, for instance, it could've been
Garbage in - garbage out, obviously. You can see though that in 98% cases CPAN has the newest version (and that only includes modules packaged in some repository) because (1). That's even more than PyPI which does not suffer from any version scheme ambiguities. Not in all cases CPAN or PyPI are actual upstreams, though, so <100% is expected for both. |
|
Until just now, I missed that the issue was the Data::Munge released it as
"0.10" and not "0.100". So the easiest path forward for this
individual software is re-releasing it as "0.100" or perhaps "0.101".
In fact, this is the first report of perl module version problem ...
To be fair, the number of reports isn't important. It's the number of
instances we're seeing this, which is easily calculated from the data. And
just because 98% shows the correct top version currently doesn't mean there
are many earlier versions of the same module there weren't being compared
properly.
I see both sides to this. On one hand, it's kind of absurd that 30 years
of legacy was ignored. On the other hand, all these package managers are
behaving the same way, so even if Repology was updated to respect the perl
versions, the problem still exists across all the repositories. It's
better than Repology highlight the problem.
unfortunately, the pass of least resistance is to talk to the perl module
authors I think. They aren't going to want their choice of version number
to break package maintenance -- especially when the solution is relatively
easy on their part.
…On Tue, Apr 25, 2023 at 6:46 AM Dmitry Marakasov ***@***.***> wrote:
Have fun telling every single Perl module author and the language itself
that they need to change what they have been doing since about 1990
There's no need to, as overwhelming majority of authors are using
compatible versions (1). In fact, this is the first report of perl module
version problem I've got in 5 years of repology and ~7k other reports.
You'd have to patch every little deployed piece of code and utility that
compares version numbers using <, which works just fine with the status quo.
There's no need to change any code or any rules, as just compatible
versions can be used. In this case, for instance, it could've been 0.097
→ 0.100.
PS: I like how your website
<https://repology.org/project/perl:data-munge/versions> says that CPAN
has an "outdated version" when that is upstream (CPAN is the original
release channel).
Garbage in - garbage out, obviously. You can see
<https://repology.org/repositories/statistics#cpan> though that in 98%
cases CPAN has the newest version (and that only includes modules packaged
in some repository) because (1). That's even more than PyPI which does not
suffer from any version scheme ambiguities. Not in all cases CPAN or PyPI
are actual upstreams, so <100% is expected for both.
—
Reply to this email directly, view it on GitHub
<#697 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAISZ5TNJEXB5WGMMLLJH53XC62SLANCNFSM6AAAAAAXJYC7L4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
Have you looked at those cases in detail? There's some interesting stuff there. For example:
That's just a few random results from the first page. (And in general, there are lots of cases where a distro version like
I disagree. I don't think it makes sense to rely on hundreds (or thousands) of module authors to manually work around bugs in automated versioning systems (and treating fractional numbers as multi-component versions is a bug). At some point, someone is going to slip up and things are going to break. Why expend all this effort when you could fix the code? To do it right, the versioning code would need to support two types of version: The multi-component format (where each part is compared separately as an integer) and the fractional number format (where the whole thing is a single number). If that's not possible, the ingestion code should pad each fractional number with 0s on the right, up to some fixed number of decimal places. That way |
It is obvious there are other similar cases, my point is that most Perl modules are versioned in a compatible way.
Repology just conveys what CPAN reports, like I've said "garbage in - garbage out". There are plans to drop CPAN in favor of MetaCPAN, but I'm still hesitant, as there are problems in MetaCPAN as well, and Repology can help reveal them.
About 179 of them ATOW. Definitely more sensible than changing all package managers (just Repology supports 86 different packaging infrastructures) and other software which compares versions to do the impossible thing (1).
Interpretation of the version in its conventional sense is definitely not a bug. I won't dispute the right of perl developers to use whatever they define version as, but you can see that both perl developers and consumers of their work benefit from versioning compatible with both worlds, as it allows
And there are no technical difficulties for using compatible versioning.
It is not technically possible as there's no way to know which type a given version belongs to (1). |
It does not, but earlier versions and irrelevant for the purposes we're discussing. |
|
@mauke Are you really not going to re-release as 0.100 or 0.101? I accept it's on the packager community to contact authors individually. Is it really a big deal to pick a version number that works univerally as well as perl? I don't see a technical solution because as AMDmi3 said, there's no way for the verson libraries to distinguish between perl and not-perl if only given the version string. |
|
@jrmarino I'm not sure what I'm looking at or what repology is. My understanding of the situation is that someone has built a process for handling versioned packages, and that process is broken: There are (at least) two incompatible ways of doing versions, the dotted integer format (A) and the fractional number format (B). Treating the latter as if it were the former results in the wrong version being marked as "latest" (which is what's happening here). To my mind, the proper fix is to change the process so it remembers whether any given version is of type A or B and compares them accordingly. If that's not feasible (too expensive), the process needs to convert all incoming versions into format A instead. But blindly processing format B as if it were format A makes no sense. I don't see any technical difficulties about this, either: At some point something is ingesting modules from CPAN. We know what type of versions is used on CPAN, so at that point we can mark or convert the version number as appropriate. (What am I missing?) Asking me to release a new version for no reason (well, no inherent reason, as there are no actual code changes) feels like the wrong way to go about it. You (that's the general "you", not you personally) have built a process that presumably is intended to solve a real-world problem. But the process is faulty: It treats data in format B as if it were format A, leading to unexpected results. And instead of changing the process so it handles the real world better, you try to change the real-world problem so it better conforms to your chosen solution. Isn't that backwards?
I mean, if you had a working process (like converting incoming CPAN version numbers from format B to format A), you wouldn't have to contact anyone individually. (At least not for this issue.)
It's a case of format A vs format B. Neither is universal. ("Universal" would include CPAN, and CPAN contains modules in both format A and format B.) The thing I'm morally offended by is being asked to manually work around a broken process that I didn't know about and that was imposed on me without asking. If it were just a one-time change, I'd have fewer objections, but this is essentially putting extra requirements on all of my future releases. Why is it suddenly my responsibility to ensure that the process you've chosen to use doesn't break? (Also, claiming that a fix is "not technically possible" seems rather disingenuous. Of course it's possible.)
Here "compatible" means "compatible with our broken process". There are no technical difficulties for you because you've externalized the required effort onto other people (i.e. module authors like me). On the other hand, there are no technical difficulties for me if your CPAN package import is fixed.
... so I think I am justified in saying: I do not (like to) fix stuff that needs to be fixed elsewhere. |
That's not case. First, it's not a single entity. It's pretty much a standard that dozens of mechanisms that compare versions are using. Secondly, while it's tempting to call it broken, it's really not.
We're getting into semantics, but yes, the two approaches can be incompatible. However, perl versions can fit into the worldview easily (you have to admit, 0.10 and 0.100 would be equivalent in perl versioning, so why not just use the instance the works everywhere?)
It's not possible. All you have are two version strings to compare. That's it. No other metadata.
What I asking you personally to do is recognize that your unknowing choice of "0.10" is breaking packages everywhere. You didn't know before, but you do now. So now that you know, you could easily fix the world by making a new release. But you're chosing a principled stance (with flawed assumptions) and leaving all packages of your software broken. They either have to fake the version like FreeBSD did, or not get the updates because of a version comparison failure.
I know you didn't know about it, but you do now. But while you are busy being "offended", countless users are being broken. I didn't write any of these libraries, but I do know enough to assure you that "the standard" can't handle "0.10" and "0.100" as being equivalent. There is no technical solution. There may be "hundreds" of perl packages like this, but that doesn't mean they are externally packaged. This is the only one I know about. I'm fine with the requirement to individually ask authors to use versioning that works everywhere. Do you really want to put out the users of your software? Your principled stance won't go anywhere - mainly because there is no solution. |
Cool, so why not use my second suggestion and convert from format B to format A when packaging CPAN modules?
Not everywhere; only those that didn't convert the version format when they packaged the module. (Which is a bit like packaging a Windows batch file to be run as a shell script, and then complaining to the author that some of the commands don't work in
[int_part, frac_part] = '1.10'.split('.')
dotted_version = int_part + '.' + frac_part.rstrip('0').ljust(9, '0')
# '1.100000000'(Plus maybe a sanity check that the padded fractional part doesn't exceed 9 characters¹.) What makes this approach infeasible? ¹) I chose 9 as the padding amount because it is unlikely for a module version to need more than 9 fractional digits and because the resulting number will always fit in a 32-bit integer, which is nice for C libraries that convert dotted components to integers in order to compare them. |
|
It's broken on hundreds of packagers. You are suggesting 100s of people
rewrite code for a handful of bad actors. Even if "the world" decided to do
that it would take months or even years to get that done. Hence the hack
freebsd did.
I wish you would just take my word that it's not technically possible. I
am not misleading you, not is amdri3.
Look, you've been informed. At this point you're knowing the source of a
problem. If you don't want to help out "the world", I can't force you. All
I can do is either hack like freebsd or see if I can remove data:: munge
from our repos easily.
John
…On Sat, May 6, 2023, 14:28 mauke ***@***.***> wrote:
It's not possible. All you have are two version strings to compare. That's
it. No other metadata.
Cool, so why not use my second suggestion and convert from format B to
format A when packaging CPAN modules?
What I asking you personally to do is recognize that your unknowing choice
of "0.10" is breaking packages everywhere.
Not everywhere; only those that didn't convert the version format when
they packaged the module.
(Which is a bit like packaging a Windows batch file to be run as a shell
script, and then complaining to the author that some of the commands don't
work in sh and could they please use only commands that do the same thing
in cmd.exe and sh?)
I didn't write any of these libraries, but I do know enough to assure you
that "the standard" can't handle "0.10" and "0.100" as being equivalent.
There is no technical solution.
[int_part, frac_part] = '1.10'.split('.')dotted_version = int_part + '.' + frac_part.rstrip('0').ljust(9, '0')# '1.100000000'
(Plus maybe a sanity check that the padded fractional part doesn't exceed
9 characters¹.)
What makes this approach infeasible?
------------------------------
¹) I chose 9 as the padding amount because it is unlikely for a module
version to need more than 9 fractional digits and because the resulting
number will always fit in a 32-bit integer, which is nice for C libraries
that convert dotted components to integers in order to compare them.
—
Reply to this email directly, view it on GitHub
<#697 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAISZ5UQX7CJV2KWTNHQ7E3XE2Q4PANCNFSM6AAAAAAXJYC7L4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Uh. You're asking me for a favor because your packaging process is broken and you want me to work around it. Calling me a "bad actor" is not the way to make that happen. |
|
Not "mine". The entire non-perl world (by your definition of broken). You've spent more time arguing for the sport of it then you would have spent resolving the problem. You're the only one that can resolve it. It's clear that you have no intention to do so. |
|
as an epilogue, I just checked our repository and nothing depends on data::munge. It must have been required at some point but no longer. It's no impact on us to just retire it. that's not a threat - just saying. |
|
Maybe I should provide some background here, wearing my hat as the current maintainer of version.pm, Perl's official abstraction to deal with this.
As far as Perl is concerned, version
I'm afraid we're the exception here. Sorry about that.
Indeed. In particular the general recommendation are:
|
|
For more details see my guide to how versions work in Perl: https://blogs.perl.org/users/grinnz/2018/04/a-guide-to-versions-in-perl.html There is a 0% chance you are going to convince enough CPAN authors to make a difference on that end. If it was possible, we wouldn't have needed this complex abstraction in CPAN's own processes. |
Well I don't see why compatible versions can't be enforced for new releases on CPAN side, but that's Perl community's affair. For outer world, like I've already mentioned, the problem is already quite small as only a tiny fraction of CPAN authors [of modules which are packaged at least somewhere] use incompatible versions. But of course it does make sense to enlighten these as well and make the problem yet smaller. What I don't understand is the opposition to the compatible versioning, which makes life easier for many people and goes hand-in-hand with the very idea of publishing code for everyones good. |
It feels to me like the point wasn't necessarily communicated clearly. It can read like "you should change your versioning to keep repology happy", which would indeed not be very convincing by itself. The real reason is "you should change it to prevent almost all distributions from having to do extra work". Arch and derived have worked around this incrementing the epoch. They would have to do it again the next time this happens. Both solutions work are functional, but do cause extra work for the various distributors. That should be a convincing argument. |
|
There's no opposition to the idea; you will see in my guide that I recommend suggestions like @Leont mentioned to avoid the issue. But ultimately it is up to each individual CPAN author to be aware that the problem even exists. |
|
To address this at the Perl-centric level would have to be done in PAUSE or MetaCPAN - I don't think it's reasonable to reject indexing in PAUSE when running afoul of external version progression, but an indexer warning would be perfectly reasonable and raise awareness. MetaCPAN has no authority as it's just a search engine but could display something for awareness. |
Projects affected
https://repology.org/project/perl:data-munge/versions
Observed behavior
The new version 0.10 is considered older or inferior to the 6-year version 0.097.
I believe 0.10 has been mistakenly sunk.
Expected behavior
verson 0.10 should be considered the latest.
It is numerically higher (0.10 > 0.097) and it's 6 years newer.
Proof
The text was updated successfully, but these errors were encountered: