Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accept ddeb from remote repositories when using "update" #19

Open
wants to merge 70 commits into
base: multiple-versions-debian
Choose a base branch
from

Conversation

Hugal31
Copy link

@Hugal31 Hugal31 commented May 24, 2021

I got an assert error when trying to mirror a repo with ddeb (assert ("Unexpected package type" == NULL);), because we do not handle ddeb on update. Correct that.

bdrung added 30 commits April 3, 2019 12:02
Switch to dh to easily rebuild autotools files.
To be able to build packages for the tests, use the host architecture
(to avoid requiring a cross-compiler).
Using the non-existing priority 'superfluous' causes warning messages.
To test the handling of upstream tarballs, switch from source format 1.0
to either 3.0 (quilt) or 3.0 (native).
Use the common naming schema $source_$version_$arch.changes for the name
of the .changes files for testing.
Closes: #857302
Signed-off-by: Benjamin Drung <[email protected]>
Compiling reprepro produces this warning:

```
signature.c: In function ‘signature_getpassphrase’:
signature.c:63:2: warning: ignoring return value of ‘write’, declared with attribute warn_unused_result [-Wunused-result]
  write(fd, p, strlen(p));
  ^~~~~~~~~~~~~~~~~~~~~~~
signature.c:64:2: warning: ignoring return value of ‘write’, declared with attribute warn_unused_result [-Wunused-result]
  write(fd, "\n", 1);
  ^~~~~~~~~~~~~~~~~~
```
The error message in guess_component misses a leading quotation mark,
for example:

Could not find 'main' in components of 'bionic': contrib'
There are multiple places where new cursors are generated. Remove
duplicate code by introducing the newcursor() function.
cursor_nexttempdata has an additional len_p output parameter (compared
to cursor_nexttemp). Make the len_p output parameter optional and
replace cursor_nexttemp by cursor_nexttempdata.

Thus cursor_nexttemp(...) becomes cursor_nexttempdata(..., NULL).
The cursor_nextpair() function has a parse_pair() function for
evaluating the returned database output. Introduce a similar
parse_data() function for the cursor_nexttempdata() function.
Move cursor struct upwards to have the struct definition in one block.
Rename table_newduplicatecursor to table_newduplicatepairedcursor to
make use this name for a data cursor.
The functions cursor_nexttempdata() and cursor_nextpair() share a
similar logic. Thus combine the duplicate code in cursor_next().

cursor_nexttempdata() set always DB_NEXT as cursor flag instead of using
the cursor->flags value. All users of cursor_nexttempdata() call
table_newglobalcursor() [1] beforehand which sets cursor->flags = DB_NEXT.

[1] Capsulated dependencies: cursor_nexttempdata -> package_next ->
package_openiterator -> table_newglobalcursor
The word 'todo' is used for marking todo items for the programmer. Thus
use 'remaining' instead of 'todo' as variable name.
If no package version is specfied, use the latest version.
The multi version support will require splitting the name and version in
multiple places. Thus moved the code in a splitnameandversion()
function.
Use variable 'result' only for the final returned result.
bdrung and others added 26 commits April 3, 2019 12:02
The DB_NODUPDATA flag may only be specified if the underlying database
has been configured to support sorted duplicates. Thus do not set the
DB_NODUPDATA flag when the database does not support duplicates. To
avoid querying the flags on each call, save the flags when opening the
table.
DB_DUPSORT allows duplicate keys in the database, but not duplicate
key/value pairs. Only if the duplicate data items are unsorted,
applications may store identical duplicate data items.

Since the references.db stores a the filekey mapping to the
codename|component|architecture triplet, there might be identical
duplicates, when upstream tarballs are references by multiple version.

Therefore switch references.db from DB_DUPSORT to DB_DUP.

closes bug ionos-cloud#9
Closes: #570623
Signed-off-by: Benjamin Drung <[email protected]>
Limit the number of versions of a package per distribution,
architecture, component, and type. The limit must be a number. If the
number is positive, all old package version that exceed these limit will
be removed when a new package version is added. If the number is zero or
negative, all package version will be kept. By default only one package
version will be kept.
When opening multiple databases in parallel (needed for the move command
or the archive option), the databases needs to be configured with
locking. Thus an database environment is needed. Open and close the
database environment when getting/releasing the database lock.

Without proper locking, the database might become corrupt:

BDB0689 db/packages.db page 6387 is on free list with type 5
BDB0061 PANIC: Invalid argument
Internal error of the underlying BerkeleyDB database:
Within packages.db subtable jessie-proposed|main|amd64 at put(uniq):
BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
BDB0060 PANIC: fatal region error detected; run recovery
db_close(packages.db, jessie-proposed|main|amd64): BDB0087
DB_RUNRECOVERY: Fatal error, run database recovery
There have been errors!
The move command will need to open two tables at the same time (the
source table and the destination table). Thus keep track of all opened
tables.
The move commands needs access to the from target to remove the packages
after adding them to the destination target.
Add the commands move, movesrc, movematched, movefilter.
Add test cases for the move, movesrc, movematched, movefilter commands.

Signed-off-by: Benjamin Drung <[email protected]>
Closes: #730572
LP: #799889
Signed-off-by: Benjamin Drung <[email protected]>
@bdrung
Copy link
Contributor

bdrung commented Jun 3, 2021

Can you add a test case for that?

@Hugal31
Copy link
Author

Hugal31 commented Jun 4, 2021

I am not very familiar with your test framework. Can you tell me:

  • How can I install "testtool"
  • Where are the tests for the uPackage generation on update, for example ?

@bdrung
Copy link
Contributor

bdrung commented Jun 4, 2021

Since I couldn't find testtool (see Debian bug #857302), I wrote some own tests using shunit2 (shell code). The tests live in tests/basic.sh and tests/multiversion.sh.

@bdrung bdrung force-pushed the multiple-versions-debian branch from 630bb1e to 3afde91 Compare July 20, 2021 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants