handle cases where extensions don't exist. #852
Conversation
for example with windows desktop 6.8.1 win64_msvc2022_arm64_cross_compiled both qtwebengine and qtpdf don't exist.
|
Should |
|
you are correct, there are additional problems with list-qt.
with the tail of the debug log
finds both qtpdf and qtwebengine, but only qtpdf actual exists. |
|
If the availability of extensions by host/target sdk/VERSION/ARCHITECTURE is static then hardcoding a repo property as you suggest is easy and runs faster as we don't have to speculatively fetch urls. OTOH, if the availability changes from release to release we may want to figure it out dynamically as I attempted (except for a hardcoded list of possible extensions that is used by install-qt but not list-qt!) |
|
@tsteven4 Check the Qt LTS part: #846 (comment) From now on, they will decouple extensions versions from Qt versions. That's why the repo structure changed Also, I think that the check error you see is just a timeout from a bad mirror: I had the same issue with the same mirror on the same test here (https://mirrors.ustc.edu.cn)And doing a simple Note Maybe using the Qt online installer could be useful to get the extensions dynamically: |
|
from @Kidev's scrapper the currently available extensions are as folllow. 6.8.1 and 6.8.0 don't show any differences in availablity. qtwebengine: qtpdf: |
|
I believe the last two commits fix the issues with list-qt --modules and list-qt --long-modules. there are still issues with android that are beyond the scope of this work. |
|
if we are going to go this way it would be nice to de-duplicate at least between list-qt --modules and --long-modules. it would be even nicer to de-duplicate with install-qt and list-qt. |
| @@ -886,9 +899,19 @@ def to_module_arch(name: str) -> Tuple[Optional[str], Optional[str]]: | |||
| module, _arch = to_module_arch(name) | |||
| if _arch == arch: | |||
| modules.add(cast(str, module)) | |||
| if version >= Version("6.8.0"): | |||
| for ext in self.fetch_extensions(): | |||
There was a problem hiding this comment.
fetch_extensions is no longer used.
|
The approach is acceptable and good enough in the current situation. |
ok by me. |
|
my list-qt testing looks good. Both --modules and --long-modules are tested. This is a log of aqt arguments followed by two digits. The first digit indicates the qtpdf extension exists, the second indicates if the qtwebengine extensions exists. |
|
@tsteven4 You may have missed some android versions in all_os: https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtpdf/681/ |
|
@Kidev, I intentionally ignored android. android has issues beyond extensions. I also considered source/debug/examples functionality to be beyond the scope of this work as well. |
I agree on the idea, features are nice, but adding too much too few people use just makes the tool less user friendly. Maybe I'm wrong, but I think people use aqt to install Qt quickly without having to deal with the Qt's installer. But devs and so on use the Maintenance tool to get the sources or the SDK. However for android I'm split, it is present in users bug reports, and is typically one of the use case of aqt in CIs... I think the current code of aqt is missing something that would make the code much more easy to maintain. It needs a consistent data structure abstracting the download.qt.io data. So the CLI code would never need to change, only the data structure would need to handle the additions of new folders and patterns and make it fit a consistent model over which we have control. This data structure could be automatically built by Github Action workflow running each day for example, creating a data file that can be pulled whenever a user runs |
|
@tsteven4 I think you changed it in this PR, but in master, this in for ext in ["qtwebengine", "qtpdf"]:
extensions_target_folder = posixpath.join(
"online/qtsdkrepository", os_name, "extensions", ext, self._version_str(), arch
)
extensions_xml_url = posixpath.join(extensions_target_folder, "Updates.xml")
extensions_xml_text = self._download_update_xml(extensions_xml_url)
update_xmls.append(UpdateXmls(extensions_target_folder, extensions_xml_text))is very bad when there is no qtwebengine available. It was making downloading 6.8+ wasm impossible as it triggered a 'too much timeouts' even if the wasm package was found first: 2024-12-15 22:45:20,543 - aqt.main - WARNING - installer 129600515533696 You are installing the Qt6-WASM version of Qt, which requires that the desktop version of Qt is also installed. You can install it with the following command:
`aqt install-qt linux desktop 6.8.1 linux_gcc_64`
2024-12-15 22:45:20,543 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/wasm/qt6_681/qt6_681_wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:20,935 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtwebengine/681/wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:21,205 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtwebengine/681/wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:21,409 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtwebengine/681/wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:21,601 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtwebengine/681/wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:21,795 - aqt.helper - DEBUG - helper 129600515533696 Attempt to download checksum at https://download.qt.io/online/qtsdkrepository/all_os/extensions/qtwebengine/681/wasm_multithread/Updates.xml.sha256
2024-12-15 22:45:21,988 - aqt.archives - WARNING - archives 129600515533696 Failed to download checksum for the file 'Updates.xml'. This may happen on unofficial mirrors.
2024-12-15 22:45:22,196 - aqt.main - ERROR - installer 129600515533696 Failed to locate XML data for Qt version '6.8.1'.For my PR I simply did: if self.target != 'wasm':
for ext in ["qtwebengine", "qtpdf"]:
extensions_target_folder = posixpath.join(
"online/qtsdkrepository", os_name, "extensions", ext, self._version_str(), arch
)
extensions_xml_url = posixpath.join(extensions_target_folder, "Updates.xml")
extensions_xml_text = self._download_update_xml(extensions_xml_url)
update_xmls.append(UpdateXmls(extensions_target_folder, extensions_xml_text))But we'll need to see what we do to merge properly |
Ok that's what I thought, I wanted to be sure as it was the last thing preventing me from updating my PR. I'll pull your changes into my PR as soon as they are merged, should be straightforward to remove the conflict as I don't change anything in this function otherwise |
for example with windows desktop 6.8.1 win64_msvc2022_arm64_cross_compiled both qtwebengine and qtpdf don't exist.
this resolves #851