containers-run command not recognized in venv installation on mac

[jsheunis]

I'm working on a mac with Python 3.9.1 base installation.

I created a fresh virtual environment using venvwith datalad-core and datalad-container installed:

> python -m venv ~/my_env
> source ~/my_env/bin/activate
> python -m pip install datalad
...
> python -m pip install datalad-container

Requirement already satisfied: datalad-container in /Users/jsheunis/my_env/lib/python3.9/site-packages (1.2.3)
Requirement already satisfied: requests>=1.2 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad-container) (2.31.0)
Requirement already satisfied: datalad>=0.18.0 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad-container) (0.19.3)
Requirement already satisfied: certifi>=2017.4.17 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (2023.7.22)
Requirement already satisfied: idna<4,>=2.5 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (2.0.6)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (3.3.0)
Requirement already satisfied: looseversion in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.3.0)
Requirement already satisfied: importlib-metadata>=3.6; python_version < "3.10" in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (6.8.0)
Requirement already satisfied: typing-extensions>=4.0.0; python_version < "3.11" in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.8.0)
Requirement already satisfied: packaging in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (23.2)
Requirement already satisfied: boto in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (2.49.0)
Requirement already satisfied: keyrings.alt in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (5.0.0)
Requirement already satisfied: platformdirs in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (3.11.0)
Requirement already satisfied: python-gitlab in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (3.15.0)
Requirement already satisfied: msgpack in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.0.7)
Requirement already satisfied: chardet>=3.0.4 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (5.2.0)
Requirement already satisfied: iso8601 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (2.1.0)
Requirement already satisfied: tqdm>=4.32.0 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.66.1)
Requirement already satisfied: humanize in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.8.0)
Requirement already satisfied: fasteners>=0.14 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (0.19)
Requirement already satisfied: distro; python_version >= "3.8" in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.8.0)
Requirement already satisfied: annexremote in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.6.0)
Requirement already satisfied: keyring!=23.9.0,>=20.0 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (24.2.0)
Requirement already satisfied: patool>=1.7 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.12)
Requirement already satisfied: zipp>=0.5 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from importlib-metadata>=3.6; python_version < "3.10"->datalad>=0.18.0->datalad-container) (3.17.0)
Requirement already satisfied: jaraco.classes in /Users/jsheunis/my_env/lib/python3.9/site-packages (from keyrings.alt->datalad>=0.18.0->datalad-container) (3.3.0)
Requirement already satisfied: requests-toolbelt>=0.10.1 in /Users/jsheunis/my_env/lib/python3.9/site-packages (from python-gitlab->datalad>=0.18.0->datalad-container) (1.0.0)
Requirement already satisfied: more-itertools in /Users/jsheunis/my_env/lib/python3.9/site-packages (from jaraco.classes->keyrings.alt->datalad>=0.18.0->datalad-container) (10.1.0)
WARNING: You are using pip version 20.2.3; however, version 23.2.1 is available.
You should consider upgrading via the '/Users/jsheunis/my_env/bin/python -m pip install --upgrade pip' command.

Subsequently containers-run is not recognized:

> datalad containers-run -n docker-make main.pdf

datalad: Unknown command 'containers-run'.  See 'datalad --help'.
Hint: Command containers-run is provided by (not installed) extension datalad-container.

and datalad wtf output does not show datalad-container as an extension.

datalad wtf

> datalad wtf

# WTF
## configuration <SENSITIVE, report disabled by configuration>
## credentials
  - keyring:
    - active_backends:
      - macOS Keyring
      - PlaintextKeyring with no encyption v.1.0 at /Users/jsheunis/.local/share/python_keyring/keyring_pass.cfg
    - config_file: /Users/jsheunis/.config/python_keyring/keyringrc.cfg
    - data_root: /Users/jsheunis/.local/share/python_keyring
## datalad
  - version: 0.18.3
## dataset
  - branches:
    - docker@57d2565
    - git-annex@e1473ab
    - master@69ca2ad
  - id: c5a79271-7d24-42aa-a0cf-38d84fd15eaa
  - path: /Users/jsheunis/Documents/psyinf/Data/miho/paper-remodnav
  - repo: AnnexRepo
## dependencies
  - annexremote: 1.5.0
  - boto: 2.49.0
  - cmd:7z: 17.04
  - cmd:annex: 10.20230227
  - cmd:bundled-git: UNKNOWN
  - cmd:git: 2.42.0
  - cmd:ssh: 8.1p1
  - cmd:system-git: 2.42.0
  - cmd:system-ssh: 8.1p1
  - humanize: 3.12.0
  - iso8601: 0.1.16
  - keyring: 23.2.1
  - keyrings.alt: 4.1.0
  - msgpack: 1.0.2
  - platformdirs: 3.5.1
  - requests: 2.25.0
## environment
  - LANG: en_US.UTF-8
  - LANGUAGE: en_US.UTF-8
  - LC_ALL: en_US.UTF-8
  - LC_CTYPE: en_US.UTF-8
  - LC_TERMINAL: iTerm2
  - LC_TERMINAL_VERSION: 3.4.5
  - PATH: /Users/jsheunis/my_env/bin:/Users/jsheunis/.rbenv/shims:/Users/jsheunis/opt/miniconda3/bin:/Users/jsheunis/opt/miniconda3/condabin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Apple/usr/bin:/Users/jsheunis/.dotfiles/bin:/usr/local/sbin:/usr/X11R6/bin:/usr/local/games:/usr/games
## extensions
  - deprecated:
    - description: Deprecated functionality
    - entrypoints:
      - datalad_deprecated.annotate_paths.AnnotatePaths:
        - class: AnnotatePaths
        - load_error: None
        - module: datalad_deprecated.annotate_paths
        - names:
      - datalad_deprecated.ls.Ls:
        - class: Ls
        - load_error: None
        - module: datalad_deprecated.ls
        - names:
      - datalad_deprecated.metadata.aggregate.AggregateMetaData:
        - class: AggregateMetaData
        - load_error: None
        - module: datalad_deprecated.metadata.aggregate
        - names:
          - aggregate-metadata
          - aggregate_metadata
      - datalad_deprecated.metadata.extract_metadata.ExtractMetadata:
        - class: ExtractMetadata
        - load_error: None
        - module: datalad_deprecated.metadata.extract_metadata
        - names:
          - extract-metadata
          - extract_metadata
      - datalad_deprecated.metadata.metadata.Metadata:
        - class: Metadata
        - load_error: None
        - module: datalad_deprecated.metadata.metadata
        - names:
      - datalad_deprecated.metadata.search.Search:
        - class: Search
        - load_error: None
        - module: datalad_deprecated.metadata.search
        - names:
      - datalad_deprecated.publish.Publish:
        - class: Publish
        - load_error: None
        - module: datalad_deprecated.publish
        - names:
    - load_error: None
    - module: datalad_deprecated
    - version: 0.3.0
  - metalad:
    - description: DataLad semantic metadata command suite
    - entrypoints:
      - datalad_metalad.add.Add:
        - class: Add
        - load_error: None
        - module: datalad_metalad.add
        - names:
          - meta-add
          - meta_add
      - datalad_metalad.aggregate.Aggregate:
        - class: Aggregate
        - load_error: None
        - module: datalad_metalad.aggregate
        - names:
          - meta-aggregate
          - meta_aggregate
      - datalad_metalad.conduct.Conduct:
        - class: Conduct
        - load_error: None
        - module: datalad_metalad.conduct
        - names:
          - meta-conduct
          - meta_conduct
      - datalad_metalad.dump.Dump:
        - class: Dump
        - load_error: None
        - module: datalad_metalad.dump
        - names:
          - meta-dump
          - meta_dump
      - datalad_metalad.extract.Extract:
        - class: Extract
        - load_error: None
        - module: datalad_metalad.extract
        - names:
          - meta-extract
          - meta_extract
      - datalad_metalad.filter.Filter:
        - class: Filter
        - load_error: None
        - module: datalad_metalad.filter
        - names:
          - meta-filter
          - meta_filter
    - load_error: None
    - module: datalad_metalad
    - version: 0.4.17
  - neuroimaging:
    - description: Neuroimaging tools
    - entrypoints:
      - datalad_neuroimaging.bids2scidata.BIDS2Scidata:
        - class: BIDS2Scidata
        - load_error: None
        - module: datalad_neuroimaging.bids2scidata
        - names:
          - bids2scidata
    - load_error: None
    - module: datalad_neuroimaging
    - version: 0.3.3+10.g61a7279
## git-annex
  - build flags:
    - Assistant
    - Webapp
    - Pairing
    - FsEvents
    - TorrentParser
    - MagicMime
    - Benchmark
    - Feeds
    - Testsuite
    - S3
    - WebDAV
  - dependency versions:
    - aws-0.24
    - bloomfilter-2.0.1.0
    - cryptonite-0.30
    - DAV-1.3.4
    - feed-1.3.2.1
    - ghc-9.4.4
    - http-client-0.7.13.1
    - persistent-sqlite-2.13.1.1
    - torrent-10000.1.1
    - uuid-1.3.15
    - yesod-1.6.2.1
  - key/value backends:
    - SHA256E
    - SHA256
    - SHA512E
    - SHA512
    - SHA224E
    - SHA224
    - SHA384E
    - SHA384
    - SHA3_256E
    - SHA3_256
    - SHA3_512E
    - SHA3_512
    - SHA3_224E
    - SHA3_224
    - SHA3_384E
    - SHA3_384
    - SKEIN256E
    - SKEIN256
    - SKEIN512E
    - SKEIN512
    - BLAKE2B256E
    - BLAKE2B256
    - BLAKE2B512E
    - BLAKE2B512
    - BLAKE2B160E
    - BLAKE2B160
    - BLAKE2B224E
    - BLAKE2B224
    - BLAKE2B384E
    - BLAKE2B384
    - BLAKE2BP512E
    - BLAKE2BP512
    - BLAKE2S256E
    - BLAKE2S256
    - BLAKE2S160E
    - BLAKE2S160
    - BLAKE2S224E
    - BLAKE2S224
    - BLAKE2SP256E
    - BLAKE2SP256
    - BLAKE2SP224E
    - BLAKE2SP224
    - SHA1E
    - SHA1
    - MD5E
    - MD5
    - WORM
    - URL
    - X*
  - local repository version: 10
  - operating system: darwin x86_64
  - remote types:
    - git
    - gcrypt
    - p2p
    - S3
    - bup
    - directory
    - rsync
    - web
    - bittorrent
    - webdav
    - adb
    - tahoe
    - glacier
    - ddar
    - git-lfs
    - httpalso
    - borg
    - hook
    - external
  - supported repository versions:
    - 8
    - 9
    - 10
  - upgrade supported from repository versions:
    - 0
    - 1
    - 2
    - 3
    - 4
    - 5
    - 6
    - 7
    - 8
    - 9
    - 10
  - version: 10.20230227
## location
  - path: /Users/jsheunis/Documents/psyinf/Data/miho/paper-remodnav
  - type: dataset
## python
  - implementation: CPython
  - version: 3.9.1
## system
  - distribution: darwin/20.6.0 10.16/x86_64
  - encoding:
    - default: utf-8
    - filesystem: utf-8
    - locale.prefered: UTF-8
  - filesystem:
    - CWD:
      - path: /Users/jsheunis/Documents/psyinf/Data/miho/paper-remodnav
    - HOME:
      - path: /Users/jsheunis
    - TMP:
      - path: /var/folders/g7/720mm8ns7hg3d28_3yyzkx_r0000gp/T
  - max_path_length: 313
  - name: Darwin
  - release: 20.6.0
  - type: posix
  - version: Darwin Kernel Version 20.6.0: Thu Mar  9 20:39:26 PST 2023; root:xnu-7195.141.49.700.6~1/RELEASE_X86_64

However, a datalad rerun command was run prior to the containers-run execution, and this process internally imported modules from datalad-container without problem.

I then created a virtual environment with miniconda:

> conda create -n remod python=3.9
> conda activate remod
> python -m pip install datalad
> python -m pip install datalad-container

I then ran the datalad containers-run -n docker-make main.pdf command and everything worked nicely. I.e. the containers-run command is recognized inside the conda environment.

[yarikoptic]

I am afraid that not entire mystery revealed since in the top you have

> python -m pip install datalad-container

Requirement already satisfied: datalad-container in /Users/jsheunis/my_env/lib/python3.9/site-packages (1.2.3)

so -- datalad-container was already installed somehow, although you have not posted output from the pip install datalad above... may be that ~/my_env existed already before you started and installation was somehow 'crippled', and then subsequent commands didn't resolve it. Try again from scratch ensuring that folder doesn't exist before, and post full output.

[jsheunis]

Sorry for delaying this:

> python -m venv ~/my_totally_unique_env_yeah
> source ~/my_totally_unique_env_yeah/bin/activate
> python -m pip install datalad

Collecting datalad
  Using cached datalad-0.19.3-py3-none-any.whl (1.3 MB)
Collecting iso8601
  Using cached iso8601-2.1.0-py3-none-any.whl (7.5 kB)
Collecting annexremote
  Downloading annexremote-1.6.4-py3-none-any.whl (25 kB)
Collecting tqdm>=4.32.0
  Using cached tqdm-4.66.1-py3-none-any.whl (78 kB)
Collecting distro; python_version >= "3.8"
  Using cached distro-1.8.0-py3-none-any.whl (20 kB)
Collecting requests>=1.2
  Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Collecting looseversion
  Using cached looseversion-1.3.0-py2.py3-none-any.whl (8.2 kB)
Collecting packaging
  Using cached packaging-23.2-py3-none-any.whl (53 kB)
Collecting python-gitlab
  Downloading python_gitlab-4.1.1-py3-none-any.whl (137 kB)
     |████████████████████████████████| 137 kB 4.2 MB/s
Collecting fasteners>=0.14
  Using cached fasteners-0.19-py3-none-any.whl (18 kB)
Collecting boto
  Using cached boto-2.49.0-py2.py3-none-any.whl (1.4 MB)
Collecting keyring!=23.9.0,>=20.0
  Using cached keyring-24.2.0-py3-none-any.whl (37 kB)
Collecting platformdirs
  Using cached platformdirs-3.11.0-py3-none-any.whl (17 kB)
Collecting keyrings.alt
  Using cached keyrings.alt-5.0.0-py3-none-any.whl (18 kB)
Collecting patool>=1.7
  Downloading patool-1.15.0-py2.py3-none-any.whl (91 kB)
     |████████████████████████████████| 91 kB 2.1 MB/s
Collecting typing-extensions>=4.0.0; python_version < "3.11"
  Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Collecting humanize
  Using cached humanize-4.8.0-py3-none-any.whl (117 kB)
Collecting chardet>=3.0.4
  Using cached chardet-5.2.0-py3-none-any.whl (199 kB)
Collecting importlib-metadata>=3.6; python_version < "3.10"
  Using cached importlib_metadata-6.8.0-py3-none-any.whl (22 kB)
Collecting msgpack
  Using cached msgpack-1.0.7-cp39-cp39-macosx_10_9_x86_64.whl (234 kB)
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.0.7-py3-none-any.whl (124 kB)
     |████████████████████████████████| 124 kB 8.9 MB/s
Collecting charset-normalizer<4,>=2
  Downloading charset_normalizer-3.3.2-cp39-cp39-macosx_10_9_x86_64.whl (122 kB)
     |████████████████████████████████| 122 kB 8.2 MB/s
Collecting idna<4,>=2.5
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Collecting requests-toolbelt>=0.10.1
  Using cached requests_toolbelt-1.0.0-py2.py3-none-any.whl (54 kB)
Collecting jaraco.classes
  Using cached jaraco.classes-3.3.0-py3-none-any.whl (5.9 kB)
Collecting zipp>=0.5
  Using cached zipp-3.17.0-py3-none-any.whl (7.4 kB)
Collecting more-itertools
  Using cached more_itertools-10.1.0-py3-none-any.whl (55 kB)
Installing collected packages: iso8601, annexremote, tqdm, distro, urllib3, charset-normalizer, idna, certifi, requests, looseversion, packaging, requests-toolbelt, python-gitlab, fasteners, boto, zipp, importlib-metadata, more-itertools, jaraco.classes, keyring, platformdirs, keyrings.alt, patool, typing-extensions, humanize, chardet, msgpack, datalad
Successfully installed annexremote-1.6.4 boto-2.49.0 certifi-2023.7.22 chardet-5.2.0 charset-normalizer-3.3.2 datalad-0.19.3 distro-1.8.0 fasteners-0.19 humanize-4.8.0 idna-3.4 importlib-metadata-6.8.0 iso8601-2.1.0 jaraco.classes-3.3.0 keyring-24.2.0 keyrings.alt-5.0.0 looseversion-1.3.0 more-itertools-10.1.0 msgpack-1.0.7 packaging-23.2 patool-1.15.0 platformdirs-3.11.0 python-gitlab-4.1.1 requests-2.31.0 requests-toolbelt-1.0.0 tqdm-4.66.1 typing-extensions-4.8.0 urllib3-2.0.7 zipp-3.17.0
WARNING: You are using pip version 20.2.3; however, version 23.3.1 is available.
You should consider upgrading via the '/Users/jsheunis/my_totally_unique_env_yeah/bin/python -m pip install --upgrade pip' command.

> python -m pip install datalad-container

Collecting datalad-container
  Using cached datalad_container-1.2.3-py3-none-any.whl (38 kB)
Requirement already satisfied: datalad>=0.18.0 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad-container) (0.19.3)
Requirement already satisfied: requests>=1.2 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad-container) (2.31.0)
Requirement already satisfied: annexremote in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.6.4)
Requirement already satisfied: chardet>=3.0.4 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (5.2.0)
Requirement already satisfied: humanize in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.8.0)
Requirement already satisfied: packaging in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (23.2)
Requirement already satisfied: tqdm>=4.32.0 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.66.1)
Requirement already satisfied: python-gitlab in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.1.1)
Requirement already satisfied: typing-extensions>=4.0.0; python_version < "3.11" in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (4.8.0)
Requirement already satisfied: fasteners>=0.14 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (0.19)
Requirement already satisfied: distro; python_version >= "3.8" in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.8.0)
Requirement already satisfied: looseversion in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.3.0)
Requirement already satisfied: msgpack in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.0.7)
Requirement already satisfied: patool>=1.7 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (1.15.0)
Requirement already satisfied: platformdirs in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (3.11.0)
Requirement already satisfied: keyring!=23.9.0,>=20.0 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (24.2.0)
Requirement already satisfied: keyrings.alt in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (5.0.0)
Requirement already satisfied: boto in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (2.49.0)
Requirement already satisfied: iso8601 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (2.1.0)
Requirement already satisfied: importlib-metadata>=3.6; python_version < "3.10" in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from datalad>=0.18.0->datalad-container) (6.8.0)
Requirement already satisfied: idna<4,>=2.5 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (2023.7.22)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from requests>=1.2->datalad-container) (3.3.2)
Requirement already satisfied: requests-toolbelt>=0.10.1 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from python-gitlab->datalad>=0.18.0->datalad-container) (1.0.0)
Requirement already satisfied: jaraco.classes in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from keyring!=23.9.0,>=20.0->datalad>=0.18.0->datalad-container) (3.3.0)
Requirement already satisfied: zipp>=0.5 in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from importlib-metadata>=3.6; python_version < "3.10"->datalad>=0.18.0->datalad-container) (3.17.0)
Requirement already satisfied: more-itertools in /Users/jsheunis/my_totally_unique_env_yeah/lib/python3.9/site-packages (from jaraco.classes->keyring!=23.9.0,>=20.0->datalad>=0.18.0->datalad-container) (10.1.0)
Installing collected packages: datalad-container
Successfully installed datalad-container-1.2.3
WARNING: You are using pip version 20.2.3; however, version 23.3.1 is available.
You should consider upgrading via the '/Users/jsheunis/my_totally_unique_env_yeah/bin/python -m pip install --upgrade pip' command.

> datalad containers-run

datalad: Unknown command 'containers-run'.  See 'datalad --help'.

Hint: Command containers-run is provided by (not installed) extension datalad-container.

[yarikoptic]

first step of troubleshooting is which datalad to ensure that it is the one from the venv and not some globally installed one. In my case, sometimes zsh manages to use some cached position instead of a "Fresh" one

[jsheunis]

> which datalad
/Users/jsheunis/my_totally_unique_env_yeah/bin/datalad

[yarikoptic]

python -c 'import datalad_containers'; datalad wtf -S extensions might be of help

[jsheunis]

> python -c 'import datalad_containers'; datalad wtf -S extensions

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'datalad_containers'
# WTF
## extensions
  - container:
    - description: Containerized environments
    - entrypoints:
      - datalad_container.containers_add.ContainersAdd:
        - class: ContainersAdd
        - module: datalad_container.containers_add
        - names:
          - containers-add
          - containers_add
      - datalad_container.containers_list.ContainersList:
        - class: ContainersList
        - module: datalad_container.containers_list
        - names:
          - containers-list
          - containers_list
      - datalad_container.containers_remove.ContainersRemove:
        - class: ContainersRemove
        - module: datalad_container.containers_remove
        - names:
          - containers-remove
          - containers_remove
      - datalad_container.containers_run.ContainersRun:
        - class: ContainersRun
        - module: datalad_container.containers_run
        - names:
          - containers-run
          - containers_run
    - module: datalad_container
    - version: 1.2.3

[yarikoptic]

sorry -- damn naming discrepancy - was looking for datalad_container module. so -- presumably those commands should be available ;) I guess someone should debug that location where whatever in datalad discovers/uses them all... here is what I see

❯ datalad -l 1 containers-run
[DEBUG  ] Command line args 1st pass for DataLad 0.19.2. Parsed: Namespace() Unparsed: ['containers-run'] 
[DEBUG  ] Processing entrypoints 
[DEBUG  ] Loading entrypoint deprecated from datalad.extensions 
[DEBUG  ] Loaded entrypoint deprecated from datalad.extensions 
[DEBUG  ] Loading entrypoint container from datalad.extensions 
[DEBUG  ] Loaded entrypoint container from datalad.extensions 
[DEBUG  ] Loading entrypoint metalad from datalad.extensions 
[DEBUG  ] Loaded entrypoint metalad from datalad.extensions 
[DEBUG  ] Done processing entrypoints 
[Level 5] Importing module datalad_container.containers_run  
...

do you see loading entry point loading for container?

[jsheunis]

lol i didn't even spot the typo. If I fix the typo, the module is imported without error.

OK, now this is weird:

> datalad containers-run

[ERROR  ] No known containers. Use containers-add

I don't know if it could be because I deactivated and reactivated the environment?

[yarikoptic]

who knows... may be your shell at first didn't realize that new datalad is available, and that which made it known? do you have another datalad available? (which -a datalad should give all in your PATH)

[jsheunis]

Indeed it looks like it:

> which -a datalad

/Users/jsheunis/my_totally_unique_env_yeah/bin/datalad
/Users/jsheunis/opt/miniconda3/bin/datalad

so there's some base-level conda installation of datalad?

If I open another shell session and activate a conda environment that has datalad installed (via pip), and then run the same command, I only see the activated environment's datalad installation:

> conda activate ct
> which -a datalad

/Users/jsheunis/opt/miniconda3/envs/ct/bin/datalad

So I guess the venv environment sees the base conda installation of datalad (first), but a conda environment sees only its only installation?

[yarikoptic]

so there's some base-level conda installation of datalad?

only you would know if you did install datalad before in the base environment there ;)

So I guess the venv environment sees the base conda installation of datalad (first), but a conda environment sees only its only installation?

no. I think it was just a fluke of the shell that it likely used cached path to base installation . Calling hash datalad should force it to "reconsider". Altogether I do not think there is anything to fix really, in particular in datalad-container, so I will close.