Lost kernel connection after cell error, kernel will not restart

[ckchow]

Applies To

• Notebooks (.ipynb files)
• Interactive Window and/or Cell Scripts (.py files with #%% markers)

What happened?

Running a notebook similar to (paraphrase)

%load_ext autoreload
%autoreload 2

mdl = MyWeirdCatboostModelWrapper()

mdl.cross_validate()

Catboost threw an error, I tried to fix the error in my library code, and when trying to rerun the cells, the kernel seems to become unresponsive. I hit Restart, and the kernel appears to restart but fails with an error message.

VS Code Version

Version: 1.96.2 (Universal) Commit: fabdb6a30b49f79a7aba0f2ad9df9b399473380f Date: 2024-12-19T10:22:47.216Z (3 wks ago) Electron: 32.2.6 ElectronBuildId: 10629634 Chromium: 128.0.6613.186 Node.js: 20.18.1 V8: 12.8.374.38-electron.0 OS: Darwin arm64 24.1.0

Jupyter Extension Version

2024.11.0

Jupyter logs

Visual Studio Code (1.96.2, undefined, desktop)
Jupyter Extension Version: 2024.11.0.
Python Extension Version: 2024.22.2.
Pylance Extension Version: 2024.12.1.
Platform: darwin (arm64).
Temp Storage folder ~/Library/Application Support/Code/User/globalStorage/ms-toolsai.jupyter/version-2024.11.0
Workspace folder ~/projects/abfm_apnea, Home = /Users/cchow
15:05:10.069 [info] Starting Kernel (Python Path: ~/projects/abfm_apnea/.pixi/envs/default/bin/python, VirtualEnv, 3.11.10) for '~/projects/abfm_apnea/notebooks/02 regressor.ipynb' (disableUI=true)
15:05:10.070 [info] Starting Kernel (Python Path: ~/projects/abfm_apnea/.pixi/envs/default/bin/python, VirtualEnv, 3.11.10) for '~/projects/abfm_apnea/notebooks/10 catboost.ipynb' (disableUI=true)
15:05:11.868 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
15:05:11.869 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -m ipykernel_launcher --f=/Users/~/Library/Jupyter/runtime/kernel-v31f430dd8cf5fcecb24be660929aa3d8d5b061792.json
    > cwd: ~/projects/abfm_apnea
15:05:11.913 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -m pip list
15:05:11.957 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -c "import pip;print('6af208d0-cb9c-427f-b937-ff563e17efdf')"
15:05:12.023 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
15:05:12.024 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -m ipykernel_launcher --f=/Users/~/Library/Jupyter/runtime/kernel-v36cb090ff7692a0de026ffe11dfacb1fd60ae0e20.json
    > cwd: ~/projects/abfm_apnea
15:05:13.841 [info] Kernel successfully started
15:05:13.843 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python /Users/~/.vscode/extensions/ms-toolsai.jupyter-2024.11.0-darwin-arm64/pythonFiles/printJupyterDataDir.py
15:05:13.913 [info] Kernel successfully started
15:05:31.292 [warn] Cell completed with errors [iu [Error]: catboost/private/libs/options/plain_options_helper.cpp:512: Unknown option {use_medicine_embeddings} with value "false"
	at n.execute (/Users/~/.vscode/extensions/ms-toolsai.jupyter-2024.11.0-darwin-arm64/dist/extension.node.js:297:4958)] {
  ename: 'CatBoostError',
  evalue: 'catboost/private/libs/options/plain_options_helper.cpp:512: Unknown option {use_medicine_embeddings} with value "false"',
  traceback: [
    '\x1B[0;31m---------------------------------------------------------------------------\x1B[0m',
    '\x1B[0;31mCatBoostError\x1B[0m                             Traceback (most recent call last)',
    'Cell \x1B[0;32mIn[4], line 1\x1B[0m\n' +
      '\x1B[0;32m----> 1\x1B[0m \x1B[43mmdl\x1B[49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43mcross_validate\x1B[49m\x1B[43m(\x1B[49m\x1B[43mn_folds\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[38;5;241;43m5\x1B[39;49m\x1B[43m,\x1B[49m\x1B[43m \x1B[49m\x1B[43mfull\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[43mfull\x1B[49m\x1B[43m)\x1B[49m\n',
    'File \x1B[0;32m~/projects/abfm_apnea/abfm_apnea/models/catboost_v1.py:106\x1B[0m, in \x1B[0;36mCatboostClassifierPipeline.cross_validate\x1B[0;34m(self, n_folds, full)\x1B[0m\n' +
      '\x1B[1;32m     97\x1B[0m     pandas_data[\x1B[38;5;124m"\x1B[39m\x1B[38;5;124mmedicines_embedding\x1B[39m\x1B[38;5;124m"\x1B[39m] \x1B[38;5;241m=\x1B[39m \x1B[38;5;28mlist\x1B[39m(embeddings)\n' +
      '\x1B[1;32m     99\x1B[0m pool \x1B[38;5;241m=\x1B[39m Pool(\n' +
      '\x1B[1;32m    100\x1B[0m     data\x1B[38;5;241m=\x1B[39mpandas_data,\n' +
      '\x1B[1;32m    101\x1B[0m     label\x1B[38;5;241m=\x1B[39mfull\x1B[38;5;241m.\x1B[39mselect(\x1B[38;5;124m"\x1B[39m\x1B[38;5;124mahi_severity_binary\x1B[39m\x1B[38;5;124m"\x1B[39m)\x1B[38;5;241m.\x1B[39mto_pandas(),\n' +
      '\x1B[1;32m    102\x1B[0m     cat_features\x1B[38;5;241m=\x1B[39mcategorical_columns,\n' +
      '\x1B[1;32m    103\x1B[0m     embedding_features\x1B[38;5;241m=\x1B[39m[\x1B[38;5;124m"\x1B[39m\x1B[38;5;124mmedicines_embedding\x1B[39m\x1B[38;5;124m"\x1B[39m] \x1B[38;5;28;01mif\x1B[39;00m \x1B[38;5;28mself\x1B[39m\x1B[38;5;241m.\x1B[39mparameters\x1B[38;5;241m.\x1B[39muse_medicine_embeddings \x1B[38;5;28;01melse\x1B[39;00m \x1B[38;5;28;01mNone\x1B[39;00m\n' +
      '\x1B[1;32m    104\x1B[0m )\n' +
      '\x1B[0;32m--> 106\x1B[0m \x1B[38;5;28;01mreturn\x1B[39;00m \x1B[43mcatboost\x1B[49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43mcv\x1B[49m\x1B[43m(\x1B[49m\x1B[43mpool\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[43mpool\x1B[49m\x1B[43m,\x1B[49m\x1B[43m \x1B[49m\x1B[43mfold_count\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[43mn_folds\x1B[49m\x1B[43m,\x1B[49m\x1B[43m \x1B[49m\x1B[43miterations\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[38;5;28;43mself\x1B[39;49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43miterations\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m    107\x1B[0m \x1B[43m            \x1B[49m\x1B[43mparams\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[43mdataclasses\x1B[49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43masdict\x1B[49m\x1B[43m(\x1B[49m\x1B[38;5;28;43mself\x1B[39;49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43mparameters\x1B[49m\x1B[43m)\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m    108\x1B[0m \x1B[43m            \x1B[49m\x1B[43mplot\x1B[49m\x1B[38;5;241;43m=\x1B[39;49m\x1B[38;5;28;43mself\x1B[39;49m\x1B[38;5;241;43m.\x1B[39;49m\x1B[43mplot\x1B[49m\n' +
      '\x1B[1;32m    109\x1B[0m \x1B[43m        \x1B[49m\x1B[43m)\x1B[49m\n',
    'File \x1B[0;32m~/projects/abfm_apnea/.pixi/envs/default/lib/python3.11/site-packages/catboost/core.py:6980\x1B[0m, in \x1B[0;36mcv\x1B[0;34m(pool, params, dtrain, iterations, num_boost_round, fold_count, nfold, inverted, partition_random_seed, seed, shuffle, logging_level, stratified, as_pandas, metric_period, verbose, verbose_eval, plot, plot_file, early_stopping_rounds, save_snapshot, snapshot_file, snapshot_interval, metric_update_interval, folds, type, return_models, log_cout, log_cerr)\x1B[0m\n' +
      "\x1B[1;32m   6978\x1B[0m \x1B[38;5;28;01mwith\x1B[39;00m log_fixup(log_cout, log_cerr), plot_wrapper(plot, plot_file\x1B[38;5;241m=\x1B[39mplot_file, plot_title\x1B[38;5;241m=\x1B[39m\x1B[38;5;124m'\x1B[39m\x1B[38;5;124mCross-validation plot\x1B[39m\x1B[38;5;124m'\x1B[39m, train_dirs\x1B[38;5;241m=\x1B[39mplot_dirs):\n" +
      '\x1B[1;32m   6979\x1B[0m     \x1B[38;5;28;01mif\x1B[39;00m \x1B[38;5;129;01mnot\x1B[39;00m return_models:\n' +
      '\x1B[0;32m-> 6980\x1B[0m         \x1B[38;5;28;01mreturn\x1B[39;00m \x1B[43m_cv\x1B[49m\x1B[43m(\x1B[49m\n' +
      '\x1B[1;32m   6981\x1B[0m \x1B[43m            \x1B[49m\x1B[43mparams\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6982\x1B[0m \x1B[43m            \x1B[49m\x1B[43mpool\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6983\x1B[0m \x1B[43m            \x1B[49m\x1B[43mfold_count\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6984\x1B[0m \x1B[43m            \x1B[49m\x1B[43minverted\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6985\x1B[0m \x1B[43m            \x1B[49m\x1B[43mpartition_random_seed\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6986\x1B[0m \x1B[43m            \x1B[49m\x1B[43mshuffle\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6987\x1B[0m \x1B[43m            \x1B[49m\x1B[43mstratified\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6988\x1B[0m \x1B[43m            \x1B[49m\x1B[43mmetric_update_interval\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6989\x1B[0m \x1B[43m            \x1B[49m\x1B[43mas_pandas\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6990\x1B[0m \x1B[43m            \x1B[49m\x1B[43mfolds\x1B[49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6991\x1B[0m \x1B[43m            \x1B[49m\x1B[38;5;28;43mtype\x1B[39;49m\x1B[43m,\x1B[49m\n' +
      '\x1B[1;32m   6992\x1B[0m \x1B[43m            \x1B[49m\x1B[43mreturn_models\x1B[49m\n' +
      '\x1B[1;32m   6993\x1B[0m \x1B[43m        \x1B[49m\x1B[43m)\x1B[49m\n' +
      '\x1B[1;32m   6994\x1B[0m     \x1B[38;5;28;01melse\x1B[39;00m:\n' +
      '\x1B[1;32m   6995\x1B[0m         results, cv_models \x1B[38;5;241m=\x1B[39m _cv(\n' +
      '\x1B[1;32m   6996\x1B[0m             params,\n' +
      '\x1B[1;32m   6997\x1B[0m             pool,\n' +
      '\x1B[0;32m   (...)\x1B[0m\n' +
      '\x1B[1;32m   7007\x1B[0m             return_models\n' +
      '\x1B[1;32m   7008\x1B[0m         )\n',
    'File \x1B[0;32m_catboost.pyx:5880\x1B[0m, in \x1B[0;36m_catboost._cv\x1B[0;34m()\x1B[0m\n',
    'File \x1B[0;32m_catboost.pyx:5912\x1B[0m, in \x1B[0;36m_catboost._cv\x1B[0;34m()\x1B[0m\n',
    '\x1B[0;31mCatBoostError\x1B[0m: catboost/private/libs/options/plain_options_helper.cpp:512: Unknown option {use_medicine_embeddings} with value "false"'
  ]
}
15:10:17.635 [info] Restart requested ~/projects/abfm_apnea/notebooks/10 catboost.ipynb
15:10:17.641 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
15:10:17.650 [info] Process Execution: ~/projects/abfm_apnea/.pixi/envs/default/bin/python -m ipykernel_launcher --f=/Users/~/Library/Jupyter/runtime/kernel-v30f0e2ddd7db88e06adce6cb5f2b8c2103339a295.json
    > cwd: ~/projects/abfm_apnea
15:10:18.064 [info] Restarted 32bb1ff7-4988-4853-800c-72bb7a294458
15:10:18.065 [error] Failed to write data to the kernel channel shell [
  <Buffer 3c 49 44 53 7c 4d 53 47 3e>,
  <Buffer 33 30 65 32 65 33 66 63 65 66 32 66 33 63 61 33 32 30 63 36 35 66 61 31 32 36 62 61 39 63 31 33 34 66 37 36 65 30 36 38 36 61 34 34 36 30 66 33 30 38 ... 14 more bytes>,
  <Buffer 7b 22 64 61 74 65 22 3a 22 32 30 32 35 2d 30 31 2d 31 30 54 32 33 3a 30 38 3a 35 35 2e 33 38 33 5a 22 2c 22 6d 73 67 5f 69 64 22 3a 22 32 30 37 32 63 ... 177 more bytes>,
  <Buffer 7b 7d>,
  <Buffer 7b 22 63 65 6c 6c 49 64 22 3a 22 76 73 63 6f 64 65 2d 6e 6f 74 65 62 6f 6f 6b 2d 63 65 6c 6c 3a 2f 55 73 65 72 73 2f 63 63 68 6f 77 2f 70 72 6f 6a 65 ... 62 more bytes>,
  <Buffer 7b 22 73 69 6c 65 6e 74 22 3a 66 61 6c 73 65 2c 22 73 74 6f 72 65 5f 68 69 73 74 6f 72 79 22 3a 74 72 75 65 2c 22 75 73 65 72 5f 65 78 70 72 65 73 73 ... 100 more bytes>
] [Error: Socket is closed
	at a.postToSocket (/Users/~/.vscode/extensions/ms-toolsai.jupyter-2024.11.0-darwin-arm64/dist/extension.node.js:304:8060)
	at /Users/~/.vscode/extensions/ms-toolsai.jupyter-2024.11.0-darwin-arm64/dist/extension.node.js:304:7804] {
  errno: 9,
  code: 'EBADF'
}

Coding Language and Runtime Version

python 3.11.10

Language Extension Version (if applicable)

2024.22.2

Anaconda Version (if applicable)

pixi 0.39.5

Running Jupyter locally or remotely?

Local

[DonJayamanne]

the kernel seems to become unresponsive. I hit Restart, and the kernel appears to restart but fails with an error message.

What is the error that you get?