Code

Add Comet integration#1

Code

Draft

Lothiraldan wants to merge 42 commits intomaincomet-ml/gpt-neox:mainfrom comet-integrationcomet-ml/gpt-neox:comet-integrationCopy head branch name to clipboard

+3,487-436

Commits

Commits on Jun 11, 2024

• First version of the GPT-Neox <> Comet integration

  Lothiraldan

  committed
  52db752

  Copy full SHA for 52db752

• Add basic Comet configuration

  Lothiraldan

  committed
  463aad9

  Copy full SHA for 463aad9

Commits on Jun 13, 2024

• Add configuration options for customizing the experiment name, tags and others fields

  Lothiraldan

  committed
  804cbcc

  Copy full SHA for 804cbcc

• Change one type hint to make validate_types happy

  Lothiraldan

  committed
  108f583

  Copy full SHA for 108f583

• Add debugging logs

  Lothiraldan

  committed
  90ef7ae

  Copy full SHA for 90ef7ae

• Fix typo

  Lothiraldan

  committed
  12e064b

  Copy full SHA for 12e064b

• Fix typo

  Lothiraldan

  committed
  d0b68d7

  Copy full SHA for d0b68d7

Commits on Jun 19, 2024

• fix python version and pytest install (EleutherAI#1234)

  Show description for 2608972
  

  3 peopleauthored
  2608972

  Copy full SHA for 2608972

Commits on Jun 25, 2024

• Add a chat data preprocessing script (EleutherAI#1239)

  Show description for 0e5f6db
  

  dmahan93

  and

  Quentin-Anthony

  authored
  0e5f6db

  Copy full SHA for 0e5f6db

Commits on Jun 28, 2024

• Fix paper reference in init_functions.py (EleutherAI#1241)

  rasbt

  authored
  1cee5b7

  Copy full SHA for 1cee5b7

Commits on Aug 6, 2024

• Add hf llama to neox conversion (EleutherAI#1247)

  Show description for c1ea2a1
  

  dmahan93

  and

  Quentin-Anthony

  authored
  c1ea2a1

  Copy full SHA for c1ea2a1

Commits on Aug 15, 2024

• bugfix: chat turns instead of repeating the conversation in preprocess_data_with_chat_template.py (EleutherAI#1258)

  Show description for 0ef2c07
  dmahan93

  authored
  0ef2c07

  Copy full SHA for 0ef2c07

• Conversion for CI from self-hosted hardware (EleutherAI#1245)

  Show description for f8c9e68
  jaimemcc-intel

  authored
  f8c9e68

  Copy full SHA for f8c9e68

Commits on Aug 23, 2024

• Megatron-LM style Sequence Parallel (EleutherAI#1257)

  Show description for 8b43196
  

  4 peopleauthored
  8b43196

  Copy full SHA for 8b43196

Commits on Aug 24, 2024

• Add new cites (EleutherAI#1255)

  Show description for e7c0182
  

  3 peopleauthored
  e7c0182

  Copy full SHA for e7c0182

Commits on Aug 27, 2024

• mamba fixes and cleaning (EleutherAI#1262)

  Show description for 591563d
  

  jahatef

  and

  Jacob Hatef

  authored
  591563d

  Copy full SHA for 591563d

• SFT improvements (labeling fixes, different packing implementations) (EleutherAI#1240)

  Show description for c786367
  dmahan93

  authored
  c786367

  Copy full SHA for c786367

Commits on Sep 3, 2024

• pre-commit and logging cleanup

  Quentin-Anthony

  committed
  6a2053b

  Copy full SHA for 6a2053b

Commits on Sep 5, 2024

• add assert for missing tokenizer_type in config (EleutherAI#1267)

  AI-WAIFU

  authored
  7548a8b

  Copy full SHA for 7548a8b

Commits on Sep 7, 2024

• Add intermediate_size to GPT-NeoX models (EleutherAI#1212)

  Show description for 0d4bdb9
  

  3 peopleauthored
  0d4bdb9

  Copy full SHA for 0d4bdb9

Commits on Sep 8, 2024

• apply pre-commit and add missing close-paren to mamba config (EleutherAI#1270)

  Quentin-Anthony

  authored
  ec82c05

  Copy full SHA for ec82c05

• Add DPO training (EleutherAI#1242)

  Show description for 77e8158
  

  dmahan93

  and

  Quentin-Anthony

  authored
  77e8158

  Copy full SHA for 77e8158

Commits on Sep 9, 2024

• LayerNorm Refactor (EleutherAI#1269)

  Show description for 836aefa
  

  8 peopleauthored
  836aefa

  Copy full SHA for 836aefa

• hotfix activation typo from EleutherAI#1212 (EleutherAI#1271)

  Quentin-Anthony

  authored
  01e74f4

  Copy full SHA for 01e74f4

• TE Import Hotfix (EleutherAI#1272)

  Show description for 61a3daa
  Quentin-Anthony

  authored
  61a3daa

  Copy full SHA for 61a3daa

• Add Reward Model training (EleutherAI#1246)

  Show description for 1c72742
  

  dmahan93

  and

  Quentin-Anthony

  authored
  1c72742

  Copy full SHA for 1c72742

• add comets requirements file

  Quentin-Anthony

  committed
  bf8e78c

  Copy full SHA for bf8e78c

• First version of the GPT-Neox <> Comet integration

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  227967f

  Copy full SHA for 227967f

• Add basic Comet configuration

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  2a3513c

  Copy full SHA for 2a3513c

• Add configuration options for customizing the experiment name, tags and others fields

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  7609829

  Copy full SHA for 7609829

• Change one type hint to make validate_types happy

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  39f9142

  Copy full SHA for 39f9142

• Add debugging logs

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  43ed6e8

  Copy full SHA for 43ed6e8

• Fix typo

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  90c499a

  Copy full SHA for 90c499a

• Fix typo

  

  Lothiraldan

  authored and

  Quentin-Anthony

  committed
  913f877

  Copy full SHA for 913f877

• pre-commit and logging cleanup

  Quentin-Anthony

  committed
  a6bddd6

  Copy full SHA for a6bddd6

• add comets requirements file

  Quentin-Anthony

  committed
  0468dae

  Copy full SHA for 0468dae

• Merge branch 'comet-integration' of https://github.com/comet-ml/gpt-neox into comet-integration

  Quentin-Anthony

  committed
  ef32d69

  Copy full SHA for ef32d69

• precommit

  Quentin-Anthony

  committed
  976cd5d

  Copy full SHA for 976cd5d

• add comet config

  Quentin-Anthony

  committed
  962314e

  Copy full SHA for 962314e

• update readme and arg comment

  Quentin-Anthony

  committed
  f0a4b70

  Copy full SHA for f0a4b70

• Merge branch 'comet-integration' of https://github.com/comet-ml/gpt-neox into comet-integration

  Quentin-Anthony

  committed
  c6681b5

  Copy full SHA for c6681b5

• revert unrelated config changes

  Quentin-Anthony

  committed
  4f76e0d

  Copy full SHA for 4f76e0d