Code

Add fastapi support#2370

Code

Closed

rainyfly wants to merge 73 commits intoPaddlePaddle:developPaddlePaddle/FastDeploy:developfrom rainyfly:add_fastapi_supportrainyfly/FastDeploy:add_fastapi_supportCopy head branch name to clipboard

+6,181

Commits

Commits on Sep 25, 2023

• add codee

  

  root

  committed
  9e7ef07

  Copy full SHA for 9e7ef07

• add copyright

  jiangjiajun

  committed
  4aa21bd

  Copy full SHA for 4aa21bd

• fix some bugs

  jiangjiajun

  committed
  6fd06f7

  Copy full SHA for 6fd06f7

• Update prefix_utils.py

  jiangjiajun

  authored
  9f569a7

  Copy full SHA for 9f569a7

• Update triton_model.py

  jiangjiajun

  authored
  bdf2748

  Copy full SHA for bdf2748

• Update triton_model.py

  jiangjiajun

  authored
  52aaffb

  Copy full SHA for 52aaffb

• fix tokenizer

  jiangjiajun

  committed
  0199cac

  Copy full SHA for 0199cac

• Add check for prefix len

  jiangjiajun

  committed
  fa151a7

  Copy full SHA for fa151a7

Commits on Sep 26, 2023

• Create README.md

  jiangjiajun

  authored
  800c6a9

  Copy full SHA for 800c6a9

• Create test_client.py

  jiangjiajun

  authored
  91ea8bb

  Copy full SHA for 91ea8bb

• Update task.py

  jiangjiajun

  authored
  5e8221e

  Copy full SHA for 5e8221e

• add debug log and fix ptuning

  jiangjiajun

  committed
  9897924

  Copy full SHA for 9897924

• update version

  jiangjiajun

  committed
  8d1e691

  Copy full SHA for 8d1e691

Commits on Oct 7, 2023

• Update triton_model.py

  jiangjiajun

  authored
  388eb9b

  Copy full SHA for 388eb9b

Commits on Oct 8, 2023

• Update README.md

  jiangjiajun

  authored
  68c15b6

  Copy full SHA for 68c15b6

Commits on Oct 10, 2023

• Support chatglm-6b (PaddlePaddle#2223)

  Show description for 30a3beb
  

  jiangjiajun

  and

  root

  authored
  30a3beb

  Copy full SHA for 30a3beb

Commits on Oct 11, 2023

• Support bloom (PaddlePaddle#2232)

  jiangjiajun

  authored
  b96a92b

  Copy full SHA for b96a92b

• Support multicards (PaddlePaddle#2234)

  jiangjiajun

  authored
  80bb8ed

  Copy full SHA for 80bb8ed

Commits on Oct 12, 2023

• [LLM] Add prefix for chatglm (PaddlePaddle#2233)

  Show description for 986b233
  rainyfly

  authored
  986b233

  Copy full SHA for 986b233

• Update engine.py

  jiangjiajun

  authored
  9fa04c3

  Copy full SHA for 9fa04c3

Commits on Oct 13, 2023

• [LLM] Fix P-Tuning difference (PaddlePaddle#2240)

  Show description for e6a7d4e
  jiangjiajun

  authored
  e6a7d4e

  Copy full SHA for e6a7d4e

Commits on Oct 16, 2023

• [LLM] Support prefix for bloom (PaddlePaddle#2237)

  Show description for 51d8697
  rainyfly

  authored
  51d8697

  Copy full SHA for 51d8697

Commits on Oct 17, 2023

• Support bloom prefix (PaddlePaddle#2245)

  Show description for 73c1507
  rainyfly

  authored
  73c1507

  Copy full SHA for 73c1507

Commits on Oct 18, 2023

• [LLM] Fix serving (PaddlePaddle#2246)

  Show description for 528e976
  jiangjiajun

  authored
  528e976

  Copy full SHA for 528e976

• fix chatglm

  jiangjiajun

  committed
  1cbbaee

  Copy full SHA for 1cbbaee

• Update config.py

  jiangjiajun

  authored
  2f2c824

  Copy full SHA for 2f2c824

Commits on Oct 19, 2023

• [LLM] Support bloom prefix (PaddlePaddle#2248)

  Show description for 66a4897
  rainyfly

  authored
  66a4897

  Copy full SHA for 66a4897

• [LLM] Add simple client

  jiangjiajun

  committed
  4d956d3

  Copy full SHA for 4d956d3

• add requirements

  jiangjiajun

  committed
  a5a261b

  Copy full SHA for a5a261b

Commits on Oct 20, 2023

• [LLM] Support dynamic batching for chatglm (PaddlePaddle#2251)

  Show description for 4c21588
  jiangjiajun

  authored
  4c21588

  Copy full SHA for 4c21588

• [LLM] Support dybatch for bloom (PaddlePaddle#2255)

  jiangjiajun

  authored
  8ff24d6

  Copy full SHA for 8ff24d6

• remove +1 for chatglm

  jiangjiajun

  committed
  3a4f8a9

  Copy full SHA for 3a4f8a9

• Update setup.py

  jiangjiajun

  authored
  e5da0f1

  Copy full SHA for e5da0f1

Commits on Oct 24, 2023

• Add check for prefix and compatible with lite

  jiangjiajun

  committed
  6da9555

  Copy full SHA for 6da9555

• add requires

  jiangjiajun

  committed
  10eefcb

  Copy full SHA for 10eefcb

• Support gpt

  jiangjiajun

  committed
  fb0f276

  Copy full SHA for fb0f276

Commits on Oct 25, 2023

• Fix triton model problem

  jiangjiajun

  committed
  7193337

  Copy full SHA for 7193337

• Update version

  jiangjiajun

  committed
  70f8469

  Copy full SHA for 70f8469

Commits on Oct 26, 2023

• Add some tools

  jiangjiajun

  committed
  b116e3e

  Copy full SHA for b116e3e

Commits on Nov 6, 2023

• test

  

  root

  committed
  b86524e

  Copy full SHA for b86524e

Commits on Nov 7, 2023

• Update triton_model.py

  jiangjiajun

  authored
  2e6bc1a

  Copy full SHA for 2e6bc1a

• Update setup.py

  jiangjiajun

  authored
  cabebc3

  Copy full SHA for cabebc3

• Update README.md

  jiangjiajun

  authored
  cdc0ff2

  Copy full SHA for cdc0ff2

• test FastDeploy

  

  root

  committed
  7c23864

  Copy full SHA for 7c23864

• Merge branch 'llm' into llm

  karagg

  authored
  cca470f

  Copy full SHA for cca470f

Commits on Nov 8, 2023

• test

  

  root

  committed
  fb5d5c5

  Copy full SHA for fb5d5c5

Commits on Nov 9, 2023

• [LLM] Add ci test scripts (PaddlePaddle#2272)

  Show description for 2d2274c
  

  karagg

  and

  root

  authored
  2d2274c

  Copy full SHA for 2d2274c

Commits on Nov 14, 2023

• delete run.sh

  

  root

  committed
  a55837e

  Copy full SHA for a55837e

• Merge branch 'PaddlePaddle:llm' into llm

  karagg

  authored
  f9c8581

  Copy full SHA for f9c8581

• delete run.sh

  

  root

  committed
  1f76abf

  Copy full SHA for 1f76abf

• update run.sh

  

  root

  committed
  9c6b2de

  Copy full SHA for 9c6b2de

• update run.sh ci.py

  

  root

  committed
  ceb49a4

  Copy full SHA for ceb49a4

Commits on Nov 15, 2023

• update ci.py

  

  root

  committed
  9499199

  Copy full SHA for 9499199

• update ci.py

  

  root

  committed
  8bf70a1

  Copy full SHA for 8bf70a1

• [LLM]update ci test script (PaddlePaddle#2285)

  Show description for 6e15209
  

  karagg

  and

  root

  authored
  6e15209

  Copy full SHA for 6e15209

• debug

  

  root

  committed
  be12232

  Copy full SHA for be12232

• debug

  

  root

  committed
  f884c1a

  Copy full SHA for f884c1a

• Merge pull request PaddlePaddle#2286 from karagg/llm

  Show description for 57e7608
  Zeref996

  authored
  57e7608

  Copy full SHA for 57e7608

• debug

  

  root

  committed
  7b80d70

  Copy full SHA for 7b80d70

Commits on Nov 16, 2023

• Merge pull request PaddlePaddle#2288 from karagg/llm

  Show description for bb68a7e
  Zeref996

  authored
  bb68a7e

  Copy full SHA for bb68a7e

• debug

  

  root

  committed
  6cb1474

  Copy full SHA for 6cb1474

• Merge pull request PaddlePaddle#2289 from karagg/llm

  Show description for 71652e3
  Zeref996

  authored
  71652e3

  Copy full SHA for 71652e3

Commits on Nov 17, 2023

• update run.sh

  

  root

  committed
  261e519

  Copy full SHA for 261e519

Commits on Nov 20, 2023

• add comment

  

  root

  committed
  836d21f

  Copy full SHA for 836d21f

• do not merge

  

  root

  committed
  87f53ea

  Copy full SHA for 87f53ea

Commits on Nov 23, 2023

• Rename test_max_batch_size.sh to test_max_batch_size.py

  jiangjiajun

  authored
  66c4563

  Copy full SHA for 66c4563

Commits on Dec 4, 2023

• update

  

  root

  committed
  79e6a1e

  Copy full SHA for 79e6a1e

Commits on Dec 5, 2023

• Merge pull request PaddlePaddle#2291 from karagg/llm

  Show description for 3376284
  Zeref996

  authored
  3376284

  Copy full SHA for 3376284

Commits on Dec 14, 2023

• Improve robustness for llm (PaddlePaddle#2321)

  Show description for fda8c37
  rainyfly

  authored
  fda8c37

  Copy full SHA for fda8c37

• detail log for llm (PaddlePaddle#2325)

  Show description for cc89731
  rainyfly

  authored
  cc89731

  Copy full SHA for cc89731

• Fix a bug for llm serving (PaddlePaddle#2326)

  Show description for 67ca253
  rainyfly

  authored
  67ca253

  Copy full SHA for 67ca253

Commits on Dec 27, 2023

• Add warning for server hangs (PaddlePaddle#2333)

  Show description for 7bddc67
  rainyfly

  authored
  7bddc67

  Copy full SHA for 7bddc67

Commits on Feb 18, 2024

• Add fastapi support

  rainyfly

  committed
  23d877f

  Copy full SHA for 23d877f