Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Torchfx dynamic shapes additional support #8

Merged
merged 53 commits into from
Jul 8, 2024
Merged

Torchfx dynamic shapes additional support #8

merged 53 commits into from
Jul 8, 2024

Conversation

cavusmustafa
Copy link
Owner

Details:

  • item1
  • ...

Tickets:

  • ticket-id

cavusmustafa and others added 30 commits June 30, 2024 18:05
@cavusmustafa
Using helper function to create concat in expand translation
8f31331
@cavusmustafa
Name change for helper function to crate concat from list
724dc37
@cavusmustafa
Updated concat_list_from_inputs to use int64
712eaea
@cavusmustafa
Arange input check updated
a35588f
@cavusmustafa
Code formating fix for arange translation
db2a8c3
@cavusmustafa
Dynamic expand temp. fix for expand being last op
004b343
@cavusmustafa
Merge branch 'master' into torchfx_dynamic_shapes_additional_support
aceeecf
@yeonbok
[GPU] Horizontal fusion for FC (openvinotoolkit#24991)
f28fe6a 
### Details:
 - Fuse QKV FCs to one FC 

### Tickets:
 - 142815
@wilson-seok
[GPU] Skip fc fake alignment when fused desc has full tensor input (o…
a30f0a5 
…penvinotoolkit#25249)

### Details:
 - Skip fc fake alignment when fused desc has full tensor input

### Tickets:
 - 144293
@ahnyoung-paul
[GPU] update shape for fused prims (openvinotoolkit#25363)
01dc53a 
### Details:
- *update shape for user when current node is the input of fused prim of
user even if the user is updated shape by other node*
 - *...*

### Tickets:
 - *145756*
@andrei-cv @p-durandin
[GPU] Fix CVS-145082 (openvinotoolkit#25340)
5e28c2a 
### Details:
 - Coverity fixes

### Tickets:
 - *CVS-145082*

---------

Co-authored-by: Pavel Durandin <[email protected]>
@pkowalc1
[GPU][unittest]: Fixed mem leak in roi align rotated unittests. (open…
ea7921c 
…vinotoolkit#25356)

### Details:
 - Fixed mem leak in cl engine in roi align rotated test.

### Tickets:
 - *CVS-145836*
@eaidova
[PT FE]: update layer norm for support non-static normalized shape (o…
977b426 
…penvinotoolkit#25365)

### Details:
- *aten::layer_norm support extended for support normalized shapes
represented as non constant and multiple dims*
 tested with stable-audio model

### Tickets:
 - TBD
@akopytko
[DOCS] port sidebar menu arrows to master (openvinotoolkit#25371)
e7b0072 
Port for:
openvinotoolkit#25370
@andrew-k-park
[GPU] Convert multiple StridedSlices to single VariadicSplit after fu…
ce80952 
…sed QKV FC (openvinotoolkit#25304)

### Details:
- Convert QKV FC + StridedSlices pattern to QKV FC + VariadicSplit for
potential optimization regarding crop runtime buffer fusing

### Tickets:
 - 144394
@t-jankowski
[Templ test] Tan: Enable whole Tensor comparison (openvinotoolkit#25320)
82af474 
### Details:
 - Enabled whole Tensor comparison

### Tickets:
 - CVS-137211
@maxnick
[CPU] Add Clamp before Convert when bool is replaced with u8 (openvin…
c89db83 
…otoolkit#25253)

### Details:
CPU plugin doesn't natively support `boolean` data type, thus it's
replaced with `u8` during the convert precision transformation pass.
However, casting numerical types to `bool` implies clamping the
numerical values with the interval [0; 1] (either true or false), so to
mimic this behavior a Clamp operation should be inserted before the
modified convert.

### Tickets:
 - CVS-145166
@sbalandi
Move gtest assert to ov_assert (openvinotoolkit#24933)
a0c6eb6 
### Details:
 - *item1*
 - *...*

### Tickets:
 - *CVS-136937*
@mvafin
[PT FE] Fix issue with type alignment in aten::cat (openvinotoolkit#2…
e732ade 
…5354)

### Details:
 - *Fix regression introduced by openvinotoolkit#25077*

### Tickets:
 - *CVS-144695*
 - *CVS-144789*
@CuriousPanCake
Add unique models to precommit PA tests (openvinotoolkit#25279)
3faa656 
Add unique models to precommit PA tests

Change the list of models covering all the big real models.

Removed:
hf-tiny-model-private/tiny-random-GPT2LMHeadModel
hf-tiny-model-private/tiny-random-BartForCausalLM,xfail
hf-tiny-model-private/tiny-random-BigBirdForCausalLM,xfail
hf-tiny-model-private/tiny-random-BigBirdPegasusForCausalLM,xfail
hf-tiny-model-private/tiny-random-BlenderbotSmallForCausalLM,xfail
hf-tiny-model-private/tiny-random-ErnieForCausalLM,xfail
hf-tiny-model-private/tiny-random-GPTNeoXJapaneseForCausalLM,xfail
hf-tiny-model-private/tiny-random-MBartForCausalLM,xfail
hf-tiny-model-private/tiny-random-MvpForCausalLM,xfail
hf-tiny-model-private/tiny-random-PLBartForCausalLM,xfail

PrunaAI/hf-tiny-model-private-tiny-random-BloomForCausalLM-bnb-4bit-smashed,xfail

PrunaAI/hf-tiny-model-private-tiny-random-BloomForCausalLM-bnb-8bit-smashed,xfail

PrunaAI/hf-tiny-model-private-tiny-random-BloomForCausalLM-HQQ-2bit-smashed,xfail
hf-internal-testing/tiny-random-MegatronBertForCausalLM,xfail
hf-internal-testing/tiny-random-BlenderbotSmallForCausalLM,xfail

Added:
hf-internal-testing/Mixtral-tiny
hf-internal-testing/tiny-random-gpt2
katuni4ka/tiny-random-xverse
katuni4ka/tiny-random-baichuan2-13b
katuni4ka/tiny-random-qwen
katuni4ka/tiny-random-aquilachat
katuni4ka/tiny-random-aquila2
katuni4ka/tiny-random-qwen1.5-moe
katuni4ka/tiny-random-codegen2
katuni4ka/tiny-random-olmo-hf
katuni4ka/tiny-random-baichuan2
katuni4ka/tiny-random-jais
katuni4ka/tiny-random-internlm
katuni4ka/tiny-random-internlm2
katuni4ka/tiny-random-minicpm
fxmarty/tiny-random-GemmaForCausalLM
fxmarty/tiny-dummy-qwen2
fxmarty/really-tiny-falcon-testing
Xenova/tiny-random-Phi3ForCausalLM
facebook/opt-350m
katuni4ka/tiny-random-dbrx
katuni4ka/tiny-random-falcon-40b
katuni4ka/tiny-random-orion,xfail,No ScaledDotProductAttention operation
observed in the graph
katuni4ka/tiny-random-chatglm2,xfail,Model references undeclared
parameters: beam_idx ()
katuni4ka/tiny-random-glm4,xfail,Model references undeclared parameters
beam_idx () attention_mask ()


### Tickets:
 - CVS-145243

Signed-off-by: Andrii Staikov <[email protected]>

---------

Signed-off-by: Andrii Staikov <[email protected]>
@vishniakov-nikolai
[OV JS] Allow partial shape creation without parameter (openvinotoolk…
bbb32b3 
…it#25311)

### Details:
- Call `getPartialShape` method on dynamic input/output lead to error.
This change fix that.
@tsavina @kblaszczak-intel
[DOCS] Running inference comments (openvinotoolkit#25285)
bf84cec 
### Details:
 - *item1*
 - *...*

### Tickets:
 - 145078

---------

Co-authored-by: Karol Blaszczak <[email protected]>
@peterchen-intel
Roll Linux X86 oneTBB back to 2021.2.4 (openvinotoolkit#25347)
4204470 
Reason: LLM beam search perf regression on XEON platforms

Signed-off-by: Chen Peter <[email protected]>
@IvanNovoselov
[Snippets] Implement runtime loop info serialization in netron graphs (…
537744c 
…openvinotoolkit#25235)

### Details:
 - *Propagate information from runtime loop info to serialized graphs*

### Tickets:
 - *-*
@Lyamin-Roman
[GPU] Added more uses of const references (openvinotoolkit#25374)
e6d5ba0
@EgorDuplensky
[CPU] Fix RNN node isSupported check (openvinotoolkit#24143)
d0b3a3c 
Incorrect check resulted into decompostion of different RNN operations
@akopytko
[DOCS] PORT to master with Breadcrumbs fix (openvinotoolkit#25380)
e4cfd8f 
Port for:
openvinotoolkit#25379
@mvafin
[PT FE] Support aten::col2im (openvinotoolkit#25376)
505a1ae 
### Details:
 - *Support `aten::col2im`*

### Tickets:
 - *CVS-101054*
@slyalin
Stateful to Stateless Transformation for LLMs (openvinotoolkit#25150)
2a9af43 
Transformation that undoing make_stateful from optimum-intel.

### How to use in Python
```python
import openvino as ov
from openvino._offline_transformations import stateful_to_stateless_transformation
core = ov.Core()
model = core.read_model('your_chatty_stateful_model_right_from_vanilla_optimum_intel.xml')
stateful_to_stateless_transformation(model)
# use `model`
```

### How to use in C++
```c++
#include <openvino/openvino.hpp>
#include <openvino/pass/stateful_to_stateless.hpp>

int main() {
    auto core = ov::Core();
    auto model = core.read_model("your_chatty_stateful_model_right_from_vanilla_optimum_intel.xml");
    ov::pass::StatefulToStateless().run_on_model(model);
    // use `model`
}
```

### TODO

- [x] Restore the original order of inputs/output (now they are not
globally ordered, but kv inputs corresponds to kv outputs by indices
with a proper offset).
- [x] Restore the original names of inputs and outputs based on
optimum-intel conventions in make_stateful.
@mryzhov
[GHA] Fixed tokenizers build (openvinotoolkit#25383)
8b01073 
### Details:
- Added path to tokenizers build to find ov wheel
(openvinotoolkit/openvino_tokenizers#183)
 
### Tickets:
 - *ticket-id*
@cavusmustafa cavusmustafa merged commit 99d1fcd into torchfx_dynamic_shapes_split_slice_support Jul 8, 2024
19 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
category: AUTO BATCH category: AUTO category: build category: CI category: Core category: CPP API category: CPU category: dependency_changes category: docs category: GPU category: HETERO category: IE Tests category: inference category: IR FE category: JS API category: LP transformations category: MO category: ONNX FE category: PDPD FE category: Python API category: PyTorch FE category: TEMPLATE category: TF FE category: TFL FE category: transformations github_actions no-match-files
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.