Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add humanvbench operators #553

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add humanvbench operators #553

wants to merge 1 commit into from

Conversation

SYSUzhouting
Copy link
Collaborator

./data-juicer/my_pretrained_method 中的文件都是从对应名称项目git clone下来的

@HYLcool HYLcool mentioned this pull request Jan 17, 2025
@HYLcool HYLcool added dj:op issues/PRs about some specific OPs good first issue Good for newcomers dj:multimodal issues/PRs about multimodal data processing labels Jan 17, 2025
@HYLcool HYLcool requested review from HYLcool and yxdyc January 17, 2025 07:00
Copy link
Collaborator

@yxdyc yxdyc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code under my_pretrained_method is not suitable for incorporation as it consists mostly of original source code from other repositories (approximately 6k~7k lines), which is not so closely related to DJ's core part. Also, the licenses of these codes need to be checked carefully.

To address this, follow these steps in order:

  • Hugging Face/ModelScope Models: If the models are available in Hugging Face or ModelScope, use them with the help of these repositories.
  • Pip Package: If the models are in a special third-party pip package, add the repository to DJ's python env requirements.
  • Fallback Options: If the above steps do not work, consider
    • finding and using some similar models from above hf/modelscope/pip, then if someone needs reproducible code, we can refer to original HumanVBench branch;
    • making the codes self-contained in the OP if the license is suitable, similar to sdxl_prompt2prompt_mapper
    • adding their source codes (or the git diff patch) under the thirdparty directory of DJ;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dj:multimodal issues/PRs about multimodal data processing dj:op issues/PRs about some specific OPs good first issue Good for newcomers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants