[FEAT] hub integration with dataset and configuration

[burtenshaw]

This is an experimental WIP pr to get feedback on the approach. I've migrated the code out of the v1 argilla client, and reimplemented for v2 changes. It would be great to get feedback on issues like this:

• testing with a mocked hub api.
• dealing with responses across argilla servers and mismatched user_ids.
• dealing with dependencies like huggingface_hub. I like the decorator used in the v1 client.

Here's a dataset that I pushed. Still uses default readme from v1:

https://huggingface.co/datasets/burtenshaw/test-argilla-dataset

To test this implementation do:

import uuid

from datetime import datetime

import argilla as rg

client = rg.Argilla(api_key="owner.apikey")
workspace = client.workspaces[0]
mock_dataset_name = (
    f"test_add_record_with_suggestions {datetime.now().strftime('%Y%m%d%H%M%S')}"
)
mock_data = [
    {
        "text": "Hello World, how are you?",
        "label": "positive",
        "id": uuid.uuid4(),
        "comment": "I'm doing great, thank you!",
        "topics": ["topic1", "topic2"],
        "topics.score": [0.9, 0.8],
    },
    {
        "text": "Hello World, how are you?",
        "label": "negative",
        "id": uuid.uuid4(),
        "comment": "I'm doing great, thank you!",
        "topics": ["topic3"],
        "topics.score": [0.9],
    },
    {
        "text": "Hello World, how are you?",
        "label": "positive",
        "id": uuid.uuid4(),
        "comment": "I'm doing great, thank you!",
        "comment_score": 0.9,  # This field will be ignored because it is not in the mapping
        "rating": 1,
        "topics": ["topic1", "topic2", "topic3"],
        "topics.score": [0.9, 0.8, 0.7],
        "ranking": ["label1", "label2", "label3"],
        "span": [
            {
                "start": 0,
                "end": 5,
                "label": "label1",
            },
            {
                "start": 6,
                "end": 11,
                "label": "label2",
            },
            {
                "start": 12,
                "end": 17,
                "label": "label3",
            },
        ],
        "vector": [1, 2, 3],
    },
]
settings = rg.Settings(
    fields=[
        rg.TextField(name="text"),
    ],
    questions=[
        rg.LabelQuestion(name="label", labels=["positive", "negative"]),
        rg.RatingQuestion(name="rating", values=[1, 2, 3, 4, 5]),
        rg.RankingQuestion(name="ranking", values=["label1", "label2", "label3"]),
        rg.TextQuestion(name="comment", use_markdown=False),
        rg.MultiLabelQuestion(
            name="topics",
            labels=["topic1", "topic2", "topic3"],
            labels_order="suggestion",
        ),
        rg.SpanQuestion(
            name="span", labels=["label1", "label2", "label3"], field="text"
        ),
    ],
    metadata=[
        rg.FloatMetadataProperty(name="comment_score"),
    ],
    vectors=[
        rg.VectorField(name="vector", dimensions=3),
    ],
)
dataset = rg.Dataset(
    name=mock_dataset_name,
    settings=settings,
    client=client,
)
dataset.create()
dataset.records.log(
    mock_data,
    mapping={
        "comment": "comment.suggestion",
        "comment_score": "comment.suggestion.score",  # This field will be ignored because it is not in the mapping
        "topics": "topics.suggestion",
        "topics.score": "topics.suggestion.score",
        "label": "label.response",
        "span": "span",
    },
)

dataset.to_hub(repo_id="burtenshaw/test-argilla-dataset")


pulled_dataset = from_huggingface("burtenshaw/test-argilla-dataset")

[Wauplin]

Final review from my side. Integration looks good to me! ✔️
Left 2 small comments regarding future maintenance.