Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] argilla: add support to distribution #5187

Merged
merged 60 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
f62d58a
feat: add dataset support to be created using distribution settings (…
jfcalvo Jul 1, 2024
017001f
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 1, 2024
f084ab7
✨ Remove unused method
damianpumar Jul 4, 2024
c8ef4c6
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 4, 2024
6df5256
feat: improve Records `responses_submitted` relationship to be view o…
jfcalvo Jul 4, 2024
dbae135
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 4, 2024
cf3408c
feat: change metrics to support new distribution task logic (#5140)
jfcalvo Jul 4, 2024
8e8b116
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
frascuchon Jul 5, 2024
808c837
[ENHANCEMENT]: `argilla-server`: allow update distribution for non an…
frascuchon Jul 8, 2024
3d74a33
chore: Add status field to record model
frascuchon Jul 9, 2024
7b7d2f5
feat: Add read-only property 'status' to the record resource
frascuchon Jul 9, 2024
736bfc9
tests: Update tests to reflect the status property
frascuchon Jul 9, 2024
f241e41
fix: wrong filter naming after merge from develop
frascuchon Jul 9, 2024
307b38c
Merge branch 'feat/add-dataset-automatic-task-distribution' into feat…
frascuchon Jul 9, 2024
67d4ee3
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 9, 2024
9b84dcf
chore: Remove message match (depends on python version
frascuchon Jul 9, 2024
08e5757
chore: Add task distribution model
frascuchon Jul 9, 2024
443b9d0
feat: Add support to task distribution
frascuchon Jul 9, 2024
303361a
tests: Update tests with task distribution
frascuchon Jul 9, 2024
43ba10f
chore: Use main TaskDistribution naning
frascuchon Jul 9, 2024
d6c186b
ci: Using feat branch docker image
frascuchon Jul 9, 2024
3e06890
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 9, 2024
aba06c7
Update argilla/src/argilla/_models/_dataset.py
frascuchon Jul 10, 2024
f2238e6
chore: Apply format suggestions
frascuchon Jul 10, 2024
b73004a
Merge branch 'feat/argilla/add-record-status-property' into feat/argi…
frascuchon Jul 10, 2024
2ea0a3e
chore: Export distribution in dataset
frascuchon Jul 10, 2024
b15de8f
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
frascuchon Jul 11, 2024
f497140
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 11, 2024
bec0b0d
feat: add session helper with serializable isolation level (#5165)
jfcalvo Jul 12, 2024
8bf8abb
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 12, 2024
85e847f
[REFACTOR] `argilla-server`: remove deprecated records endpoint (#5206)
frascuchon Jul 12, 2024
1041487
chore: Add task distribution setter for dataset
frascuchon Jul 12, 2024
22263d8
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 12, 2024
c219764
[ENHANCEMENT] `argilla`: add record `status` property (#5184)
frascuchon Jul 12, 2024
ced0220
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 12, 2024
0c85b9d
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 15, 2024
a9375c1
[REFACTOR] cleaning list records endpoints (#5221)
frascuchon Jul 15, 2024
46f2640
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
frascuchon Jul 15, 2024
f77341e
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
frascuchon Jul 15, 2024
b456600
improvement: capture and retry database concurrent update errors (#5227)
jfcalvo Jul 16, 2024
8dd1c7e
chore: update CHANGELOG.md
jfcalvo Jul 16, 2024
4417af6
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 16, 2024
f284720
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 16, 2024
1a50c3a
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 16, 2024
ba3dc49
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
frascuchon Jul 17, 2024
08b29e0
Merge branch 'feat/add-dataset-automatic-task-distribution' into feat…
frascuchon Jul 17, 2024
20ae663
🔀 Update UI for distribution task (#5219)
leiyre Jul 18, 2024
d77e9a8
fixing tests
frascuchon Jul 18, 2024
c9b865b
chore: Add distribution check
frascuchon Jul 18, 2024
9dba7ef
chore: set tools line-height to 88 characters
jfcalvo Jul 18, 2024
103556e
Revert "chore: set tools line-height to 88 characters"
jfcalvo Jul 18, 2024
e7d4b75
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 18, 2024
128e8a0
Merge branch 'develop' into feat/add-dataset-automatic-task-distribution
jfcalvo Jul 18, 2024
504ff7b
[ENHANCEMENT] improve es mappings for responses (#5228)
frascuchon Jul 18, 2024
940a812
Merge branch 'feat/add-dataset-automatic-task-distribution' into feat…
frascuchon Jul 18, 2024
42b80aa
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Jul 18, 2024
b0b1846
[Docs] task distribution (#5246)
nataliaElv Jul 19, 2024
46bf786
Merge branch 'develop' into feat/argilla/add-support-to-distribution
frascuchon Jul 19, 2024
b3bca5f
Apply suggestions from code review
frascuchon Jul 19, 2024
0665271
Merge branch 'develop' into feat/argilla/add-support-to-distribution
frascuchon Jul 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -22,25 +22,11 @@ export const useDatasetProgressViewModel = ({

progressRanges.value = [
{
id: "submitted",
name: t("datasets.submitted"),
id: "completed",
name: t("datasets.completed"),
color: "#0508D9",
value: progress.value.submitted,
tooltip: `${progress.value.submitted}/${progress.value.total}`,
},
{
id: "conflicting",
name: t("datasets.conflicting"),
color: "#8893c0",
value: progress.value.conflicting,
tooltip: `${progress.value.conflicting}/${progress.value.total}`,
},
{
id: "discarded",
name: t("datasets.discarded"),
color: "#b7b7b7",
value: progress.value.discarded,
tooltip: `${progress.value.discarded}/${progress.value.total}`,
value: progress.value.completed,
tooltip: `${progress.value.completed}/${progress.value.total}`,
},
{
id: "pending",
Expand Down
5 changes: 5 additions & 0 deletions argilla-frontend/translation/de.js
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,11 @@ export default {
datasetSettings: "einstellungen",
userSettings: "meine einstellungen",
},
datasets: {
left: "übrig",
completed: "Vollendet",
pending: "Ausstehend",
},
recordStatus: {
pending: "Ausstehend",
draft: "Entwurf",
Expand Down
4 changes: 1 addition & 3 deletions argilla-frontend/translation/en.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,7 @@ export default {
},
datasets: {
left: "left",
submitted: "Submitted",
conflicting: "Conflicting",
discarded: "Discarded",
completed: "Completed",
pending: "Pending",
},
recordStatus: {
Expand Down
28 changes: 14 additions & 14 deletions argilla-frontend/v1/domain/entities/dataset/Metrics.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,67 +20,67 @@ describe("Metrics", () => {

describe("total", () => {
it("should return the total number of records", () => {
const metrics = new Metrics(1, 0, 0, 0, 0);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.total;

expect(result).toEqual(1);
expect(result).toEqual(15);
});
});

describe("responded", () => {
it("should return the number of responded records", () => {
const metrics = new Metrics(5, 5, 3, 1, 1);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.responded;

expect(result).toEqual(5);
expect(result).toEqual(10);
});
});

describe("pending", () => {
it("should return the number of pending records", () => {
const metrics = new Metrics(5, 4, 3, 1, 0);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.pending;

expect(result).toEqual(1);
expect(result).toEqual(5);
});
});

describe("progress", () => {
it("should return the progress of responded records", () => {
const metrics = new Metrics(5, 4, 3, 1, 0);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.progress;

expect(result).toEqual(0.8);
expect(result).toEqual(0.6666666666666666);
});
});

describe("percentage", () => {
it("should return the percentage of draft records", () => {
const metrics = new Metrics(5, 4, 3, 1, 1);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.percentage.draft;

expect(result).toEqual(20);
expect(result).toEqual(6.666666666666667);
});

it("should return the percentage of submitted records", () => {
const metrics = new Metrics(5, 4, 3, 1, 1);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.percentage.submitted;

expect(result).toEqual(60);
expect(result).toEqual(26.666666666666668);
});

it("should return the percentage of discarded records", () => {
const metrics = new Metrics(5, 4, 3, 1, 1);
const metrics = new Metrics(15, 4, 5, 1, 5);

const result = metrics.percentage.discarded;

expect(result).toEqual(20);
expect(result).toEqual(33.333333333333336);
});
});
});
16 changes: 4 additions & 12 deletions argilla-frontend/v1/domain/entities/dataset/Metrics.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ export class Metrics {
};

constructor(
private readonly records: number,
public readonly responses: number,
public readonly total: number,
public readonly submitted: number,
public readonly discarded: number,
public readonly draft: number
public readonly draft: number,
public readonly pending: number
) {
this.percentage = {
pending: (this.pending * 100) / this.total,
Expand All @@ -22,21 +22,13 @@ export class Metrics {
}

get hasMetrics() {
return this.records > 0;
}

get total() {
return this.records;
return this.total > 0;
}

get responded() {
return this.submitted + this.discarded + this.draft;
}

get pending() {
return this.total - this.responded;
}

get progress() {
return this.responded / this.total;
}
Expand Down
4 changes: 1 addition & 3 deletions argilla-frontend/v1/domain/entities/dataset/Progress.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
export class Progress {
constructor(
public readonly total: number,
public readonly submitted: number,
public readonly discarded: number,
public readonly conflicting: number,
public readonly completed: number,
public readonly pending: number
) {}
}
Original file line number Diff line number Diff line change
Expand Up @@ -107,13 +107,7 @@ export class DatasetRepository implements IDatasetRepository {
largeCache()
);

return new Progress(
data.total,
data.submitted,
data.discarded,
data.conflicting,
data.pending
);
return new Progress(data.total, data.completed, data.pending);
} catch (err) {
throw {
response: DATASET_API_ERRORS.ERROR_DELETING_DATASET,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,12 @@ import { largeCache } from "./AxiosCache";
import { Metrics } from "~/v1/domain/entities/dataset/Metrics";

interface BackendMetrics {
records: {
count: number;
};
responses: {
count: number;
total: number;
submitted: number;
discarded: number;
draft: number;
pending: number;
};
}

Expand All @@ -25,11 +23,11 @@ export class MetricsRepository {
);

return new Metrics(
data.records.count,
data.responses.count,
data.responses.total,
data.responses.submitted,
data.responses.discarded,
data.responses.draft
data.responses.draft,
data.responses.pending
);
} catch {
/* lint:disable:no-empty */
Expand Down
58 changes: 25 additions & 33 deletions argilla-frontend/v1/infrastructure/repositories/RecordRepository.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,7 @@ export class RecordRepository {
constructor(private readonly axios: NuxtAxiosInstance) {}

getRecords(criteria: RecordCriteria): Promise<BackendRecords> {
if (criteria.isFilteringByAdvanceSearch)
return this.getRecordsByAdvanceSearch(criteria);

return this.getRecordsByDatasetId(criteria);
return this.getRecordsByAdvanceSearch(criteria);
}

async getRecord(recordId: string): Promise<BackendRecord> {
Expand Down Expand Up @@ -188,35 +185,6 @@ export class RecordRepository {
}
}

private async getRecordsByDatasetId(
criteria: RecordCriteria
): Promise<BackendRecords> {
const { datasetId, status, page } = criteria;
const { from, many } = page.server;
try {
const url = `/v1/me/datasets/${datasetId}/records`;

const params = this.createParams(from, many, status);

const { data } = await this.axios.get<ResponseWithTotal<BackendRecord[]>>(
url,
{
params,
}
);
const { items: records, total } = data;

return {
records,
total,
};
} catch (err) {
throw {
response: RECORD_API_ERRORS.ERROR_FETCHING_RECORDS,
};
}
}

private async getRecordsByAdvanceSearch(
criteria: RecordCriteria
): Promise<BackendRecords> {
Expand Down Expand Up @@ -264,6 +232,30 @@ export class RecordRepository {
};
}

body.filters = {
and: [
{
type: "terms",
scope: {
entity: "response",
property: "status",
},
values: [status],
},
],
};

if (status === "pending") {
body.filters.and.push({
type: "terms",
scope: {
entity: "record",
property: "status",
},
values: ["pending"],
});
}

if (
isFilteringByMetadata ||
isFilteringByResponse ||
Expand Down
4 changes: 1 addition & 3 deletions argilla-frontend/v1/infrastructure/types/dataset.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,6 @@ export interface BackendDatasetFeedbackTaskResponse {

export interface BackendProgress {
total: number;
submitted: number;
discarded: number;
conflicting: number;
completed: number;
pending: number;
}
15 changes: 11 additions & 4 deletions argilla-server/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,22 @@ These are the section headers that we use:

## [Unreleased]()

### Fixed

- Fixed SQLite connection settings not working correctly due to a outdated conditional. ([#5149](https://github.com/argilla-io/argilla/pull/5149))
### Added

## [2.0.0rc1](https://github.com/argilla-io/argilla/compare/v1.29.0...v2.0.0rc1)
- Added support to specify `distribution` attribute when creating a dataset. ([#5013](https://github.com/argilla-io/argilla/pull/5013))
- Added support to change `distribution` attribute when updating a dataset. ([#5028](https://github.com/argilla-io/argilla/pull/5028))

### Changed

- Change `responses` table to delete rows on cascade when a user is deleted. ([#5126](https://github.com/argilla-io/argilla/pull/5126))
- [breaking] Change `GET /datasets/:dataset_id/progress` endpoint to support new dataset distribution task. ([#5140](https://github.com/argilla-io/argilla/pull/5140))
- [breaking] Change `GET /me/datasets/:dataset_id/metrics` endpoint to support new dataset distribution task. ([#5140](https://github.com/argilla-io/argilla/pull/5140))

### Fixed

- Fixed SQLite connection settings not working correctly due to a outdated conditional. ([#5149](https://github.com/argilla-io/argilla/pull/5149))

## [2.0.0rc1](https://github.com/argilla-io/argilla/compare/v1.29.0...v2.0.0rc1)

### Removed

Expand Down
Loading
Loading