[ENHANCEMENT / BUGFIX] argilla: manage responses for deleted users

[frascuchon]

Description

This PR adds support to SDK to work with responses from deleted users. In those cases, the user_id is None. The current SDK will set the DELETED_USER.id value, which can be used to identify responses from deleted users.

Notes:

Even if we're defining a DELETE_USER constant, the responses are using the user_id. So, not sure if this user constant makes more sense if we use user instances on responses instead of their IDs.

Type of change

(Please delete options that are not relevant. Remember to title the PR according to the type of change)

• New feature (non-breaking change which adds functionality)
• Refactor (change restructuring the codebase without changing functionality)
• Improvement (change adding some improvement to an existing functionality)

How Has This Been Tested

(Please describe the tests that you ran to verify your changes. And ideally, reference tests)

• Test A
• Test B

Checklist

• I added relevant documentation
• I followed the style guidelines of this project
• I did a self-review of my code
• I made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I filled out the contributor form (see text above)
• I have added relevant notes to the CHANGELOG.md file (See https://keepachangelog.com/)

[burtenshaw]

One high level question: how are we dealing wiht the situation where a record contains responses from multiple deleted users.

For example, when we try to export them here, they will be combined into one.

[frascuchon]

One high level question: how are we dealing wiht the situation where a record contains responses from multiple deleted users.

For example, when we try to export them here, they will be combined into one.

Thanks @burtenshaw. Yes, this solution may not work for those cases. I'm thinking more about the export format and I think we should store data in a more lightweight, maybe we don't need to store responses with users if the SDK has mechanisms to assign responses to users in an easier way. This can also help to create "portables" dataset snapshots (moving between argilla servers)

[burtenshaw]

One high level question: how are we dealing wiht the situation where a record contains responses from multiple deleted users.
For example, when we try to export them here, they will be combined into one.

Thanks @burtenshaw. Yes, this solution may not work for those cases. I'm thinking more about the export format and I think we should store data in a more lightweight, maybe we don't need to store responses with users if the SDK has mechanisms to assign responses to users in an easier way. This can also help to create "portables" dataset snapshots (moving between argilla servers)

Ok. It makes sense to move that to a new PR on export mapping, where we also deal with this.

How should we solve this finish PR? One thing I thought of was switching from a nested dictionaries to arrays so that we export:

{
    "responses": [ 0,1,1 ],
    "response_user_ids": [ "uuid","uuid","uuid"],
    "...":"..."
}

That way, in this situation the response_user_ids could be non-unique and response values are usable.

[frascuchon]

After changes included in:

• [BUGFIX] server: Skip responses without user ids when indexing #5093
• [BUGFIX] Skip responses with deleted users when log records #5095

Maybe the simplest solution would be to delete responses on cascade until we decide the approach when deleting users. cc @burtenshaw @jfcalvo

[jfcalvo]

Maybe the simplest solution would be to delete responses on cascade until we decide the approach when deleting users.

@frascuchon I agree with that.

[burtenshaw]

Maybe the simplest solution would be to delete responses on cascade until we decide the approach when deleting users.

@frascuchon It's not ideal but I can see why it makes sense. What's the status of this PR?

[frascuchon]

Finally, this PR won't apply. See #5126