Add management command to run all necessary tasks after a release.

[brianjp93]

• fixes Add periodic tasks for updating release report data #1519
• add button to version admin to execute task
• use @ locked decorator to ensure only one instance of the task is running at any time.
• If a user_id is given to the command, send that user email notifications when it starts and ends.

[sdarwin]

During a release, the script publish_release.py in the boostorg/release-tools repository will be hitting a REST endpoint to import new versions. admin/versions/version/new_versions/

Now, two buttons and two REST endpoints...

To consider - should these new steps be automatically triggered at the end of the new_versions import process. Or add a third button and endpoint which will consolidate all the tasks.

[GregKaleka]

How long is this taking locally? I wonder if using Celery's Canvas to create an actual task chain would be wiser. Maybe a future optimization if this is performing reasonably.

[brianjp93]

@sdarwin I wasn't aware of that script. I think it makes sense if that script hits a new endpoint which consolidates required tasks post-release. This PR already makes an endpoint available at /admin/versions/version/post_release_tasks/. Perhaps we can hit that instead?

[brianjp93]

@GregKaleka Locally, it took 14 minutes but definitely has the potential to take longer. I think that the piece with the worst potential is the fetching of slack messages, but they are also being fetched intermittently so it may not be so bad.

Would a task chain be helpful? Maybe some pieces could be run in parallel but that would make tracking the overall task's progression more difficult.

[sdarwin]

@brianjp93 , what are the dependencies between these? what is the timing between them

/admin/versions/version/new_versions/
/admin/versions/version/post_release_tasks/

Just exploring the issue. some ideas. not certain.

• Both scripts need to run
• Whether triggered manually or from publish_release.py, it would be convenient to press one button and both of those are taken care of.
• post_release_tasks should wait until new_versions completed successful
• post_release_tasks should not be executed if new_versions failed
• post_release_tasks should not be executed if new_versions imported a beta, rather than a standard release
• post_release_tasks should not be executed twice? Or maybe that is fine. ( if it already ran previously, for whatever the current boost release is)

For example, what if the endpoint post_release_tasks called new_versions as the first step. Or new_versions called post_release_tasks as the final step. Conditionally. Or, a third endpoint calls them both, in order.

[brianjp93]

@sdarwin This new task is meant to be an all encompassing group of tasks to be run at release time.
So currently, the post_release_tasks task is executing the new_versions task at the beginning of its execution already.
https://github.com/boostorg/website-v2/pull/1562/files#diff-52be108087cf158d973a355981ee8780c899de3d35516ab91780d2bd7886c76cR34-R42

• post_release_tasks is meant to be idempotent, it should be fine to run twice although protections are in place to ensure only one instance is running simultaneously.

[sdarwin]

Ha! Then nevermind!

I saw the name "post_release_tasks", and the description "after a release", and assumed these were later tasks AFTER the release was already done. (slack reports, mailing list statistics, etc).

In fact, import_versions is an important part of the release process itself.

The release is not live until the website displays it.

So the script endpoint will be the post_release_tasks, replacing new_versions.

It would not be so far-fetched to name this release_tasks instead of post_release_tasks.

[brianjp93]

Ah I see, my wording is wrong then. Sorry for the confusion, I will update the naming.

[sdarwin]

No problem, you can make a case for both names, they are both reasonable, in a way.

[sdarwin]

Before, publish_release.py would hit this api endpoint regardless of beta or release:

/admin/versions/version/new_versions/

And now?

Still /admin/versions/version/new_versions/ for betas,

and /admin/versions/version/release_tasks/ for full releases?

What happens if you run /release_tasks/ at a random time, in-between release cycles, or after a beta ?

---

Edit: I see you wrote "meant to be idempotent". So, nothing new will happen, I guess.

[brianjp93]

@sdarwin
I believe that some of the tasks in release_tasks are valuable to both the beta and full releases. For example, update_library_versions, update_maintainers, update_authors.

If these tasks are necessary for the beta as well as full releases, it may make sense to use release_tasks in both cases. If not, new_versions can be hit for betas and release_tasks for full releases.

Maybe @rbbeeston has a better idea about if this new task should run after beta releases as well as full releases?

[rbbeeston]

We won't be running reports on beta releases. Typically we would run a draft report, check for issues, and rerun until there is one ready to make public. So I don't see a clear need to do anything special for them.

[rbbeeston]

I'm goos with this when Greg approves

[GregKaleka]

Looks good! One very minor cleanup suggestion, but approved.