Proposal for a patch release of OSCAL 1.1.3 to fix constraint bugs and improve documentation

[aj-stein-gsa]

Hello @iMichaela and others, I hope all is well.

My colleagues and I am in FedRAMP are in the process of updating our constraints and noticed a collection of issues that indirectly or directly "block" the ability to fully validate FedRAMP SSPs and other digital authorization package documents as there are overly restrictive constraints, inaccurate documentation, or other related issues. We have proposed the following fixes to the main branch, as they can be merged into the release-1.1 branch and tagged for a possible v1.1.3 branch accordingly. Below is a list of such issues.

• Remove redundant definition of location-uuid #2058
• Relax component protocol constraint for #1913 #2063
• Removed incorrect enforcement of asset-id property at implemented-com… #2064
• Added missing field values in Metaschema modules #2066
• Correct digest lengths #2068
• Do not require a protocol/@name for #1772 #2069
• Clarify protocol port-range docs for #2065 #2070
• Cherry-pick #2024 protocol port-range constraints hotfix into main for future release #2071
• hotfix invalid syntax #2078 (NOTE: Updated because this is a low-risk fix to a recent PR's syntax error from an unrelated PR in develop separate of this proposal, I and the FedRAMP Automation Team support its inclusion.)

These changes are targeted to relax (by changing or removing certain constraints) in a way that does not break backwards compatibility. Therefore, we propose a patch release, a la NIST OSCAL Team's semantic versioning guidance and release process in this project's wiki.

To that end, I wanted to ask if we can have coordinate with you to prepare and publish a 1.1.3 release? I want to propose this release and poll you and the community to gauge interest in our coordination to complete this work.

Sincerely,
A.J. on behalf of the FedRAMP Automation Team

/cc @brian-ruf @david-waltermire @RS-Credentive (as they opened several of the issues and PRs declared above)

[RS-Credentive]

I don't see a downside to any of these, and I think this is a very modest but beneficial set of changes.

[aj-stein-gsa]

The NIST Team asked why a patch release targeting main and not develop in a specific PR review. That seems pertinent to the general GitHub discussion for the proposal here. I will post this reference here for others who wanted to know more as I have been told I am spamming the [email protected] list inappropriately.

#2071 (comment)

[iMichaela]

Also the bug #2059 must be addressed in the v1.1.3
As mentioned in several other comments, the develop branch was staged with only bugs and dependency updates that need to go into 1.1.3 All not-yet merged PRs against the develop branch can wait for later, unless there is an appetite for including the CVSS v4 support. The CVSS missing fields is a bug that must be fixed in 1.1.3.
I recommend making the PRs against develop and I can use develop to create a PR against release-1.1 for the v1.1.3 . Extra review can be performed then during approval of the PR.
NIST guidance indicate patches to be created through PRs against release-x.y (release-1.1 in this case ) and this is the plan with everything we have merged already in develop.

[aj-stein-gsa]

Also the bug #2059 must be addressed in the v1.1.3 As mentioned in several other comments, the develop branch was staged with only bugs and dependency updates that need to go into 1.1.3.

That is not in my proposal, are you asking us to weigh in and code the change because it must be included in this release, not a follow on 1.1.4 or 1.1.x release? Since you say must, it seems important for this proposal to be accepted? It increases the scope a litte.

Extra review can be performed then during approval of the PR. NIST guidance indicate patches to be created through PRs against release-x.y (release-1.1 in this case ) and this is the plan with everything we have merged already in develop.

As someone who edited and maintained that document, I will repeat it again: the point of making the PRs directly to the release branch is to avoid this situation where changes to targeted fixes for a patch release on a main/release-1.1 must include new work that is not slated for release, but I would propose separate of this thread adjusting the guidance to follow how you operate now and I will not follow the prior approach anymore.

[iMichaela]

targeted fixes for a patch release on a main/release-1.1 must include new work that is not slated for release

Slated by whom? NIST addressed bugs and slated them for the next patch release. I see no reason for 2 patch releases.

[aj-stein-gsa]

Slated by whom? NIST addressed bugs and slated them for the next patch release. I see no reason for 2 patch releases.

OK I am not going to keep circling around the topic and will just move on.

I see everything in my proposal, still with 80% support (albeit only four votes other than me), is now complete. All PRs, against my recommendation and documentation, were merged into develop as you had required for this proposal to move forward. Additionally, you have included #237 and #2067, which I had not initially scoped in.

@iMichaela , you had elsewhere discussed how you felt #2059 is important to fix. I or someone else could act on that recommendation, but that may take a little more testing and evaluation to ensure it works which I am not sure I can do so quickly. That is not a quick tactical fix and requires surfacing up constraints for one model to be effectively reused across models. I like to have patch releases be small in scope. It allows all implementers, not FedRAMP, move up or revert back to an older version. It is not for lack of caring or other concerns you asked about, just to make pulling in small-scoped fixes in with low risk. But you can let me know in follow-up to the questions and answers below if this is integral to making a release, as it was not part of my proposal.

So I have some questions.

1. Does the approval and merge mean that the NIST OSCAL Team is supportive of the proposal and will prepare for a 1.1.3 release? If the answer is "no" or "unsure, we're still undecided," that is also fine.
2. If the answer to 1 is yes, and SAP constraints conflicting in the local-definition/objects-and-methods #2059, not in the scope of my proposal, is work that must be coded, reviewed, and merged before NIST will initiate a release? If so, after merge into develop, what is an estimated timeline for release of 1.1.3 if this is a requirement?
3. If the answer to 1 is yes, but SAP constraints conflicting in the local-definition/objects-and-methods #2059 is not required for NIST to commit to a release, what is an estimated timeline for release of 1.1.3?

[iMichaela]

Also #2037 is long overdue and was raised as a bug since the Metaschema was updated but OSCAL metaschema definition files were not. I am planning on including it into 1.1.3 patch unless there is a good reason not to.

[iMichaela]

I will focus next of #2059 since it implies a simple removal of a constraint (or two), and few of the dependabot PRs to include in the v1.1.3 . Everything else is ready. Is there anything else needed now? I heard zero community voices other than GSA & NIST. I will need someone to review the #2059 as soon as I submit the PR. I am planning a patch release no later than Monday, Nov 18. Theoretically I am on PTO today, but if I manage to address #2059 today based on the comments under the issue, then I'll do my best to factor into today's schedule.

[aj-stein-gsa]

Hi @iMichaela thanks for your prompt responses to issues and preparation for a 1.1.3 release. It seems there are some concerns over priorities and miscommunication re your interpretation in #2076 (comment) of what I asked for #2072 (reply in thread).

I know we all have different priorities, my goal was not to commit you or the team to an immediate deadline, just to ask if you had an informed opinion on the feasability of a release and a tentative estimated timeline. It seems you believe a PR to fix #2059 to be integral to the 1.1.3 release separate of my proposal. If I have misunderstood, apologies. Whether or not it is integral to fix #2059, do you want to delay a tentative release date for 1.1.3 and adjust accordingly? I am not here to dictate, I am here to discuss. Thanks.

[iMichaela]

Hi @aj-stein-gsa - I am 100% supportive of a patch release (1.1.3) and it was on my ToDo with high priority and #2059 was a bug I perceived more important to FedRAMP than other entity. It was actually brought forward by our common friend John ... and he did it probably more than a year ago. Our (NIST) earlier examples did not exercise local-definitions so we were not aware of the bug until John brought it forward. Since many GRC tools, especially the JSON based ones, are not even implementing the constraints, the bug stayed undiscovered and with only few complains from the community.
How is FedRAMP handling the bug if we do not fix it now when FedRAMP is working on the assessment layer.
I /we at NIST what to be supportive and fair to all in the same time. Not an easy task. Can I/we NIST live with the bug longer - yes. Can FedRAMP live with it? For how long. Again - I am all for working together to get it right. The timelines are flexible more at our end (NIST) than FedRAMP.
So, if:

1. we do not include the hot fix, we can release 1.1.3 tomorrow. But I want to make sure FedRAMP is aware of it because the bug affects FedRAMP assessment with certainty.
2. we just remove the constraints from core OSCAL and FedRAMP needs them, it can enforce them as desired, as part of the FedRAMP external constraints set, then we can release tomorrow .
3. we fix the bug by updating the constraints, then tests might be faster and the 1.1.3 release will be delayed by 1-2 days, because the constraints, per my observation do not generate invalid catalogs or profiles.
4. we fix the bug by changing the constraints AND their location (the file where they are defined) - then thorough testing is necessary.

So what do you think we should do, based on FedRAMP's need of moving fast forward?

PS NOTE: I still have unprocessed comments under the PR, so I might have missed something important and pertaining to this discussion. Did not have more cycle today...

[aj-stein-gsa]

Hi @iMichaela I apologize for the delayed response.

I took the time to discuss with the team and FR leadership. At this time, it would be ideal if we can advance with a 1.1.3 release with all PRs reviewed and accepted in develop. All others we can coordinate with you on a successive minor or patch release.

We know the other issues may take time to validate and assess and we are more than happy to coordinate for future 1.1.x and 2.x.y releases accordingly once we all have taken the time to review and validate with community feedback.

Is that approach agreeable?

[aj-stein-gsa]

And sorry to be clear that means option 1. Apologies.

[aj-stein-gsa]

@iMichaela so are you amenable to the proposed plan in my update last night? I can work with you and the team for subsequent leases after these are released, but wanted to know if we can square this away and work towards a release.

(NOTE: My colleagues will be around next week, but just a FYI I will be around tomorrow and then out on leave until I return to work on Friday 29 November 2024, so I am giving you a heads up if I stop responding after a certain time tomorrow, the 22nd.)

[iMichaela]

@aj-stein-gsa - Sorry - I was swamped and did not get to your message (better say - to the GitHub's email telling me there are msgs for me) until today. I can release OSCAL 1.1.3 which I've done before, no problem. I'll do it on Monday since the week ran away already. We will also have to release, for the community and for our CI/CD pipeline, the corresponding oscal-cli, which I can try doing, but haven't done it , and I might have questions. Whom should I ping if I have oscal-cli release questions since you were the last developer/maintainer of that repo?

[aj-stein-gsa]

Sounds good and thanks for the update.

[aj-stein-gsa]

Thanks for the release, it seems now is the time to close out this discussion post and its threads with 1.1.3 released. 😄