NIST Team: If you could make one big change, what is it? Why?

[aj-stein-nist]

Hello NIST Team,

As discussed during this week's sprint planning, I want to give staff on the NIST OSCAL Team a space to recommend a significant change to the technical architecture of OSCAL, be it the models, documentation generation/management, the supporting infrastructure, or anything in between. In this case, I want to emphasize signficant: if it is a small tactical change and the required work can easily fit into one or several small issues that can be addressed in one or a few sprints, it is too small in scope and not a long-term change that needs discussion. (Open those issues, but this is more ambitious!)

So before the end of Sprint 67 (ending on 1 May 2023), if you are a NIST OSCAL Team member, I want to hear from you. If you have a proposal, please enter it as a comment here to explain the change and the impact you believe it will have. We will review them as a team in May, and maybe one or more of the proposals will move forward.

Thanks,
A.J.

(NOTE: It is also ok you do not have a proposal, it is not mandatory, but I welcome one from each team member here. If you are uncomfortable posting it here, please reach out to me via Gitter or email.)

[Compton-US]

I would like to see a StructureDefinition model (similar to FHIR), or similar mechanism, to detail how OSCAL is implemented, any customizations, and whether any requirements differ from baseline OSCAL.

For example (but not limited to):

• I could outline the models supported in my implementation. (e.g., Catalog and Profile only)
• I could detail each assembly that is used beyond the baseline cardinalities of OSCAL (e.g., The SSP must have responsible parties).
• I could outline any props that are implemented, the reason, and whether each one is required or desired. (e.g., FDA Classification)
• I could provide context-specific detail/documentation important to my implementation.

This model could be used as a validation when model content is submitted to my organization. It could also allow me to query a system prior to preparing a document to determine requirements. For an automated system, in an interoperability environment, systems could query each other for requirements, and deliver the requirements as requested by the StructureDefinition of the receiving system. Receiving systems could immediately validate conformance, and provide specific, machine interpretable errors.

This document could also be used as the basis of a human-readable implementation guide (also something I've used in FHIR), for implementers to understand the requirements of an organization, for example FedRAMP submission, or even within a large organization to understand reporting requirements.

This could also benefit vendors by allowing products to deliver (or load) a StructureDefinition as a statement of capabilities so that other systems can adapt appropriately or warn the owner of incompatibilities in model content. (For example, a product that does not (yet) support profile resolution, but it is required.)

[nikitawootten-nist]

We could improve the onboarding experience, encourage community support, contributions, and overall efficiency of the OSCAL team by making adjustments to our repository structure and practices.

In particular, I see the following as concerns:

• The build process has too many overlapping concerns, making it difficult to perform maintenance without significant discovery.
  • How do I test if X is working properly?
  • What else will change if I touch Y?
  • Where can I find more documentation about Z?
• There is much confusion about how some pieces of the repository are generated (model documentation, schemas), leading to significant churn in order to generate releases.
  • Pull requests often include accompanying "Skip CI" PRs just to simulate what the side effects of pushing would be.
  • Even pre-releases have significant anxiety associated with them.

I would like to propose a few key changes to the repository:

• All generated content is kept out of tree and replaced with release artifacts as applicable:
  • Schemas are automatically generated for a tagged release and as part of a PR.
  • Model documentation is generated as part of the hugo website build.
    • Instead of dragging the content of previous releases along, the action could fetch the definitions from previous tags and regenerate as needed.
    • Caching may be required to prevent "rebuilding the world", though schema generation can be sped up significantly.
• There is hard separation between the "pieces" of the OSCAL repository:
  • .github/, decisions/: Remain unchanged in purpose, though Github actions workflows will change.
  • src/: Contains the Metaschema definitions for OSCAL models and any supporting documentation including formal specifications.
  • website/: Contains the website and supporting build scripts (which in turn rely on the xslt/ directory).
  • xslt/: Contains the "XSLT implementation" of OSCAL as well as documentation, driver scripts and the submodule to the upstream Metaschema repository.
  • This would prevent problems such as:
    • A build/ directory floating with too many individual concerns
• Each "piece" of the OSCAL repository should be individually runnable and testable using standard tooling (Makefiles, Dockerfiles, etc)
  • GitHub actions workflows should for the most part be calling these build targets without any additional complexity (exceptions being link checking with GHA, etc.)
  • As an example:
    • The src/ repository could have a Makefile target to generate schemas off of the OSCAL metaschema (which would in turn call the generate schema driver script in the xslt/ directory)
  • Standard targets ("build", "test") should be used to make it easy for community members to use the repository.
• Historical artifacts (content/, src/content/, src/util/utils) should be removed

These suggested changes are large and time consuming, however they can be completed incrementally. Implementors could for example focus on overhauling the generated content first, before disentangling build/, src/, and docs/ into the new src/, xslt/ and website/ directories.

[iMichaela]

I believe that an OSCAL Metric Definition Model that allows for definitions of:

• Rule(s),
• Parameters
• Test
• Results Assertions
  is needed. It will incorporate the "Rules" and "Tests" work we initiated and be used similar to the Component Definition Model to generate Metric Definitions for controls, components or capabilities. A repository of Metric Definitions can be envisioned as metrics can be reused by Catalog, Profile, Component Definiton authors, System Owners or Assessors.