-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathe4s-policies-1.0.yaml
113 lines (110 loc) · 9.8 KB
/
e4s-policies-1.0.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
e4s_policy_version: 1.0
e4s_policy_format_version: 0.01
policy_groups:
- Official:
title: "The policies below are criteria for E4S membership"
preamble: >
To qualify for E4S
membership, a package must demonstrate compatibility with each of these
policies. Under special circumstances, a package may be granted an
exception to a policy.
Tag: P
policies:
- id: 1
name: Spack-based Build and Installation
description: >
Each E4S member package supports a scriptable Spack build and
production-quality installation in a way that is compatible with
other E4S member packages in the same environment. When E4S build,
test, or installation issues arise, there is an expectation that
teams will collaboratively resolve those issues.
- id: 2
name: Minimal Validation
description: >
Testing Each E4S member package has at least one test that is executable through the E4S validation test suite (https://github.com/E4S-Project/testsuite). This will be a post-installation test that validates the usability of the package. The E4S validation test suite provides basic confidence that a user can compile, install and run every E4S member package. The E4S team can actively participate in the addition of new packages to the suite upon request.
- id: 3
name: Sustainability
description: >
All E4S compatibility changes will be sustainable in that the changes go into the regular development and release versions of the package and should not be in a private release/branch that is provided only for E4S releases.
- id: 4
name: Documentation
description: >
Each E4S member package should have sufficient documentation to support installation and use.
- id: 5
name: Product Metadata
description: >
Each E4S member package team will provide key product information via metadata that is organized in the [E4S DocPortal](https://e4s-project.github.io/DocPortal.html) format. Depending on the filenames where the metadata is located, this may require [minimal setup](https://github.com/E4S-Project/E4S-Documenter/blob/master/README.md).
- id: 6
name: Public Repository
description: >
Each E4S member package will have a public repository, for example at GitHub or Bitbucket, where the development version of the package is available and pull requests can be submitted.
- id: 7
name: Imported Software
description: >
If an E4S member package imports software that is externally developed and maintained, then it must allow installing, building, and linking against a functionally equivalent outside copy of that software. Acceptable ways to accomplish this include (1) forsaking the internal copied version and using an externally-provided implementation or (2) changing the file names and namespaces of all global symbols to allow the internal copy and the external copy to coexist in the same downstream libraries and programs. This pertains primarily to third party support libraries and does not apply to key components of the package that may be independent packages but are also integral components to the package itself.
- id: 8
name: Error Handling
description: >
Each E4S member package will adopt and document a consistent system for signifying error conditions as appropriate for the language and application. For e.g., returning an error condition or throwing an exception. In the case of a command line tool, it should return a sensible exit status on success/failure, so the package can be safely run from within a script.
- id: 9
name: Test Suite
description: >
Each E4S member package will provide a test suite that does not require special system privileges or the purchase of commercial software. This test suite should grow in its comprehensiveness over time. That is, new and modified features should be included in the suite.
- Future:
title: "Future Revision E4S Community Policies"
preamble: >
The policies below are being considered for future revisions of the E4S
Community Policies. These policies require further refinement or
planning prior to adoption. The topics that these policies address
provide information about likely subject areas for E4S policies going
forward.
Note: FP stands for “future policy”"
Tag: FP
policies:
- id: 1
name: Portability
description: >
Each E4S member package team will make a best effort at portability to common platforms. Depending on the function of the member package, considerations may include operating systems, compiler toolchains, architectures and accelerators. Lack of support for configurations should be denoted in appropriate Spack packages when possible.
including standard Linux distributions, common compiler toolchains, and different architectures and accelerators. Consider: self assessed portability score/metric - OS, GPU, etc?
- id: 2
name: Flexible Test Selection Support
description: >
Each E4S member package will support the ability to specify that specific tests will be run for a given system allocation limit on execution time and node configuration resources. Particular use cases include the ability to a) run a subset of the test suite to complete within a few hours on standard workstation-level hardware b) be runnable in batch-only environments, that is, systems that require the use of PBS or other submission scripts.
- id: 3
name: Dependency Version Tracking
description: >
Each E4S member package will document the versions of packages with which it can work or upon which it depends, in Spack. Similarly, incompatible versions of packages should also be appropriately tracked in Spack. The developers of E4S member packages will coordinate the needed versions of various packages for each E4S release.
- id: 4
name: Memory Testing
description: >
Each E4S member package makes it possible to run their test suite under Valgrind in order to test for memory corruption issues. NOTE: The SDK team is planning to refine this policy so as to not specifically single out Valgrind.
- id: 5
name: Comprehensive Test Suite
description: >
Each E4S member package will provide a comprehensive test suite that can be run by users and does not require special system privileges or the purchase of commercial software.
- id: 6
name: Documentation
description: >
Each E4S member package should have sufficient documentation to support installation, use, and further development, as assessed periodically by its community of peers.
- FutureLibrary:
title: "Library Policies: Apply to linkable libraries and similar products"
preamble: >
Note: FLP stands for “future library policy”"
Tag: FLP
policies:
- id: 1
name: User-managed Exception Handling
description: >
Each E4S member library package will have no hardwired calls to abort, exit, or MPI_Abort(). Also, no package should have hardwired print statements for error conditions. Each package should document which error codes/exceptions are recoverable, which may result in lost resources (for example, unfreed memory), and which indicate that the process may be in an undefined or totally broken state (for example, after a segmentation violation). It is the responsibility of the calling routine not to simply continue the computation when a “hard” error is returned; the calling routine will likely, by default, call an abort, but again that should be possible to override.
- id: 2
name: User-managed I/O Control
description: >
No package should have hardwired print or I/O statements that cannot be turned off through a programmatic interface; output should never be hard-wired to stdout or stderr. It is recommended that packages provide a way for users to turn on output and allow them to direct where it goes. Also, packages may print to stdout by default but only on one process (i.e., root rank “0”). But packages may also be completely silent by default (and require that users turn on outputting in the appropriate way).
- id: 3
name: Symbols and Namespacing
description: >
Each E4S member package will use a limited and well-defined symbol, macro, library, and include file name space. For example, there should be no publicly visible include files such as utils.h, or packages named libutil.a or macros named YES or TRUE. Namespacing of include files can be handled either by prepending each include file with a package name, for example <XXXmat.h>, or by placing and referencing all include files in a subdirectory with a package name, for example <XXX/mat.h>. Note that using a -I/XXX/ and referencing via <mat.h> would not be acceptable namespacing.
- id: 4
name: Memory Leaks
description: >
Each E4S member package will free all system resources it has acquired as soon as they are no longer needed, including closing open files, freeing memory, freeing MPI communicators, and freeing MPI data types created by the package. In particular, it is important that E4S compatible code not have any growing memory leaks (such as leaking memory during every timestep). Any resources created by the package that should be freed by the user, rather than by the package, must be fully documented. Note: Exceptions are permitted for situations where other software quality considerations are more important.