OWASP recognizes the need for project consumers to quickly ascertain the maturity of a project. Project reviews are not mandatory, but they are necessary if a Project Leader wishes to graduate to the next level of maturity within the OWASP Projects Infrastructure. Projects can be reviewed when an Incubator Project is ready to graduate into the OWASP Lab designation, and project releases can be reviewed if the Project Leader wants the quality of their product to be vouched for by OWASP. The goal of a review is to establish a minimal baseline of project characteristics and product quality.
The Reviewer Pool is made up of a group of individuals that aim to ensure there are qualified reviewers making quality reviews of OWASP Projects.The Reviewer Pool is made up of individuals hand selected by our OWASP Technical Project Advisors. These individuals have proven themselves dedicated to executing quality reviews of projects. Starting in 2014, members of the Reviewer Pool will be asked to fill in their user profile, which will be visible to all OWASP consumers, as a testament to why their reviews have merit and relevance. Members of the Reviewer Pool serve a critical role in ensuring the quality of projects, and should gain added recognition in OWASP.
Project Reviews provide a way to look comprehensively at the overall maturity of a project. Additionally, there is significant value in allowing projects to solicit general feedback to improve the quality of their projects. There are two types of reviews OWASP can provide for Project Leaders: Project Graduation Reviews and Project Feedback Reviews.
PROJECT GRADUATION REVIEWS Project Leaders can submit an application for a project review to assess whether they can graduate to the next stage in the OWASP Project Lifecycle. These reviews are conducted in the same way the Feedback Reviews are. The only difference is that the project might be able to graduate to the next stage if their project assessment is positive.
PROJECT FEEDBACK REVIEWS Project Leaders can submit an application for a project review to assess the quality of their project, and to get general professional feedback from the OWASP Community. Reviews of this type can only be done every six (6) months due to the high number of projects in our inventory.
There are several assessments our reviewers use to determine the quality, health, and usability of a project. These assessments were developed over a 6 month period of time by our OWASP Technical Project Advisory Team. The Technical Project Advisors were recruited as volunteers to help the organization review and update the assessment criteria and project graduation process. After months of testing different assessment criteria and processes, the advisors determined that projects need to be assessed in three primary areas by both a dedicated reviewer and the community at large. Below you will find the three assessments our reviewers use to determine the quality, health, and usability of a project. When reviewing a project, all of the assessments must be used to determine a more accurate and well rounded picture of the current state of the project.
PROJECT HEALTH ASSESSMENT Project health reviews are not mandatory, but they are necessary if a project wants to graduate to the next level of maturity. The Project Leader can submit an application for a project health review by submitting a request for review using the OWASP contact us form. After the application is received, the project will be assigned two (2) reviewers that will help assess the project. The review centers around the following core concepts:
- Project Maintenance: These questions assess whether the Project Leader is keeping his/her project materials
up-to-date.
- Quality Expectations: These questions assess whether the project product is of value to users and the
software security industry.
- Project Best Practices: These questions are meant to assess whether a project, and its Leader, are following
OWASP best practices.
These questions were designed to distill the core characteristics of a healthy OWASP Project, as any concern about a project’s quality can be aligned to one of the above questions. This assessment is qualitative in nature; therefore, the outcome is subjective to the unit of measurement used, the reviewer. This is why reviewer selection for each project is crucial.
PROJECT QUALITY ASSESSMENT The quality assessment is used to determine how good the product produced by the project is. It is meant to determine whether the product is easy to use, and whether users can find materials to help them use the product. Moreover, this assessment is meant to determine whether the product has continued support and whether the Project Leader is maintaining the product and improving upon it regularly. This assessment is conducted by the same two (2) reviewers that conduct the Project Health Assessment. Please note, this assessment is qualitative in nature, as well. Therefore, the outcome is subjective to the unit of measurement used, the reviewer.
PROJECT USABILITY AND VALUE ASSESSMENT The Project Usability and Value criteria were created during the 2013 Project Summit as a means of properly determining product value to users. The Project Health and Quality Assessments were created before the Summit. However, during the Summit, it was determined that there needed to be some way of measuring product value to users that was separate from the assessments the reviewers conducted. Only then could we have a well rounded picture of the current state of a project. This is where this particular assessment was developed and why. The Project Usability and Value Assessment is a survey aimed at capturing the value a product is providing to its users. It is based on the four (4) OpenSAMM business functions and twelve (12) Security Practices. The survey is to be sent out to product users, and the assessment will only be complete after at least 10 users finish the survey. After this assessment is complete, the (2) reviewers selected to assess the project will take the results, and use the feedback to complete the qualitative project quality and health assessments. The aim is to have quantifiable metrics that demonstrate usability and value based on actual user input, and a qualitative assessment of health and quality based on expert opinion.