Skip to main content

R&D Quality Criteria

The "R&D Quality Criteria" is a tool and checklist for Quality Control Managers (QCM), and is used to plan quality audits and assessments for project milestones and report quality status.

This Quality Checklist is a summary of ABB R&D milestone checklist items at each main project milestone:

  • M2 Quality Criteria
  • M3 Quality Criteria
  • M4 Quality Criteria
  • M5 Quality Criteria
  • M6 Quality Criteria

image.png

The quality criteria shall be checked by QCM before each required milestone. RO and CM support with the necessary information to help QCM perform the quality assessment.

The respective gaps and findings shall be analyzed and listed in a Quality Report by QCM and communicated to relevant stakeholders.

For release aspects (alpha, beta, etc.), see Release Stages Guide.

M2​

Development status​

  • System/Technology Requirements are approved.
  • System epics are approved (Including epic links and area path).
  • Epics are agreed upon (including description, acceptance criteria, high-level estimate, target date and area/iteration path).
  • A "baseline" of the agreed system epics, epics, and scope bugs is created (M2 agreement).
  • Implementation Proposals (if needed) are approved.
  • Safety: Product and Safety Requirement Specifications are approved.
  • Safety: Safety System Architecture, System FMEA, Preliminary Hazard Report, and Tool Selection Report are created and approved.
  • Safety: System Software Critically analysis and Impact Analysis Report are created (draft).

Test status​

  • The Product Test Strategy and Plan is approved.
  • The Product Test plan/Test suites high level plan created in ADO.
  • A method of ensuring planned requirements are developed and tested is established (method for requirements traceability).

Bug status​

  • A "baseline" of the agreed scope bugs exist (see development status).

Release status​

  • Project Description and Plan created and approved (including scope and capacity plan).
  • Quality Plan including KPIs is created and approved.
  • Dashboards are created.
  • Configuration Management Plan is created and approved.
  • Names on RACI is created and approved.
  • A Document Control Plan is created (draft)
  • User Document Control Plan is created (draft).
  • The Cyber Security preliminary assessment is done.
  • Already known new or modified 3rd party software is updated in DFN (draft).

M2 status​

  • M2 is planned and the R&D checklist is up to date.
  • All previous remarks resolved or accepted and resolution planned.

System status​

  • System Test Strategy and Plan is documented and agreed upon.
  • System Test plan/Test suites high level plan created in ADO.
  • System Integration Plan is approved, and milestones are derived.
  • A beta site delivery plan is created.

M3​

Development status​

  • Drafts of key technical documentation, like architecture, threat model, design and interface exist.
  • Change requests of system requirements completed for any changes in scope since M2.
  • Epics/feature progress is on target to be completed by the planned date.
  • DoD of closed epics/features has been applied.

Test status​

  • Product integration and test (PIT) is progressing and showing good quality of deliveries.

Bug status​

  • Bugs of severity critical and high are under control (a reasonable amount).
  • Scope bug progress is on target to be completed by the planned date.
  • Introduced bugs are being fixed as they are found, backlog is not continuously growing.

Release status​

  • Documentation is up to date.
  • Quality dashboard reviewed (KPI & Data quality).

M3 status​

  • M3 is planned and the R&D checklist is up to date.
  • All previous remarks resolved or accepted and resolution planned.

System status​

  • Change requests of system requirements completed for any changes in scope since G2.
  • System epics progress is on target to be completed by the planned date.
  • DoD of closed system epics has been applied.
  • System integration and test (SIT) is progressing and showing good quality of deliveries.

M4​

Development status​

  • All software and hardware functionality (including change requests and scope bugs) are implemented.
  • The design documentation is reviewed and approved. Consider software, hardware, firmware, cyber security, etc.
  • Safety: safety-related documents like architecture model, Hazop, and FMEA are updated and approved.

Test status​

  • Unit tests and component tests (e.g, MT/DT/FT/CT) are finalized. Related test descriptions are approved, tests executed and test reports are approved.
  • Product integration and test (PIT) completed.
  • Product type test (PTT) descriptions are reviewed and approved (ready for the start of PTT).
  • The product has achieved beta status. That is, the beta criteria are fulfilled, documented, and the decision meeting was held.
  • Pre-DSAC has been successfully completed, findings are documented in ADO.
  • Safety: SVT descriptions are reviewed and approved (ready for the start of formal tests).
  • Hardware: Environmental tests (ET) are executed. Related test descriptions are approved, tests executed, and test reports are available.

Bug status​

  • Bugs of severity critical are investigated and corrected.
  • Bugs of severity high have been investigated, analyzed, and selected for implementation or deferral.
  • The number of bugs that have not reached successful validation yet (closed state) is reasonable.
  • The number of critical and high-severity bugs reported by week is declining.

Release status​

  • User manuals have been updated (draft). User manuals with new functionality have been reviewed at least once.
  • A draft of the Release Note is available.
  • OSS Scan is done, reports are received and there are work items in place to mitigate eventual findings.
  • The Cyber Security assessment is done and reviewed with Cyber Security Team, and there are work items in place to mitigate eventual findings.
  • The list of 3rd party software in use is updated and approved in DFN.
  • Quality dashboard reviewed (KPI & Data quality).

M4 status​

  • M4 is planned and the R&D checklist is up to date.
  • All previous remarks resolved or accepted and resolution planned.

System status​

  • System Test plan/Test suites are reviewed and approved.
  • System test strategy and plan updated, reviewed, and approved.
  • System integration and test (SIT) completed.
  • The checklist for the start of STT is approved, and the meeting was held. Ready to start STT in all products and test benches.
  • User manuals have been updated (draft). User manuals with new functionality have been reviewed at least once.
  • The NLS project (if any) has passed M2.

M5​

Development status​

  • All planned work items (scope backlog, requirements, scope bugs, and change requests) are implemented and validated.
  • Safety: the impact analysis reports are reviewed and approved.

Test status​

  • PTT is finalized. Tests are executed and test reports are approved.
  • The requirements-test traceability is in place (as per the method defined in M2). Test coverage can be demonstrated.
  • DSAC has been successfully completed, findings are documented in ADO.
  • Safety: SVT is finalized. Tests are executed, test reports are approved, and CoT is done.

Bug status​

  • Bugs of severity critical are investigated, corrected, and validated.
  • Bugs of severity high, medium, and low have been investigated and are a) implemented and validated or b) deferred (dispositioned and justified by CCB).

Release Status​

  • All user manuals are reviewed and approved.
  • The Release Note is reviewed and approved.
  • The Cyber Security assessment is approved, including approval of all exceptions.
  • The OSS scanning is completed, and all exceptions are accepted.
  • The 3rd party software in use is approved, and all exceptions are accepted.
  • FCA and PCA are performed, and reports are approved.
  • Quality dashboard reviewed (KPI & Data quality).

M5 status​

  • M5 is planned and the R&D checklist is up to date.
  • All previous remarks resolved or accepted and resolution planned.

System status​

  • STT is finalized. Tests are executed and test reports are approved.
  • The RAT checklist is completed and approved.
  • RAT is finalized. Tests are executed and test reports are approved.
  • The IVA checklist is completed and approved.
  • The status of system requirements validation is clear (passed/failed/not done).
  • ST recommendations for the system for release are given. Recommendations and any remarks by STT are documented.

M6​

Development status​

  • NA

Test status​

  • NA

Bug status​

  • NA

Release status​

  • Final report is approved.
  • Document Control Plan up to date.
  • User Document Control Plan approved.
  • Stand-alone deliveries (if any are planned at M2) are finalized.
  • Quality dashboard reviewed (KPI & Data quality).

M6 status​

  • M6 is planned and the R&D checklist is up to date.
  • No old remarks remain.

System status​

  • The NLS project has passed M5.

References​

Owner: Quality and KPI Team