Skip to main content

What are you looking for?

Explore our services and discover how we can help you achieve your goals

Client Workspace

DevOps maturity assessment: a scorecard for delivery, reliability and security with evidence for every level

A DevOps maturity assessment is credible only when every level is backed by an artefact a reviewer can open. This Netbase scorecard rates fifteen practices across delivery, reliability and security on four levels, names the evidence each level needs, and caps the overall score at the weakest area so a fast pipeline cannot hide weak recovery or security.

Book a solution review See the related solution

By Netbase Commerce and Platform Engineering Desk · Reviewed by David (CEO) · Updated · 10 min read

star

This guide is for a CTO, head of engineering or founder whose product already runs in production and who wants to know what to fix first. It sits under cloud platform engineering, and the work it usually leads to is DevOps and platform engineering.

In this guide

Why do most DevOps maturity scores mislead?

Most maturity questionnaires ask a team to rate itself, and a team rates its intentions. "We do code review" scores the same whether every change is reviewed or only the risky ones. The fix is simple: a level counts only when the reviewer has seen the artefact for it, such as a pipeline run, a restore-test record or a scan report.

The second problem is averaging. A team with an excellent deployment pipeline and no tested backups averages out to "good", yet one bad incident shows that it is not. This scorecard therefore caps the overall level at the weakest area.

The third problem is scope. Many models score delivery speed and treat reliability and security as extras. The DORA metrics themselves pair throughput with instability measures, so speed alone is never the score. Here, delivery, reliability and security carry equal weight.

What do the four levels mean?

The levels describe how a practice is run, not how advanced the tools are.

  • Level 1, ad hoc: the practice depends on a person; nothing is written down or repeatable.
  • Level 2, repeatable: the practice is written down and followed, but run by hand.
  • Level 3, automated: the practice runs in tooling on every change or on a schedule, and leaves a record.
  • Level 4, measured: the record is reviewed on a fixed cadence, and the results change what the team does next.

A level with no artefact scores the level below. Level 4 is not a goal for every practice; a small product can stop at level 3 where the risk is low.

The scorecard: delivery practices

  • Version control and review

    Level 2 evidence
    Branch rules document
    Level 3 evidence
    Merge blocked without an approving review
    Level 4 evidence
    Review-time trend discussed in a retrospective
  • Build and test pipeline

    Level 2 evidence
    Written build steps
    Level 3 evidence
    Pipeline run log for each change
    Level 4 evidence
    Failed-build trend with actions taken
  • Deployment

    Level 2 evidence
    Release checklist
    Level 3 evidence
    Scripted deployment from the pipeline
    Level 4 evidence
    Deployment frequency and lead time reviewed monthly
  • Environments

    Level 2 evidence
    List of environments and owners
    Level 3 evidence
    Environments created from code
    Level 4 evidence
    Drift report checked on a schedule
  • Rollback

    Level 2 evidence
    Written rollback steps
    Level 3 evidence
    Rollback run in the pipeline in a test
    Level 4 evidence
    Rollback time recorded after real releases

The scorecard: reliability practices

  • Monitoring and alerting

    Level 2 evidence
    List of alerts and who receives them
    Level 3 evidence
    Alerts defined in code, linked to runbooks
    Level 4 evidence
    Alert noise review with alerts removed
  • Service level objectives

    Level 2 evidence
    Written target for the main user journey
    Level 3 evidence
    Indicator measured on a dashboard
    Level 4 evidence
    Error budget used in release planning
  • Backup and restore

    Level 2 evidence
    Backup schedule document
    Level 3 evidence
    Restore test log with date and result
    Level 4 evidence
    Restore tests repeated and timed on a cadence
  • Incident response

    Level 2 evidence
    Contact list and escalation steps
    Level 3 evidence
    Incident tickets with timeline
    Level 4 evidence
    Post-incident reviews with closed actions
  • Capacity and cost

    Level 2 evidence
    Known limits written down
    Level 3 evidence
    Usage and cost dashboards per service
    Level 4 evidence
    Capacity review feeding the roadmap

The service level objective row follows the Google SRE book definition: an indicator is a measure of the service, and an objective is a target for that measure. The incident row follows its postmortem guidance: written reviews with tracked actions. For the monitoring row, see what to instrument first.

The scorecard: security practices

  • Secrets

    Level 2 evidence
    Secrets kept out of the repository by rule
    Level 3 evidence
    Secrets in a vault, injected at runtime
    Level 4 evidence
    Rotation log and access review
  • Dependencies

    Level 2 evidence
    Dependency list per service
    Level 3 evidence
    Dependency scan report on each build
    Level 4 evidence
    Open findings tracked to closure dates
  • Code security

    Level 2 evidence
    Secure review checklist
    Level 3 evidence
    Static analysis results in the pipeline
    Level 4 evidence
    Finding trend reviewed with the team
  • Access control

    Level 2 evidence
    Role list for production
    Level 3 evidence
    Role-based access with MFA enforced
    Level 4 evidence
    Periodic access review record
  • Build integrity

    Level 2 evidence
    Build steps documented
    Level 3 evidence
    Build runs on a hosted platform with provenance
    Level 4 evidence
    Provenance verified before deployment

The build-integrity row borrows from the SLSA build levels, where a hosted platform signs the provenance and harder levels add isolation. The security rows map to the four practice groups of the NIST Secure Software Development Framework.

How do you turn fifteen scores into one level?

  1. Score each practice at the highest level whose evidence the reviewer has opened.
  2. Take the lowest practice score inside each area as that area's floor.
  3. Average the five practice scores per area, then cap the area at its floor plus one.
  4. Take the lowest of the three area scores as the overall level.
  5. List the three practices whose next level removes the cap, with an owner for each.

The cap is the point of the method. A level 4 pipeline feeding level 2 security mostly ships vulnerabilities faster, so the overall score says level 2 and the next actions are security work, not more pipeline work.

Running the assessment

  1. Agree the scope

    One product or one platform, its production environment and the team that runs it.

  2. Collect artefacts before the session

    Ask for links to pipeline runs, dashboards, restore logs and scan reports.

  3. Score together in one session

    The reviewer opens each artefact; anything not opened scores the level below.

  4. Record the score as data

    Keep the fifteen scores and their artefact links in a file that can be compared later.

  5. Pick three upgrades

    Choose the practices that lift the capped area, each with an owner and a review date.

  6. Re-score after the work ships

    Re-run the same scorecard, not a new one, so the trend means something.

Who should run the assessment?

Option Strength Risk Choose it when
Self-assessment by the team Fast, cheap, builds ownership Scores intentions unless a peer checks the evidence The team is stable and wants a baseline
Outside assessment Independent reading of the evidence Assessor may not know the product's constraints A board, buyer or investor needs a neutral view
Score from a vendor tool Quick and repeatable from connected systems Measures what the tool can see, misses manual practices Most practices already run in connected tooling

A mixed route works well: the team self-scores with artefact links, then an outside reviewer checks the links for the capped area only.

When should you not run a maturity assessment?

Skip it when the product has not found its market yet; the right practices then are a pipeline, backups and secrets kept out of code, and a score adds little. Skip it when a migration or rebuild is already decided; assess the target design instead, through cloud migration and modernization. And skip it when one person runs everything, because every practice will score level 1 for the same reason.

Where AI fits in a maturity assessment

AI helps in the evidence step. An AI assistant can read pipeline configurations, alert rules and incident tickets and draft a first score with the artefact links it found, so the session starts from a prepared sheet. Netbase works with the major commercial and open-source models and chooses them per project.

Engineers still open every artefact, decide each score and own the upgrade plan. An AI draft that cannot point to an artefact scores nothing.

What delivery record exists, and what does not?

Netbase works as offshore development partner on a multi-tenant cloud ERP delivered as SaaS on AWS for a US client, not named, and builds and runs its own SaaS products, such as Cloodo Workspace. Netbase names AWS, Google Cloud, DigitalOcean and Cloudflare as the providers it uses, and lists security practices such as secure code review, role-based access control, MFA for admin dashboards, vulnerability scanning and disaster recovery.

Delivery runs in a six-step lifecycle with weekly reviews and KPI dashboards. Most Netbase projects are delivered on fixed-price contracts. No published Netbase record describes a maturity assessment for a client, and no uptime, recovery time or cost result is claimed.

Alternatives: other ways to judge a delivery team

A metrics-first view tracks a few delivery numbers over time; the guide on measuring engineering delivery health explains which to pick. It shows trends but not which practice causes them. A release-gate view checks each release against acceptance criteria; see software quality governance. It protects releases but does not score reliability. When the assessment shows a gap the team cannot close alone, the comparison of cloud consulting and DevOps companies lists criteria for choosing help.

What are the limits of this scorecard?

  • The fifteen practices fit a web or SaaS product on a public cloud; regulated or embedded systems need more rows.
  • The levels are ordinal; an average of 2.6 is a reading aid, not a measurement.
  • Evidence proves a practice exists, not that it works well; pair the score with incident history.

Plan the next step with a Netbase consultant

Frequently asked questions

It depends on how ready the evidence is. When the team collects artefact links before the session, scoring fifteen practices fits one working session; most of the effort is in finding the evidence, not in the scoring.

No. The DORA metrics measure delivery outcomes such as lead time and change failure rate. A maturity score rates the practices that produce those outcomes, and it adds reliability and security practices the metrics do not show.

Most product teams are well served at level 3 in every area, with level 4 only where an incident or a customer requirement justifies the extra review effort.

Yes. Ask the vendor for the same artefact links and score them the same way; a vendor that cannot show a pipeline run or a restore log scores the level below.

Next step

If you want a second reader for your scorecard, book a solution review and bring the artefact links.

AI-ready cloud platform engineering for SaaS and commerce workloads AI-ready cloud platform engineering for SaaS and commerce workloads

Netbase provides cloud platform engineering for SaaS and commerce teams that need a secure, repeatable home for their software and its AI features, on AWS, Google Cloud, DigitalOcean or Cloudflare. We design the architecture, write the environments as code, connect managed and AI services and hand over a platform your team can run, built the way we run ours.

Learn More
line
AI-assisted cloud migration with a cut-over plan and a way back AI-assisted cloud migration with a cut-over plan and a way back

Netbase provides cloud migration services for companies running business applications on ageing servers or hosting to move them to AWS, Google Cloud, DigitalOcean or Cloudflare without losing data or control. AI speeds up the code and dependency assessment; every migration still has an inventory, a rehearsed cut-over, cost controls from day one and a tested rollback point.

Learn More
line
AI-assisted DevOps and platform engineering for routine releases AI-assisted DevOps and platform engineering for routine releases

Netbase provides DevOps and platform engineering for SaaS and commerce teams to make releases routine instead of risky. We set up CI/CD pipelines, repeatable environments and observability on AWS, Google Cloud, DigitalOcean or Cloudflare, with AI assistance for tests, reviews and incident triage, so every change ships the same way and problems surface before customers notice.

Learn More
line
SaaS product accelerator: launch an AI-ready SaaS on proven Netbase modules SaaS product accelerator: launch an AI-ready SaaS on proven Netbase modules

A SaaS product accelerator is a set of reusable Netbase modules for accounts, billing, roles and integrations that helps founders and product teams launch subscription software faster, with room for in-product AI from the first release. Reusing these modules can shorten development time, and the approach is proven on Printcart, Netbase's own web-to-print SaaS.

Learn More
line

Work with Netbase remotely

Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.

Contact Netbase

Discuss a project

Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.
Project enquiries

project@netbasejsc.com

WhatsApp

+84 937 869 689

Office address

91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam

Get in touch

Tell us what you want to build, modernize, or operate.

Tell us what you want to build, modernize, or operate.

Contact Netbase