This guide is for CTOs, heads of engineering and engineering leads of teams of roughly three to fifteen engineers who run their own production system without a dedicated SRE function. It sits beside Netbase's cloud platform engineering service and the DevOps and platform engineering work that sets up the alerts on-call answers.
In this guide
- What does a small team actually need before it starts on-call?
- Which rota pattern fits a team of your size?
- How should severity decide who is paged?
- What belongs in a runbook an engineer can follow at night?
- Setting up on-call in seven steps
- The post-incident review template
- When should a small team not run 24-hour on-call?
- Where AI fits in incident response
- What delivery record exists, and what does not
- Alternatives: other ways to cover incidents
- What are the limits of this guide?
- Frequently asked questions
- Next step
What does a small team actually need before it starts on-call?
A small team needs four things before the first night on call: a written answer to "what hours do we promise", alerts that point at a user-visible symptom, a named person with the authority to make a business call, and a place where the live incident is written down.
If no customer contract or business process depends on the system outside office hours, a business-hours rota with a best-effort phone tree is honest and cheap. If orders, payments or customer logins run at night, someone must be reachable, and the rota has to be built for that.
Google's SRE book sets a useful ceiling: no more than 25% of an engineer's time on call, and at most two incidents in a 12-hour shift, because each incident needs time for analysis, fixes and a review. A small team can copy the ceiling, if not the staffing: when incidents regularly exceed one shift's capacity, the fix is fewer or better alerts.
Which rota pattern fits a team of your size?
| Rota pattern | How it works | Main cost | Choose it when |
|---|---|---|---|
| Business hours plus best effort | One engineer owns alerts in office hours; nights use a phone tree with no response promise | Night incidents wait | No contract or process needs a night response |
| Weekly primary with secondary | One primary takes pages for a week; a secondary is called if the primary does not answer | Each engineer is on call often in a small team | Five or more engineers share the rota and nights matter |
| Follow-the-sun | Two groups in different time zones each cover their own daytime | Two groups must share runbooks and handovers | The team already works across two regions with overlap hours |
| Shared rota with a partner | An external team covers part of the hours under written runbooks | Context lives in two organisations | The system is stable, documented and the partner has run it |
Google's guidance names eight engineers as the minimum for a single-site rota covering every hour, or six per site for two sites. Most small teams are below that, which is why the business-hours pattern is often the correct one rather than a compromise. Move to a weekly primary with a secondary when the business promise requires it, and lighten that person's project work for the week.
Follow-the-sun removes night shifts, which the SRE book links to health effects, but it only works when both groups can fix the same faults. Each shift ends with a handover note: open incidents, silenced alerts and production changes.
How should severity decide who is paged?
A severity table turns a stressful judgement into a lookup; PagerDuty's incident response guide publishes one example. Keep three levels, define each by user impact, and write who is paged, who decides and when the business owner hears about it.
-
SEV-1
- Definition
- Customers cannot complete a core journey, or data is lost or exposed
- Who is paged
- Primary, secondary and the engineering lead
- Who is told
- Business owner within the first half hour
-
SEV-2
- Definition
- A core journey is degraded or a workaround exists
- Who is paged
- Primary
- Who is told
- Business owner in the next daily update
-
SEV-3
- Definition
- Internal impact only, or a single customer
- Who is paged
- No page; a ticket for office hours
- Who is told
- Nobody outside the team
The escalation path follows the table. If the primary does not acknowledge a page, the secondary is paged automatically; if neither answers, the engineering lead is. A primary who has worked a problem without progress should escalate rather than continue alone. Google's incident guidance gives three triggers for declaring an incident: a second team is needed, the outage is visible to customers, or the issue is unsolved after an hour of concentrated analysis.
Name the incident roles even when one person holds two. Google separates the incident commander, who holds the overall state, from the operations lead who applies the fix and the person who communicates with stakeholders. In a team of five, the lead often commands while the primary fixes, so the person fixing is not interrupted.
What belongs in a runbook an engineer can follow at night?
Every alert that can page someone needs a runbook, and each runbook should fit on one page. Use the same five headings every time:
- Symptom. What the alert means in user terms, and which journey is affected.
- Confirm. The dashboard, query or log search that proves the alert is real.
- Mitigate. The safe first actions in order: roll back the last release, fail over, scale, restart, disable a feature flag.
- Escalate. Who to call next and the point at which to call them.
- Afterwards. What to record in the incident document before handing over.
Write runbooks from real incidents and link each from its alert. Have someone who did not write a runbook reread it regularly; if they cannot follow it, it is wrong. The alert signals themselves come from the telemetry choices described in observability stack design with OpenTelemetry.
Setting up on-call in seven steps
-
Write the promise
Agree with the business owner which hours and which journeys need a response, and write it down.
-
Cut the alerts
Keep only alerts tied to a user-visible symptom; move the rest to dashboards or tickets.
-
Pick the rota pattern
Choose from the table above by team size and promise, and publish the schedule where everyone can see it.
-
Write the severity table
Three levels, defined by impact, with who is paged and who is told.
-
Write the first runbooks
One per paging alert, in the five-heading format, linked from the alert.
-
Rehearse once
Run a planned failure in office hours and let the person on call use only the runbook.
-
Review after every SEV-1 and SEV-2
Use the template below and track every action to closure.
The post-incident review template
A blameless review looks for contributing causes in the system, the process and the information people had, not for someone to blame. Google's SRE book describes a postmortem as a written record of the incident, its impact, the actions taken, the root causes and the follow-up actions, and lists triggers such as user-visible downtime, any data loss, a rollback by the on-call engineer and a monitoring failure.
-
Summary
- What to write
- Two sentences: what users saw and for how long
- Who writes it
- Incident commander
-
Timeline
- What to write
- Times of detection, escalation, mitigation and resolution, from the incident document
- Who writes it
- Primary on call
-
Impact
- What to write
- Journeys, customers and data affected
- Who writes it
- Engineering lead with the business owner
-
Contributing factors
- What to write
- Technical, process and information gaps, without names
- Who writes it
- Everyone involved
-
What went well
- What to write
- Alerts, runbooks and decisions that helped
- Who writes it
- Everyone involved
-
Actions
- What to write
- Each action with one owner, a due date and a priority
- Who writes it
- Engineering lead
An action without an owner and a due date is not an action. Review open actions in planning until they close; the share of reviews with every action closed says more than the incident count. The incident row of the DevOps maturity assessment scorecard scores exactly this evidence.
When should a small team not run 24-hour on-call?
Do not run night on-call when no contract, customer journey or business process needs a night response; a published support window is better than an exhausted team. Do not start a rota while alerts fire for causes nobody can act on at night; people learn to ignore pages.
Do not put one engineer on a permanent rota because they know the system best. Write the runbooks and spread the rota instead. Give time off after a night with pages, so the next day starts rested.
Where AI fits in incident response
AI helps around the incident, not inside the decision. An assistant can summarise a long incident channel into a first timeline, draft a post-incident review from the incident document and suggest which existing runbook matches an alert. Netbase works with the major commercial and open-source AI tools and models, chosen per project.
Engineers still confirm every timeline entry, choose and run each mitigation, decide severity and escalation by name and own the review's actions; no model acknowledges a page, rolls back a release or closes an incident.
- In delivered work
MLOps pipeline
An MLOps pipeline Netbase delivered for a client that is not named; an AI delivery record, not an on-call service.
- Available capability
AI agents in operations
AI-assisted incident summaries and review drafts are a way Netbase approaches the work, not yet tied to a published case.
What delivery record exists, and what does not
Netbase built Cloodo, an AI-powered digital workplace that is live at cloodo.com, so its engineers work on a system that has real users. Netbase names AWS, Google Cloud, DigitalOcean and Cloudflare as the cloud providers it uses.
Delivery runs in a six-step lifecycle that ends in ongoing support, with weekly reviews and KPI dashboards. Most Netbase projects are delivered on fixed-price contracts. No published Netbase record describes an on-call rota or an incident history, and no response time, uptime or result figure is claimed here.
Alternatives: other ways to cover incidents
| Alternative | Fits when | Selection criterion |
|---|---|---|
| Managed services from the cloud provider | Most incidents come from infrastructure the team runs itself | Moving a database or queue to a managed service removes a class of pages |
| A support partner for first response | The system is stable and documented | The partner can follow your runbooks and escalate to your engineers |
| A published support window | No night promise exists | Customers accept a response the next working day |
| Investing in release safety first | Most incidents follow a release | Rollback in one step and smaller releases cut incidents at the source |
When incidents follow releases, start with the pipeline; a cloud migration cutover plan uses the same rollback-point thinking for one large change.
What are the limits of this guide?
- It gives patterns and decision rules, not staffing numbers for a particular team or contract.
- Labour law and employment contracts set rules for night work and on-call compensation; check them locally.
- Paging tools differ in how they escalate; check your tool's documentation before relying on automatic escalation.
Plan the next step with a Netbase consultant
Frequently asked questions
Google's SRE guidance names eight engineers for a single-site rota covering every hour, or six per site for two sites. A smaller team can still cover nights, but each person is on call more often, so cut alerts first.
Yes, when the runbooks and alerts are in place first. Engineers paged for their own services fix causes faster, but a rota without runbooks burdens whoever knows the system best.
It is a written review that looks for the system, process and information gaps that let an incident happen, without blaming a person. It ends with a short list of actions, each with one owner and a due date, tracked until it closes.
Escalate when the primary and secondary do not acknowledge a page, when customers cannot complete a core journey, when data is lost or exposed, or when the primary has worked a problem for a while without progress. The severity table should say this in advance.
Next step
If your team is about to start on-call, Netbase can review alerts, runbooks and the escalation path with you as part of a cloud platform engineering engagement. Book a solution review.
Related services and solutions
AI-ready cloud platform engineering for SaaS and commerce workloads Netbase provides cloud platform engineering for SaaS and commerce teams that need a secure, repeatable home for their software and its AI features, on AWS, Google Cloud, DigitalOcean or Cloudflare. We design the architecture, write the environments as code, connect managed and AI services and hand over a platform your team can run, built the way we run ours.
Learn More
AI-assisted cloud migration with a cut-over plan and a way back Netbase provides cloud migration services for companies running business applications on ageing servers or hosting to move them to AWS, Google Cloud, DigitalOcean or Cloudflare without losing data or control. AI speeds up the code and dependency assessment; every migration still has an inventory, a rehearsed cut-over, cost controls from day one and a tested rollback point.
Learn More
AI-assisted DevOps and platform engineering for routine releases Netbase provides DevOps and platform engineering for SaaS and commerce teams to make releases routine instead of risky. We set up CI/CD pipelines, repeatable environments and observability on AWS, Google Cloud, DigitalOcean or Cloudflare, with AI assistance for tests, reviews and incident triage, so every change ships the same way and problems surface before customers notice.
Learn More
SaaS product accelerator: launch an AI-ready SaaS on proven Netbase modules A SaaS product accelerator is a set of reusable Netbase modules for accounts, billing, roles and integrations that helps founders and product teams launch subscription software faster, with room for in-product AI from the first release. Reusing these modules can shorten development time, and the approach is proven on Printcart, Netbase's own web-to-print SaaS.
Learn More
Work with Netbase remotely
Every Netbase engagement is delivered remotely, so clients in any country and any industry can work with us; Netbase has worked with clients in 80 countries. Contracts and invoices are issued by Netbase JSC, and payment is made by bank transfer to Netbase JSC's company account or PayPal Business. Talk to the CEO, Huy Nguyen (David), on WhatsApp or by email.
Discuss a project
Netbase JSC helps organizations design, build, modernize, and operate digital products and AI-enabled business systems.project@netbasejsc.com
+84 937 869 689
91 Nguyen Chi Thanh, Dong Da, Hanoi, Vietnam
Get in touch
Tell us what you want to build, modernize, or operate.