System Design Playbook
Architecture patterns, design examples, and decision frameworks for production-grade PanOS-based systems
System Design Playbook
This playbook helps you design robust systems on top of PanOS.
It focuses on practical trade-offs, not abstract diagrams.
1) Pattern: Single-Service Runtime
When to Use
- One bounded business capability
- Strict startup and operational simplicity requirements
- Low infrastructure overhead
Topology
Host VM
└─ PanOS guest
└─ node /app/service.jsStrengths
- Minimal failure surface
- Easier debugging and incident response
- Faster onboarding for contributors
Risks
- Feature growth can produce accidental monoliths
- Harder to scale by domain when responsibilities multiply
2) Pattern: Sidecar Utility Pair
When to Use
- Main service requires specialized helper logic
- You want strict separation for observability, cache, or translation
Topology
PanOS guest
├─ node /app/core.js
└─ node /app/sidecar.jsIPC Options
- Unix socket (low overhead)
- Loopback HTTP (debuggable)
- File queue in
/tmp(simple and resilient for low throughput)
Example Local IPC Contract
{
"kind": "cache.refresh",
"requestId": "req-123",
"payload": { "keys": ["a", "b"] }
}3) Pattern: Event Pipeline Runtime
When to Use
- Burst ingestion
- Append-only processing
- Loose coupling between ingest and publish
Flow
ingest -> validate -> enrich -> store -> publishReference Implementation Skeleton
export async function pipeline(event) {
const valid = validate(event);
const enriched = await enrich(valid);
await store(enriched);
await publish(enriched);
}4) Pattern: Plugin-Driven Platform Runtime
When to Use
- Multi-tenant or frequently changing integration requirements
- Domain adapters must be added without redeploying core behavior
Plugin Contract
export interface RuntimePlugin {
id: string;
setup(): Promise<void>;
handle(event: unknown): Promise<unknown>;
teardown?(): Promise<void>;
}Design Rule
Core should know only plugin contract, not plugin details.
5) Data Flow Design Example
Goal
Expose controlled telemetry to a dashboard while preserving low runtime overhead.
[App events] -> [Normalizer] -> [Rolling buffer] -> [HTTP /metrics]Lightweight Buffer Strategy
const buffer = [];
const maxEntries = 500;
export function pushMetric(metric) {
buffer.push({ ts: Date.now(), ...metric });
if (buffer.length > maxEntries) buffer.shift();
}
export function getMetrics() {
return { count: buffer.length, samples: buffer.slice(-50) };
}6) Reliability Design Example
Retry Budget Model
network_call:
maxAttempts: 4
delayMs: [100, 250, 500, 1000]
failFastOn: [401, 403]Retry Helper
export async function withRetry(operation, delays = [100, 250, 500, 1000]) {
let lastError;
for (const delay of delays) {
try {
return await operation();
} catch (error) {
lastError = error;
await new Promise((resolve) => setTimeout(resolve, delay));
}
}
throw lastError;
}7) Security Design Example
Boundary-Oriented Hardening
- Validate all host-provided env vars.
- Restrict writable paths.
- Avoid dynamic code loading from untrusted sources.
- Emit security-significant events to logs.
Config Validation Example
function required(name) {
const value = process.env[name];
if (!value) throw new Error(`Missing required env var: ${name}`);
return value;
}
const config = {
apiBaseUrl: required("API_BASE_URL"),
apiToken: required("API_TOKEN"),
};8) Observability Design Example
Logging Envelope
{
"ts": "2026-03-09T12:00:00.000Z",
"level": "info",
"service": "panos-runtime-api",
"event": "request.completed",
"requestId": "req-8f8d",
"latencyMs": 12
}Metrics to Keep
- boot duration
- request latency percentiles
- runtime memory usage
- restart count
- queue depth (if async)
9) Capacity Planning Design Example
Baseline Method
- Establish idle footprint.
- Run representative workload for 15 minutes.
- Capture P50/P95 latency and max RSS.
- Allocate +30% headroom for production safety.
Sample Capacity Note
workload: 80 req/s
memory: 340 MB peak
cpu: 0.7 vCPU average
recommendation: 1 vCPU / 768 MB minimum10) Decision Matrix
| Goal | Recommended Pattern | Why |
|---|---|---|
| Fast delivery, low complexity | Single-Service Runtime | Lowest operational overhead |
| Domain separation with low overhead | Sidecar Utility Pair | Clean boundaries, simple local IPC |
| Throughput and async resilience | Event Pipeline Runtime | Backpressure-friendly model |
| Frequent custom integrations | Plugin-Driven Platform | Extensible without core rewrites |
11) Architecture Review Checklist
- Clear runtime boundaries are documented.
- Failure modes are explicit and tested.
- Observability events are structured.
- Upgrade and rollback strategy is defined.
- Secrets and config paths are validated at startup.
Next Read
Use 14-reference-implementations.mdx for end-to-end implementation blueprints.