Delivery Best Practices
Enterprise Solana delivery means releases are reversible, bytecode is verifiable, and observability proves health within minutes of deploy. These practices reduce change-fail rate and MTTR.
How to Use This List
- Gate mainnet promotion on sections A and B
- Apply C before any public launch or TVL milestone
- Review D monthly with platform and security leads
A - Verifiable & Reversible
- Build with
anchor build --verifiablein CI. Artifact hash published on every tag. - Retain last three
.sofiles per program. Rollback does not depend on one engineer's laptop. - Run
solana-verifybefore and after mainnet deploy. Hash matches release notes. - Document rollback vs forward-fix decision tree. Linked from every deploy ticket.
- Never deploy mainnet from unmerged branch. Squads proposal references git SHA.
B - Authority & Governance
- Upgrade authority on multisig before launch. Hot keys only on break-glass doc.
- Separate deploy wallets per environment. Devnet keys never touch mainnet programs.
- Timelock on community-visible upgrades when policy requires. Emergency path documented.
- Pause instruction tested quarterly. Containment under 5 minutes on drill.
- IDL semver in release notes. Clients pin compatible
@coral-xyz/anchor0.32.1 types.
C - Coordinated Release Train
- Single change window for breaking IDL updates. Program + frontend + indexer in sync.
- Devnet smoke mandatory. Top flows simulate and send before mainnet.
- Feature flags for new instruction paths. On-chain guard for security-critical flags.
- Env validation fails build on missing
PROGRAM_ID. Preview cannot target mainnet. - Indexer team notified pre-deploy. Backfill plan for new accounts or events.
D - Observable Deploys
- Synthetic landing tx within 15 minutes post-deploy. Alert on failure.
- Program error rate dashboard per release tag. Compare to pre-deploy baseline.
- RPC failover configured in production client. Primary and secondary URLs tested.
- DORA metrics reviewed monthly. Deploy frequency, change-fail, MTTR trends.
- Post-mortem action items close within 30 days. Or exec escalation.
FAQs
Which section blocks hotfix?
B pause authority never blocked; full A-D required before re-enabling full traffic after fix.
Minimum team size for this?
Two engineers can follow A-B; C-D scale with launch size.
Verifiable builds slow CI?
Small cost vs audit and incident cost; cache target dir.
Immutable programs?
Still apply client and ops practices; skip upgrade-specific items.
How prove observability?
Link Grafana dashboard and synthetic check run in deploy ticket.
Agave upgrade in train?
Add SIMD review item; test on Surfpool 0.12.0 before mainnet program deploy.
External audit gate?
No mainnet user funds until audit findings Sev-High closed or accepted in ADR.
Who owns list compliance?
Tech lead signs deploy ticket; security reviews B monthly.
Relation to DORA?
These practices directly improve change-fail rate and MTTR.
Startup MVP skip?
Never skip A-B if mainnet holds user funds; C-D can phase in.
Related
- DORA Metrics for Web3 - measure outcomes
- Program Upgrade Strategy - upgrade detail
- Prevention Checklists - incident prevention
Stack versions: This page was written for Agave 4.1.1, Solana CLI 3.0.10, Anchor 0.32.1, anchor-lang 0.32.1, Rust 1.91.1, @solana/kit 7.0.0, Surfpool 0.12.0, and LiteSVM 0.6.x.