Serialization Key Points
Solana stores application state as raw bytes in program-owned accounts. Serialization is the contract that turns those bytes into typed fields and back: which layout, how large, how you grow, and how you evolve without bricking live data.
This page is the conceptual map for Account Data Serialization - why you serialize, Borsh structs, discriminators, space and rent, realloc, versioning and migration, and zero-copy trade-offs on Agave 4.1.1.
Summary
- Program-owned account
datais an opaque buffer; your serialization layout is the on-chain schema that every init, update, client decode, and upgrade must agree on. - Insight: Wrong size, missing discriminator, or a silent layout change fails creates, wastes rent, burns compute, or misinterprets funds and authorities. Serialization choices are database design, not a style preference.
- Key Concepts: opaque account buffer, Borsh field order, account discriminator, SPACE / LEN, rent-exempt minimum, realloc, version byte / migration ix, zero-copy (Pod / bytemuck).
- When to Use This Model: Defining new account types, sizing rent, choosing Borsh vs zero-copy, planning program upgrades, or debugging
InvalidAccountDataand init failures. - Limitations/Trade-offs: You pay for every byte in rent and often in CU; variable-length fields need caps; zero-copy needs fixed, aligned layouts; migrations never rewrite every account atomically.
- Related Topics: Serialization Basics, Account Discriminators, Borsh Structs, Space Calculation, realloc & Growing Accounts, Data Versioning & Migration.
Foundations
An account's data field is a byte slice the runtime does not interpret. Ownership decides who may write; serialization decides what the bytes mean.
Without a shared layout, the program and its clients disagree on field boundaries. A u64 can become half of a Pubkey, an authority check can read the wrong offset, and a "valid" length can still carry the wrong type.
Serialization means: encode structured state into the buffer on write; decode only after length and type checks on read. Instruction data is also bytes, but ephemeral per transaction. Account data persists across slots and upgrades, so it needs fixed constants, discriminators, and migration plans.
Why serialize at all when you could hand-index raw offsets? Explicit codecs (Borsh, Anchor wrappers, zero-copy casts) document order and size, catch length mistakes earlier, and keep TypeScript/Rust clients aligned via IDL or shared constants. Hand packing works for tiny headers; nested fields, options, or multi-type programs need a named schema.
Borsh is the default workhorse for account bodies and instruction payloads. It is deterministic and packed: field declaration order is wire order, integers are little-endian, bool is one byte, Pubkey is 32 bytes, and the stream has no Rust alignment padding. That predictability makes rent math and cross-language decoding practical.
Account data (typical Borsh account):
[ 8-byte discriminator ][ field0 ][ field1 ][ ... ][ optional version / tail ]
type tag packed body in declaration orderDiscriminators sit at the front of multi-type programs. Before parsing the body, compare the prefix to the expected constant so a Vault is never read as a User. Anchor 0.32.1 uses eight bytes of SHA256("account:TypeName"). Native programs pick unique constants and publish them to clients. Without a tag, any long enough account your program owns can be cast to the wrong struct (type confusion).
Space is decided at allocate time. Creation funds a rent-exempt minimum for data_len. The usual formula:
SPACE = discriminator_len + body_len
+ caps for variable pieces (Vec: 4 + max_n * elem_size; Option: 1 + inner)Document LEN / SPACE next to the struct. Anchor InitSpace / space = must match. Clients and CPI creates use the same number so rent and allocation do not drift.
Mechanics & Interactions
Borsh read/write path
On init: allocate SPACE, assign your program as owner, fund rent, write the discriminator, then serialize the initial body. On update: borrow data, check length and discriminator, deserialize, apply rules, serialize back (or into a realloc'd buffer).
// Conceptual Borsh account body (disc often written separately)
#[derive(BorshSerialize, BorshDeserialize)]
pub struct User {
pub authority: Pubkey, // 32
pub points: u64, // 8
pub bump: u8, // 1
}
impl User {
pub const LEN: usize = 32 + 8 + 1;
}
pub const USER_SPACE: usize = 8 + User::LEN; // disc + bodyHost tests that serialized length equals LEN catch off-by-one bugs before mainnet. Map deserialize failures to InvalidAccountData (or Anchor account errors), not panics.
Discriminator check before body parse
if data.len() < 8 || data[..8] != USER_DISC {
return Err(ProgramError::InvalidAccountData);
}
let user = User::deserialize(&mut &data[8..])?;Write the discriminator only on legitimate init. Close by zeroing or reclaiming so stale tags do not linger. Never change a live type's discriminator; that orphans every account of that type. Evolve with a version field after the disc, not a new disc for the same type.
Space, rent, and clients
let lamports = Rent::get()?.minimum_balance(USER_SPACE);Larger accounts lock more SOL as rent-exempt deposit. Uncapped variable fields make rent unbounded and invite griefing; prefer fixed arrays or max_entries. Off-chain, @solana/kit 7.0.0 codecs and Anchor/Codama clients must decode the same layout; treat IDL or shared constants as source of truth.
realloc when the buffer must change
Schemas and collections grow. realloc changes data_len under owner control. Growth needs a payer for the rent delta; shrink can return excess lamports when done correctly. After growth, zero new bytes (or write defaults) so stale memory does not become new fields. Cap max length against rent/CU griefing. Anchor 0.32.1 uses realloc constraints; native code uses the program realloc path.
old_len ----------------------> new_len
[ disc | known fields | .... ]
realloc
[ disc | known fields | NEW zeros then init ]
^
fund rent delta from payerVersioning and migration
Upgrading bytecode does not rewrite user accounts. If V2 needs another u64, V1 accounts still have the old length. Patterns that work:
| Strategy | Mechanism | When it fits |
|---|---|---|
| Append-only fields | New fields only at end; old accounts stay short until migrated | Small additive changes |
| Version byte / enum | Body version field selects parser | Concurrent layouts |
| Dedicated migrate ix | Read V1, write V2, optionally realloc | Controlled upgrades |
| Lazy migrate | Migrate on next user touch | Spread CU and rent cost |
| New account type | New disc + copy flow | Breaking redesigns |
Dual-read V1 and V2 during transition. Do not assume every account upgraded after deploy. Gate migrate (admin or account authority) so strangers cannot grief rent paths. Do not remove or reorder fields in place; that is how you lose funds or authorities.
Zero-copy path (contrast)
For large fixed layouts (order books, tick arrays, bitmaps), full Borsh parse/serialize every instruction burns CU with size. Zero-copy maps the buffer to a #[repr(C)] Pod + Zeroable struct (bytemuck or Anchor AccountLoader) and mutates in place. Requirements: fixed size, no String/Vec in the Pod view, alignment discipline, and usually size_of::<T>() as SPACE. Discriminator still matters (often a u64 at offset 0). Capacity is planned up front; growing pure zero-copy means a deliberate migration, not casual field appends.
Advanced Considerations & Applications
Treat the layout as a public API. Reviewers should answer: disc uniqueness, SPACE formula, authority fields, max variable sizes, and the migrate story if the program stays upgradeable.
| Approach | Strengths | Costs / constraints | Best fit |
|---|---|---|---|
| Borsh full account | Flexible fields, easy append, simple versioning | CU scales with size; full rewrite on save | Config, profiles, small/medium state |
| Zero-copy Pod | In-place updates, low CU on large fixed tables | Fixed size, alignment, harder schema churn | Books, slabs, bitmaps, hot large state |
| Hybrid (Pod header + Borsh tail) | Fast hot fields + flexible tail | Two layouts to document and test | Header + occasional variable blob |
| Oversized allocate now | Avoids early realloc | Extra rent locked early | Known growth ceiling |
| realloc on demand | Pay rent when needed | Payer, zero-init, CU, max caps | Schema upgrades, growing collections |
Instruction data vs account data: keep schemas separate. Instruction enums change more freely (ephemeral); account layouts need migration. Sharing one evolving struct for both invites collisions between ix tags and account version fields.
Immutable programs freeze logic and practical schema evolution. If you will revoke upgrade authority, reserve headroom (padding or oversized SPACE) or accept a new program ID plus migration UX for new fields.
Clients and indexers lag the chain. Prefer version fields (and migration events) they can filter. Ship dual-version decoders in @solana/kit 7.0.0 / Codama clients during rollouts, matching the program.
Audit focus: missing disc checks, wrong SPACE, unbounded Vec/String, realloc without zeroing, unauthenticated migrate, and treating ownership as type (one program, many account kinds).
On an Agave 4.1.1-aligned stack (CLI 3.0.10, Anchor 0.32.1, Rust 1.91.1, LiteSVM 0.6.x, Surfpool 0.12.0): roundtrip serialize on host, fuzz bad discs and short buffers, and migrate from real V1 byte snapshots.
Common Misconceptions
- "Ownership is enough; I do not need a discriminator." Owner proves which program may write. Discriminators prove which of your layouts the buffer is.
- "I can reorder Borsh fields; names match on both sides." Borsh is positional. Reorder breaks live accounts and old clients.
- "SPACE is just the sum of Rust field sizes." Include disc, Option tags, Vec length prefixes, and string maxes. For Pod use
size_of, not a guessed sum. - "realloc initializes new bytes safely by default." Treat zeroing and explicit field init as required after growth.
- "Deploying new program code upgrades all account layouts." Loaders swap bytecode; data accounts stay until your migrate path rewrites them.
- "Zero-copy is always faster, so always use it." On small accounts Borsh is often fine; Pod rigidity and migration cost dominate. Profile first.
- "Variable strings in accounts are fine if users are honest." Unbounded sizes enable rent and CU griefing. Cap lengths or keep content off-chain.
- "Changing the discriminator cleanly marks V2." That orphans V1 under the old type. Prefer a version field with a stable disc.
- "If deserialize succeeds, the account is trustworthy." Still check disc, size, and authorities. Fit-to-layout is not semantic correctness.
- "Clients can infer layout from program ID alone." Many account types share one program ID. Clients need disc plus schema (IDL, Codama, or constants).
FAQs
Why do Solana programs serialize account data at all?
Account data is a raw buffer with no runtime schema. Serialization is the agreed layout that turns bytes into fields for programs and clients, and back into durable storage.
When should I choose Borsh for accounts?
Default to Borsh for small and medium state, configs, user records, and anything that may gain fields later. It is the common Anchor and native pattern and works well with versioning.
What is an account discriminator?
A short fixed prefix (commonly 8 bytes) that identifies the account type before the body is parsed. It blocks reading a Vault as a User when both share one program owner. See Account Discriminators.
How does Anchor 0.32.1 pick discriminators?
Account discriminators are the first 8 bytes of SHA256("account:TypeName"). Instruction discriminators use a "global:..." preimage. Native programs may use manual constants if clients agree.
How do I calculate SPACE and rent?
SPACE = disc + body (+ caps for variable fields), then Rent::get()?.minimum_balance(SPACE) (or client/CLI equivalent). Keep a named constant and test it. See Space Calculation.
What is the usual size of Pubkey, u64, bool, and Option in Borsh?
Pubkey 32 bytes, u64/i64 8 bytes little-endian, bool 1 byte, Option<T> 1 byte tag plus the inner size when Some.
When do I need realloc?
When the account must grow for new fields or more entries, or shrink after pruning. Fund rent on growth, zero or init new bytes, and enforce a max length. See realloc & Growing Accounts.
How should I version account layouts after a program upgrade?
Keep the discriminator stable, add a version field, dual-read during rollout, and ship an explicit or lazy migrate path (realloc if V2 is larger). See Data Versioning & Migration.
What is zero-copy and when is it worth it?
Zero-copy casts account bytes to a fixed Pod struct for in-place updates, skipping full Borsh parse/serialize. Use it when profiling shows deserialize dominates on large fixed tables; accept rigid size and harder migrations.
Can I mix Borsh and zero-copy?
Yes. Common pattern: fixed Pod header (disc, sequence, hot counters) plus a bounded Borsh or raw region. Document both halves and test length boundaries.
Does serialization cost compute units?
Yes. Copying and parsing large buffers costs CU. Zero-copy helps on large accounts; on small ones the gap is often minor next to syscalls and CPIs.
Where should clients get the layout from?
From the same source of truth as the program: Anchor IDL, Codama / @solana/kit 7.0.0 codecs, or exported disc and field-order constants. Hand-rolled clients drift.
What fails if SPACE is wrong at init?
Too small: serialize or later writes fail or truncate. Too large: users overpay rent until you shrink. Off-by-eight usually means a forgotten discriminator.
Should every account type include a version byte from day one?
Strongly recommended for upgradeable programs. A single u8 after the disc is cheap insurance versus a forced big-bang migration.
How do instruction codecs relate to account codecs?
Both are byte layouts, but instruction data is per-transaction and easier to change. Account layouts survive upgrades and need discriminators, space planning, and migration.
Related
- Serialization Basics - Borsh vs zero-copy starter patterns
- Account Discriminators - type tags and confusion defenses
- Borsh Structs - field order, LEN, and struct design
- Space Calculation - SPACE formulas and rent-exempt minimums
- realloc & Growing Accounts - resize, rent delta, zeroing
- Data Versioning & Migration - version fields and migrate paths
- Zero-Copy Accounts - Pod layouts and CU trade-offs
- Serialization Best Practices - layout, migration, and test checklist
Stack versions: This page was written for Agave 4.1.1, Solana CLI 3.0.10, Anchor 0.32.1, anchor-lang 0.32.1, Rust 1.91.1, @solana/kit 7.0.0, Surfpool 0.12.0, and LiteSVM 0.6.x.