Stage 1 — Foundations: The Bedrock of Decentralized Systems

Stage 1 of 6 · Estimated 4–6 weeks

Stage Overview

What This Stage Builds

This stage constructs the cognitive and technical foundation upon which every subsequent blockchain concept rests. You will develop fluency in networking, cryptography, data structures, distributed systems theory, and game theory — not as abstract academic exercises, but as the exact engineering primitives that make blockchains possible.

Why This Stage Is Critical

Every blockchain failure — every exploit, every broken consensus, every flawed protocol — traces back to a misunderstanding of fundamentals. Engineers who skip this stage build fragile systems. They copy patterns without understanding invariants. They write smart contracts without understanding the execution environment beneath them. This stage ensures you are not that engineer.

Competence at the End of This Stage

You can explain how data moves across the internet at a packet level. You can implement basic cryptographic primitives from scratch. You can construct and verify Merkle trees. You can reason about distributed consensus failures using formal models. You can articulate why game-theoretic incentives are not optional in open networks — they are the architecture.


Conceptual Map

Internet & Networking
├── How data moves (TCP/IP, HTTP, DNS)
├── Client-server vs peer-to-peer models
└── NAT traversal, gossip protocols
        │
        ▼
Cryptography
├── Hash functions (SHA-256, Keccak)
├── Asymmetric cryptography (ECDSA, EdDSA)
├── Digital signatures
└── Commitment schemes
        │
        ▼
Data Structures
├── Hash-linked lists (blockchain as a structure)
├── Merkle trees
├── Patricia/Radix tries
└── Bloom filters
        │
        ▼
Distributed Systems
├── CAP theorem
├── Failure models (crash, Byzantine)
├── State machine replication
├── Byzantine Fault Tolerance
└── Practical BFT algorithms
        │
        ▼
Game Theory
├── Nash equilibrium
├── Mechanism design
├── Incentive compatibility
└── Schelling points

Topic 1: How the Internet Works

Explanation

The internet is a layered packet-switching network. Data is broken into packets, each independently routed across interconnected autonomous systems (ASes). The architecture follows the OSI/TCP-IP model:

DNS resolves human-readable names to IP addresses via a hierarchical, cached lookup system (recursive resolvers → root servers → TLD servers → authoritative servers).

Why It Matters

Blockchain nodes communicate over the internet. Understanding packet routing, NAT traversal, firewall behavior, and TCP reliability directly impacts your ability to reason about node connectivity, network partitions, eclipse attacks, and propagation delays — all of which affect consensus safety and liveness.

Real-World Context

Tools / Technologies

Tool Purpose
Wireshark Packet capture and protocol analysis
tcpdump Command-line packet analyzer
curl / httpie HTTP request inspection
dig / nslookup DNS resolution debugging
traceroute / mtr Network path analysis
netcat Raw TCP/UDP socket communication

Deep Mastery Questions

  1. If a TCP packet is lost between two blockchain nodes, what mechanism ensures the transaction data is eventually delivered? What are the performance implications?
  2. Why does Bitcoin use TCP rather than UDP for block propagation? What trade-offs does this create?
  3. How does NAT traversal work, and why is it relevant for peer-to-peer blockchain networks where nodes sit behind home routers?
  4. Explain how an attacker could perform an eclipse attack by manipulating a node's DNS resolution or peer table. What defenses exist?
  5. What is the difference between a network partition and high latency? How does each affect blockchain consensus differently?
  6. Why does Ethereum's peer discovery use UDP (for discv5) while actual data exchange uses TCP (for devp2p/RLPx)?
  7. If you had to design a protocol for propagating a 1MB block to 10,000 nodes as fast as possible, what approach would you take and why?

Hands-On Exercises

  1. Packet tracing: Use Wireshark to capture a full HTTP request-response cycle. Identify the TCP handshake, HTTP headers, and payload.
  2. DNS walkthrough: Use dig +trace example.com to trace the full DNS resolution path from root servers to the authoritative answer.
  3. Raw socket server: Write a simple TCP echo server and client in Python. Send a message, observe the bytes on the wire with tcpdump.
  4. Latency mapping: Use mtr to trace the route to several Ethereum RPC endpoints (infura.io, alchemy.com). Note hop counts and latency.

Mini Build Task

Build a simple peer-to-peer chat application using raw TCP sockets in Python. Two peers should be able to discover each other on a local network (via a hardcoded list or a simple discovery mechanism), establish a TCP connection, and exchange text messages bidirectionally. This exercise forces you to handle connection setup, message framing, and concurrent I/O — the same problems that blockchain networking layers solve.


Topic 2: Peer-to-Peer Networking Fundamentals

Explanation

In client-server architecture, a central server mediates all communication. In peer-to-peer (P2P) networks, every participant is both client and server. This eliminates single points of failure but introduces coordination complexity.

Key P2P concepts:

Why It Matters

Every blockchain is a P2P network. The security guarantees of a blockchain are only as strong as the networking layer's ability to ensure that all honest nodes receive all valid transactions and blocks in a timely manner. Understanding gossip efficiency, eclipse attack vectors, and network topology is essential for protocol design.

Real-World Context

Tools / Technologies

Tool Purpose
libp2p Modular P2P networking stack (used by Ethereum 2.0, IPFS, Polkadot)
devp2p Ethereum's P2P networking protocol suite
Kademlia DHT algorithm for structured P2P routing
GossipSub Pub/sub gossip protocol in libp2p

Deep Mastery Questions

  1. Why is gossip protocol convergence O(log n) in the number of rounds? What assumptions does this require?
  2. What is an eclipse attack, and how does it differ from a Sybil attack? How can a blockchain node defend against each?
  3. In the Kademlia DHT, what is the XOR distance metric, and why was it chosen over other distance functions?
  4. Why does Bitcoin limit outbound connections to 8? What would happen if this number were 1? What if it were 1,000?
  5. How does GossipSub's mesh-based approach improve upon pure flooding for block propagation?
  6. What is the "nothing at stake" problem in P2P gossip — specifically, why might a rational node choose not to relay blocks?

Hands-On Exercises

  1. Implement Kademlia routing: Write a simplified Kademlia node in Python that maintains a routing table with k-buckets. Implement FIND_NODE and STORE RPCs.
  2. Gossip simulation: Simulate a gossip protocol across 100 nodes. Measure how many rounds it takes for all nodes to receive a message as you vary the fanout parameter (number of peers each node forwards to).
  3. libp2p experimentation: Set up two libp2p nodes (in Go or Rust) and have them discover each other via mDNS and exchange messages.

Mini Build Task

Build a gossip-based message dissemination simulator. Create a network of 50+ simulated nodes. When one node introduces a message, it propagates via gossip. Track and visualize: time to full propagation, redundant messages received, and the effect of node failures (randomly drop 10% of nodes mid-propagation). Output a report showing convergence time and message overhead.


Topic 3: Cryptography — Hashing, Digital Signatures, and Asymmetric Keys

Explanation

Cryptography provides the mathematical guarantees that make trustless systems possible.

Hash Functions

A cryptographic hash function H maps arbitrary-length input to a fixed-length output with three properties:

  1. Preimage resistance: Given h, it is computationally infeasible to find m such that H(m) = h.
  2. Second preimage resistance: Given m₁, it is infeasible to find m₂ ≠ m₁ such that H(m₁) = H(m₂).
  3. Collision resistance: It is infeasible to find any m₁ ≠ m₂ such that H(m₁) = H(m₂).

Bitcoin uses SHA-256 (double hashing: SHA256(SHA256(x))). Ethereum uses Keccak-256 (the original SHA-3 submission, not the NIST-standardized version).

Asymmetric Cryptography

A key pair (sk, pk) where sk is secret and pk is public. The core primitive: given pk, deriving sk is computationally infeasible.

Blockchain systems primarily use elliptic curve cryptography (ECC):

Digital Signatures

A signature scheme consists of three algorithms:

  1. KeyGen() → (sk, pk): Generate a key pair.
  2. Sign(sk, m) → σ: Produce a signature.
  3. Verify(pk, m, σ) → {0, 1}: Verify the signature.

ECDSA (Elliptic Curve Digital Signature Algorithm) is used in Bitcoin and Ethereum. The signature (r, s) is computed as:

Critical: Nonce reuse in ECDSA is catastrophic — it leaks the private key. This is not theoretical: the PlayStation 3's ECDSA implementation was broken because Sony used a fixed nonce.

Commitment Schemes

A two-phase protocol:

  1. Commit: Publish C = H(value || nonce).
  2. Reveal: Publish (value, nonce). Anyone can verify H(value || nonce) = C.

Used in commit-reveal voting, randomness generation, and submarine sends.

Why It Matters

Every transaction on every blockchain is authenticated via digital signatures. Address derivation, transaction validation, block hashing, Merkle root computation — cryptography is not a module of the blockchain; it is the substrate.

Real-World Context

Tools / Technologies

Tool Purpose
openssl CLI for hashing, key generation, and signature operations
hashlib (Python) Compute SHA-256, Keccak-256
eth-keys (Python) Ethereum key and signature operations
noble-secp256k1 (JS) Pure JS secp256k1 implementation
libsodium Modern cryptographic library (Ed25519, X25519)
CyberChef Web-based tool for encoding/hashing experiments

Deep Mastery Questions

  1. Why does Bitcoin use double SHA-256 (SHA256(SHA256(x)))? What specific attack does this mitigate?
  2. Explain why ECDSA nonce reuse leaks the private key. Derive this mathematically.
  3. What is the difference between SHA-3 (NIST standard) and Keccak-256 (used by Ethereum)? Why does Ethereum use the non-standard version?
  4. If a hash function produces 256-bit outputs, how many hashes must you compute (on average) to find a collision? What theorem gives you this bound?
  5. Why is secp256k1 considered "less standard" than NIST P-256, and why did Bitcoin choose it anyway?
  6. How does an Ethereum address derivation work, step by step, from a randomly generated private key?
  7. What is a length-extension attack, and which hash functions are vulnerable to it? How does HMAC prevent it?
  8. Why can't you recover a public key from an Ethereum address? Why does Ethereum transaction verification still work?

Hands-On Exercises

  1. Hash exploration: Compute SHA-256 and Keccak-256 of the string "blockchain". Change one character and compute again. Observe the avalanche effect.
  2. Key generation: Generate an secp256k1 key pair using Python's eth-keys library. Derive the Ethereum address from the public key.
  3. Sign and verify: Sign a message with your private key. Verify the signature with the public key. Then tamper with the message and show verification fails.
  4. Nonce reuse attack: Given two ECDSA signatures that share the same nonce k (use synthetic data), recover the private key algebraically.
  5. Commitment scheme: Implement a commit-reveal scheme in Python. Commit to a value, reveal it, and verify.

Mini Build Task

Build a simple digital identity system. A user generates a key pair, signs a "profile" (name, timestamp), and publishes the signature and public key. A verifier takes the published data and confirms: (a) the profile was signed by the holder of the corresponding private key, and (b) the profile has not been tampered with. Implement in Python using eth_keys and eth_abi for encoding.


Topic 4: Data Structures — Merkle Trees and Hash-Linked Structures

Explanation

Hash-Linked Lists (Blockchain as a Data Structure)

The blockchain is, at its most fundamental, a singly-linked list where each node contains the hash of the previous node:

Block N: { data*N, hash(Block N-1) }
Block N-1: { data*{N-1}, hash(Block N-2) }

This creates tamper evidence: modifying any historical block changes its hash, which invalidates every subsequent block's back-pointer. Integrity verification requires only the latest block's hash.

Merkle Trees

A Merkle tree (hash tree) is a binary tree where:

Key property: Merkle proofs allow O(log n) verification that a specific data element is included in the dataset, given only the root hash and a logarithmic number of sibling hashes (the "proof path").

Patricia/Merkle Patricia Tries (MPT)

Ethereum's state is stored in a Modified Merkle Patricia Trie — a combination of:

This allows: (1) O(log n) key-value lookups, (2) a single hash (state root) that commits to the entire global state, (3) efficient state proofs.

Bloom Filters

A probabilistic data structure that answers set membership queries with:

Ethereum uses Bloom filters in transaction receipts to enable efficient log filtering without scanning all logs.

Why It Matters

Merkle trees enable light clients — nodes that verify blockchain data without downloading the full chain. Patricia tries enable Ethereum's stateful model. Without these structures, blockchains would require every participant to store and process the complete history, making the system impractical.

Real-World Context

Tools / Technologies

Tool Purpose
merkletreejs (JS) Merkle tree construction and proof generation
py-merkle-tree Python Merkle tree library
ethereum/go-ethereum (Geth) Reference implementation of Ethereum's MPT
rlp (Python) RLP encoding used in Ethereum's trie

Deep Mastery Questions

  1. Given a Merkle tree with 1 million leaves, how many hashes are needed for a Merkle proof? What is the exact proof size in bytes if using SHA-256?
  2. Why must Merkle tree implementations handle the case where the number of leaves is not a power of two? What security issue arises if you simply duplicate the last leaf?
  3. Explain how Ethereum's state root changes when a single account's balance is updated. How many nodes in the MPT are modified?
  4. What is a "hash tree attack" on a Merkle tree, and how does prepending a 0x00/0x01 byte to leaf/internal nodes prevent it?
  5. Why does Bitcoin use Merkle trees for transactions but not for UTXO state? What structure does Bitcoin use for UTXO tracking?
  6. How do Bloom filters in Ethereum receipts work, and what are their false positive rates with typical parameters?

Hands-On Exercises

  1. Build a Merkle tree from scratch: Implement a binary Merkle tree in Python. Given 8 data items, construct the tree, compute the root, and generate a proof for one item. Verify the proof.
  2. Tamper detection: Modify one leaf in your tree. Show that the Merkle root changes and the old proof no longer verifies.
  3. Variable-size tree: Handle non-power-of-two leaf counts correctly. Test with 5, 7, and 10 leaves.
  4. Bloom filter implementation: Build a simple Bloom filter. Insert 100 items, test for membership (true and false cases), measure the empirical false positive rate vs. the theoretical prediction.

Mini Build Task

Build a verifiable file integrity system. Given a directory of files, compute a Merkle tree over the file hashes. Store the root hash. Later, given a single file and a Merkle proof, verify that the file was part of the original directory without needing the other files. This is exactly how SPV works in Bitcoin.


Topic 5: Distributed Systems Basics and Byzantine Fault Tolerance

Explanation

Distributed Systems Fundamentals

A distributed system is a collection of independent computers that appear as a single system to users. The fundamental challenges:

The CAP Theorem (Brewer's Theorem)

For any distributed data store, you can guarantee at most two of:

Since network partitions are inevitable in real systems, the practical choice is between CP (consistent under partition, but some requests may be rejected) and AP (available under partition, but data may be inconsistent).

Blockchains make a nuanced choice: they provide eventual consistency (AP-leaning) with strong guarantees after sufficient confirmations.

Failure Models

  1. Crash failure: A node stops and never recovers. Requires 2f + 1 total nodes to tolerate f crash failures.
  2. Byzantine failure: A node behaves arbitrarily — it can lie, send conflicting messages, or collude. Requires 3f + 1 total nodes to tolerate f Byzantine failures.

State Machine Replication (SMR)

The core abstraction: model each node as a deterministic state machine. If all nodes start in the same state and process the same sequence of inputs, they arrive at the same state. The consensus problem reduces to: agree on a total ordering of inputs.

Byzantine Fault Tolerance (BFT)

FLP Impossibility

Fischer, Lynch, and Paterson (1985) proved that no deterministic consensus protocol can guarantee both safety and liveness in an asynchronous system with even one crash failure. Practical systems work around this via:

Why It Matters

Blockchain consensus is a specific instance of Byzantine fault-tolerant state machine replication. Without understanding SMR, BFT bounds, and the CAP theorem, you cannot reason about why blockchains make the design trade-offs they do — why Bitcoin has 10-minute blocks, why Ethereum needs finality gadgets, or why "fast finality" chains sacrifice some liveness guarantees.

Real-World Context

Tools / Technologies

Tool Purpose
Tendermint Core BFT consensus engine
CometBFT Fork of Tendermint, used in Cosmos SDK
Raft (etcd) Crash fault-tolerant consensus (non-Byzantine, for comparison)
Jepsen Distributed systems testing framework

Deep Mastery Questions

  1. Why does Byzantine fault tolerance require 3f + 1 nodes, while crash fault tolerance requires only 2f + 1? Derive this from first principles.
  2. Explain the FLP impossibility result in plain language. How do practical Byzantine consensus protocols (PBFT, Tendermint) circumvent it?
  3. In PBFT, why is the "view change" protocol necessary? What happens if the primary (leader) is Byzantine?
  4. How does Bitcoin's Nakamoto consensus relate to traditional BFT? Is it more or less tolerant of Byzantine nodes than PBFT?
  5. The CAP theorem says you can't have all three. Where does Bitcoin fall? Where does a PBFT-based chain fall?
  6. What is "accountable safety" in Ethereum's Casper FFG, and why is it a stronger guarantee than standard BFT safety?
  7. What is the difference between "safety" and "liveness" in consensus? Which does Bitcoin prioritize? Which does Tendermint prioritize?
  8. If a network partition splits validators 60/40 in a Tendermint chain, what happens? What if the same split happens in Bitcoin?

Hands-On Exercises

  1. Implement Raft: Build a simplified Raft consensus protocol (leader election + log replication only) across 3 processes. Demonstrate that the system continues operating when 1 process crashes.
  2. Byzantine generals simulation: Simulate the Byzantine generals problem with 4 generals, 1 traitor. Show that consensus is achieved. Then try with 3 generals, 1 traitor, and show that consensus fails.
  3. PBFT walkthrough: Trace through PBFT's three phases (pre-prepare, prepare, commit) on paper for a 4-node system with 1 Byzantine node. Show how the honest nodes still reach agreement.
  4. Partition simulation: Build a 5-node system where you can introduce network partitions. Observe how consensus behavior changes under partition.

Mini Build Task

Build a simplified Byzantine fault-tolerant replicated log. Implement a 4-node system where:


Topic 6: Game Theory Basics for Protocol Design

Explanation

Why Game Theory in Blockchain

In traditional systems, you trust the server (or the operator). In open blockchain networks, participants are anonymous and potentially adversarial. You cannot rely on trust — you must design systems where rational self-interest produces correct system behavior.

Core Concepts

Why It Matters

Every design decision in a blockchain protocol is a game-theoretic statement. Transaction fee mechanisms, validator rewards, slashing conditions, governance voting — all are games played by rational (and sometimes irrational) agents. If the incentives are wrong, the protocol breaks — not because of a bug in code, but because of a bug in the game.

Real-World Context

Tools / Technologies

Tool Purpose
Gambit Game theory analysis software
Nashpy (Python) Compute Nash equilibria
Agent-based modeling (Mesa, Python) Simulate multi-agent strategic behavior
Cadence (cadCAD) Python framework for crypto-economic modeling

Deep Mastery Questions

  1. Why is "mine on the longest chain" approximately (but not exactly) a dominant strategy in Bitcoin? What is the exception, and why does it matter? (Hint: selfish mining.)
  2. Explain EIP-1559's fee mechanism as a mechanism design problem. What behavior does it incentivize? What behavior does it discourage?
  3. In Ethereum's proof-of-stake, why is the slashing penalty for correlated failures higher than for individual failures? What game-theoretic principle does this enforce?
  4. What is the "nothing at stake" problem in naive proof-of-stake, and how does it relate to Nash equilibrium?
  5. How does MEV (Maximal Extractable Value) arise from game-theoretic incentives? Why can't it be "solved" by protocol rules alone?
  6. In a DAO vote with token-weighted governance, what is the free-rider problem? How does it affect governance quality?
  7. What is the difference between a credible threat and a non-credible threat in the context of blockchain slashing?

Hands-On Exercises

  1. Prisoner's dilemma simulation: Implement iterated prisoner's dilemma with multiple strategies (always cooperate, always defect, tit-for-tat, random). Tournament-style: run all strategies against each other over 1000 rounds. Analyze which strategy dominates.
  2. Nash equilibrium computation: Using Nashpy, compute the Nash equilibrium for a 2-player game representing a mining strategy choice (mine honestly vs. selfish mine) with specified payoffs.
  3. Fee market simulation: Simulate a blockchain fee market with 100 users and limited block space. Users have private valuations for inclusion. Implement a first-price auction and an EIP-1559-style mechanism. Compare outcomes: revenue, user experience, and waste.

Mini Build Task

Build a crypto-economic simulation of the "nothing at stake" problem. Simulate a proof-of-stake chain where validators can vote on multiple forks at zero cost. Show that rational validators always vote on every fork (since it costs nothing and increases expected reward). Then introduce slashing conditions: validators who vote on conflicting forks lose their stake. Show that slashing changes the Nash equilibrium so that honest single-chain voting becomes the dominant strategy.


Mini Projects

Project 1: Verifiable Data Registry

Build a system where users can register data entries (e.g., document hashes) into a Merkle tree. The system publishes the Merkle root. Users can later prove that their data was registered by presenting a Merkle proof. Add signature-based access control: only the original registrant (verified by ECDSA signature) can update their entry.

Project 2: P2P File Sharing System

Build a simple P2P file sharing system where nodes can advertise files they have and request files from peers. Use gossip-based discovery and direct TCP transfers. Handle: peer joining, peer leaving, file chunk verification (using hashes from a Merkle tree of file chunks).

Project 3: Byzantine Agreement Simulator

Build an interactive simulator that visualizes the Byzantine generals problem. Allow the user to set: number of generals, number of traitors, and message strategy for traitors. Show step-by-step message passing and final agreement. Demonstrate the 3f + 1 threshold empirically.


Capstone Project: Simplified Blockchain Prototype (Data Layer Only)

Build a simplified blockchain that implements the data structure and cryptographic foundations — without consensus or networking:

  1. Block structure: Each block contains a list of transactions, a timestamp, the hash of the previous block, and a Merkle root of the transactions.
  2. Transaction structure: Each transaction contains a sender (public key), recipient, amount, and a digital signature (ECDSA/secp256k1).
  3. Validation: Implement transaction validation (signature verification, basic double-spend detection via UTXO or account model).
  4. Merkle proofs: Given a block, generate a Merkle proof for any transaction.
  5. Tamper detection: Demonstrate that modifying any historical transaction invalidates the chain.
  6. Serialization: Serialize/deserialize blocks and transactions to/from bytes (use RLP or a custom format).

This is not yet a working blockchain (no consensus, no networking) — it is the data layer that all subsequent stages will build upon.


Common Mistakes & Misconceptions

Mistake Reality
"SHA-256 encrypts data" Hash functions are one-way; they do not encrypt. There is no key and no decryption.
"Blockchain is immutable" Blockchain is tamper-evident, not tamper-proof. If an attacker gains sufficient consensus power, they can rewrite history.
"More nodes = more secure" Security depends on the distribution of stake/hashrate, not the raw number of nodes. 10,000 nodes run by one entity provide less security than 100 independent nodes.
"Consensus means everyone agrees" Consensus means a sufficient supermajority agrees. Minority nodes may disagree (leading to forks).
"P2P means no structure" Most P2P networks have significant structure: routing tables, peer scoring, topology management.
"CAP theorem means you pick two and lose one" The trade-off is nuanced and depends on the type of partition and the required consistency model.

Security Considerations

At this stage, develop the following security mindset:


Readings & Resources

Books

Papers

Documentation & Online Resources

Technical Blogs


Previous Home Page Stage 1 of 6 · Foundations Next 02 · Blockchain Core