Mastering Bitcoin: Programming the Open Blockchain - An Overview

An introduction to the Bitcoin protocol, great for understanding how Proof-of-Work works.

Bitcoin Protocol. Blockchain. Consensus Rules. Proof-of-Work.

'Mastering Bitcoin' might be the right book for you if you want a technical breakdown of how Bitcoin works beneath the surface. In this overview, I condense some of the highlights if you are looking for a quick digest, give you an outline to help you decide if you should read it, and share my own impressions.

I read this book because I had just written low-level Bitcoin wallet code, and I wanted to consolidate what I had learned. I chose this book specifically from a colleague's recommendation.

Although I did read it in print, one interesting aspect of this book is that it is actually an open-source project, so you can check it out here: https://github.com/bitcoinbook/bitcoinbook

Synopsys

Bitcoin is a cryptographic and economical solution for achieving state consensus on distributed systems without relying on trust. Essentially, it is a system that allows for peer-to-peer electronic value transfers without double-spend.

Modern implementations and usages of the protocol abstract out the low-level details of how it works and present users with concepts such as coins, balance, accounts, and addresses. In reality, the protocol operates over transactions for which the value is encumbered to be redeemed by digital signatures.

The record of these transactions is kept on a public ledger maintained by nodes in a distributed network of peers. Additionally, some of those nodes participate in a Proof-of-Work competition used to achieve and maintain the distributed consensus - colloquially referred to as mining.

Who Should Read this Book?

  • People from a technical background seeking an introduction to the Bitcoin protocol;
  • Developers that are about to write client-side bitcoin code, such as a wallet;
  • Miners seeking a deeper understanding of the network;
  • Developers that work in other chains such as ETH, to get the context of how different or similar the platforms are;
  • Developers that have already worked with the Bitcoin protocol as a way to consolidate empirical knowledge.

The Author

Andreas Antonopoulos is a Greek-British computer scientist.
He started out as a freelance consultant and, in 2012, started speaking at conferences about bitcoin.

Antonopoulos released the first edition of 'Mastering Bitcoin' in 2016.

Fun fact: in 2017 he received over 100 BTC in donations for his work within the community. Which you can read more about here.

Summary

Users don't need a central trusted authority to transact value in the Bitcoin system. No bank. No third party. Just peer-to-Peer value transfers. How is that possible?

The culprit is Satoshi Nakamoto, the illustrious inventor of Bitcoin, who came up with a new solution for the "Byzantine Generals' Problem." - a problem in distributed computing where generals must agree on a joint battle plan while communicating via unreliable messengers or in the presence of treacherous generals.

Bitcoin consists of: A decentralized peer-to-peer network (the bitcoin protocol) A public transaction ledger (the blockchain) A set of rules for independent transaction validation and currency issuance (consensus rules) A mechanism for reaching global decentralized consensus on the valid blockchain (Proof-of-Work algorithm)

― Andreas Antonopoulos, Mastering Bitcoin

The core foundational architecture of the Bitcoin system is a chain of transactions, where the output of one transaction is the input of the next. Sounds easy, right? The problem lies in determining which chain to agree on collectively and ensuring that the agreed-upon chain is valid - no double spend, etc. This state consensus of which chain of transactions matches the truth emerges from mining.

Mining is a process whereby specialized computers running bitcoin software (mining nodes), also known as miners, participate in a Proof-of-Work competition.

Each miner spends time and energy brute-forcing the solution to a cryptographic puzzle. When a miner finally succeeds in his effort, he increases the size of the chain of transactions (with transactions from users transacting value) and gets a reward. This reward is dependent on the rest of the network accepting the miner's "work" as valid, which means that the transactions he submitted must also be valid. If he tried to sneak a double-spend in there, all his hard work had been for naught since none of his peers will continue building on top of his contribution to the chain of transactions.

The term Blockchain comes from the increment of transactions that the miner appends to the public ledger, a Block.

Keys

A very little key will open a very heavy door.

― Charles Dickens

The Blockchain is pretty fascinating. However, the Bitcoin system is more than just the Blockchain. The user's software, which we would typically refer to as the client-side software, is equally important. This piece of software is usually called a Bitcoin Wallet. Yet, Wallets should really be called Keychains since they store keys, not coins.

Wallets store private keys that only the user has access to. Users instrument private keys to both digitally sign transactions and generate public keys and addresses via one-way cryptographic functions.

Usage of asymmetric cryptography to convert private keys into Addresses, the same type of 'trap-door' functions are also used to generate digital signatures from private keys

These private keys (k) are basically very large numbers, randomly selected, and virtually impossible to guess that permit the user to spend unspent transaction outputs (UTXOs) stored in the Blockchain.

A digital signature serves three purposes in bitcoin: authentication, nonrepudiation, and integrity.

― Andreas Antonopoulos, Mastering Bitcoin

Both deriving Public keys from Private keys and signing transactions requires the software to employ Elliptic Curve math.

Transactions

Transaction outputs contain a field that includes a kind of riddle, known as the locking script. If a user is able to solve this riddle (usually by providing an adequate signature), then he can use it as a transaction input in another transaction (spending the value therein). Each transaction input is signed independently.

{
  "version": 1,
  "locktime": 0,
  "vin": [
    {
      "txid": "7957a35fe64f80d234d76d83a2a8f1a0d8149a41d81de548f0a65a8a999f6f18",
      "vout": 0,
      "scriptSig" : "3045022100884d142d86652a3f47ba4746ec719bbfbd040a570b1deccbb6498c75c4ae24cb02204b9f039ff08df09cbe9f6addac960298cad530a863ea8f53982c09db8f6e3813[ALL] 0484ecc0d46f1918b30928fa0e4ed99f16a0fb4fde0735e7ade8416ab9fe423cc5412336376789d172787ec3457eee41c04f4938de5cc17b4a10fa336a8d752adf",
      "sequence": 4294967295
    }
  ],
  "vout": [
    {
      "value": 0.01500000,
      "scriptPubKey": "OP_DUP OP_HASH160 ab68025513c3dbd2f7b92a94e0581f5d50f654e7 OP_EQUALVERIFY OP_CHECKSIG"
    },
    {
      "value": 0.08450000,
      "scriptPubKey": "OP_DUP OP_HASH160 7f9b1a7fb68d60c536c2fd8aeaa53a8f3cc025a8 OP_EQUALVERIFY OP_CHECKSIG",
    }
  ]
}
Bitcoin transaction. vout are the transaction outputs. vin are the inputs. The scriptPubKey on the vouts will be unlocked with scriptSig of vin of a future transaction.

This locking script is written in Bitcoin Script, a Forth-like reverse-polish notation stack-based (LIFO) execution language. Every bitcoin validating note will execute the locking and unlocking script together to validate transactions.

The code example above is what most Bitcoin transactions look like, Pay-to-Public-Key-Hash or “P2PKH” script. We know this by looking at the locking script:

OP_DUP OP_HASH160 <Public Key Hash> OP_EQUALVERIFY OP_CHECKSIG

Which sets the conditions for the owner of the private key that matches that Public Key Hash to spend this output.

However, this is not the only kind of locking script, there's also Pay-to-Script-Hash (P2SH), which can specify complex unlocking scenarios. Multisignature Scripts are the most common example of P2SH Scripts, which might for example, only allow the escrow of funds when a certain number of signatures are provided.

Lightweight Nodes and Payment Channels

Not all validating nodes need to maintain a full copy of the Blockchain. Simple Payment Verification (SPV) lightweight nodes can validate transactions by using the Block headers and requesting branches of the merkle tree from other nodes. This means that nodes with modest hardware can validate transactions and are enough for most use cases. Of course, businesses that need faster and more secure validation might still prefer to run full nodes, but that is not necessary.

A merkle tree, or binary hash tree, is a way to summarize large sets of data. In Bitcoin, nodes construct merkle trees by hashing data from pairs of blocks recursively until there is only one hash - the merkle root.

Payment Channels is another technique that allows parties to perform transactions without access to a full node or even any node at all. With Payment Channels, two parties can transact value outside the Blockchain, by constructing a series of transactions where only two need to be submitted for mining on the Blockchain: the funding and settlement transactions.

Mining

Every node has a pool of valid yet unconfirmed transactions, also known as the mempool, Bitcoin’s ‘waiting room.’ When constructing blocks, miners grab transactions from this pool.

Mining itself is a relatively simple process. It involves hashing the block header repeatedly, changing a nonce, until the double SHA256 hashing function produces a smaller hash than the target set by the network. Essentially a hash that contains a certain number of leading zeros.

This target is adjusted such that new Blocks are mined roughly every 10 minutes. The target and difficulty are inversely related. This difficulty is the number of leading zeros the target has and, therefore, how hard it is for miners to produce a suitable hash.

Security

Another common mistake is to take transactions “off-blockchain” in a misguided effort to reduce transaction fees or accelerate transaction processing. An “off-blockchain” system will record transactions on an internal, centralized ledger and only occasionally synchronize them to the bitcoin blockchain.

― Andreas Antonopoulos, Mastering Bitcoin

Security holes come from two familiar places, managing users’ keys and processing transactions off-chain. For example, implementing a custodian wallet or a wallet that obfuscates derivation paths cutting users off from their keys opens the door to misplacement or misappropriation of keys. In the same way, processing transactions off-chain might break the consensus mechanism if the centralized ledger is out of sync with the decentralized public one.

Thus, when designing complex bitcoin applications, it’s vital to understand where trust is placed. If the application vests trust in anything but the blockchain, that is a possible point of failure.


My Review

I very much enjoyed reading 'Mastering Bitcoin.'

It's a technical book; it's complete and dives into the low-level technical aspects, such as how Script works.

Since I read it after writing bitcoin wallet software, it was funny and refreshing to see all the eureka moments written in print  - undoubtedly an excellent way to consolidate knowledge.

Although initially published in 2014, the book aged well aside from a couple of particularities:

  • Bitcoin values are high in the examples - Alice buys cups of coffee for $500. This is a fun reminder of how much BTC prices have surged and not bad per se.
  • Some application examples are outdated, like what libraries and wallet applications to use - It would have been better to avoid those kinds of references.

However, I actually liked that the book is almost a decade old. It does a better job at offering context. I also found the addition of the original bitcoin whitepaper in the appendices a nice touch.

One critique I can make concerns the content distribution across the different chapters. It felt like some chapters had a lot more juice than others. Namely, I enjoyed the chapters about Keys, Wallets, Transactions, and Mining a lot more than others that sort of felt like filler content. This made me feel like a book half the size that focuses on those core concepts would offer readers more value. However, this is not a fair point as it only comes about because those chapters are outstanding.

Another aspect I think could have been present is Bitcoin's historical context. However, perhaps that was intentional to avoid stepping into controversial territory and staying within scope. Again, this a technical, not a history book. Still, I think it would have been interesting to explore, for example, the history of the Bitcoin hard-forks and how early bitcoin development took place. I believe the content about segwit and Lightning Network, would be better if framed within its historical context. But again, that is probably another book of another genre altogether.

🍬 Keywords

Pay-to-Public-Key-Hash (P2PKH), encumbrance, asymmetric cryptography, digital signatures, elliptic curve multiplication, encryption, encoding, emergent consensus, hash, opcode, merkle tree, mempool, payment channels, Proof-of-Work