Introduction

Dapper is Google's distributed tracing system, which laid the foundation for subsequent open‑source implementations such as Zipkin and SkyWalking. Solving the problem of tracing in production environments requires not only a suitable data model but also careful consideration of low code invasiveness and performance impact after deployment, so that the system can be widely adopted and continuously monitored. Dapper's approach is:

• Use span as the basic data structure, modeling a trace as a tree with spans as nodes.
• Modify libraries and use instrumentation to achieve zero code intrusion.
• Employ various sampling techniques to improve performance.
• Build a comprehensive data collection and monitoring infrastructure.

This post explains these aspects in detail.

Introduction

Amazon Aurora is a cloud-native, storage-compute separated database that offers significantly superior performance and scalability compared to traditional RDBMS. The compute layer of Aurora is based on open-source MySQL code, but delegates its logging, storage, and recovery modules to the storage layer. Compute nodes send redo logs to storage nodes, which persist the logs using a quorum protocol, resembling the replicated state machine model of consensus algorithms. Aurora's innovative design reduces network I/O in cloud environments by an order of magnitude and drastically cuts down the time for expensive operations like recovery.

From a learning perspective, this paper is good. It progressively presents various problems and their solutions with strong logical flow. Furthermore, for each technical point, it first explains the RDBMS approach and its shortcomings before detailing Aurora's implementation. However, I found reading this paper somewhat challenging. First, it carries a distinct commercial promotional tone, containing many glossy marketing phrases, a few customer stories, etc. Second, most papers I've read are in the system area, and I'm less familiar with many database-specific terms.

This article will cover Aurora's performance, architecture, consistency principles, and various technical details.

Introduction

After MapReduce and GFS, I finally finished reading the last of Google's three foundational papers – Bigtable. Similar to Dynamo, many techniques described in the Bigtable paper have been widely adopted by various subsequent projects, so reading it felt familiar. This post will comprehensively discuss Bigtable's programming model, underlying storage, workflow, performance, and other aspects.

Programming Model

A Bigtable is a sparse, distributed, persistent multi-dimensional sorted map. The map is indexed by a row key, column key, and a timestamp;

1	(row:string, column:string, time:int64) -> string

This definition can be a bit tricky to grasp. Although Bigtable is a NoSQL database, it provides a semi-structured data model, allowing us to draw an analogy to an RDBMS. For a table, row can be thought of as the primary key for a row, column as the field name for a column, and time as a version number for this row. The mapped string value is the field value for that specific row and column.

Introduction

From a learning perspective, the Dynamo paper is well-written. Firstly, it explains many fundamental concepts in plain language, such as consistent hashing, vector clocks, and Merkle Trees. Secondly, its design is outstanding, offering much for study and reference. Finally, it further decouples and analyzes problems we often take for granted, deepening my understanding of the relevant knowledge.

This post will cover the following:

• Some fundamental concepts: Consistent Hashing, Vector Clocks, SLA (Service Level Agreement).
• Design considerations for Dynamo, including the "always writable" design goal and the problem of data conflict and its resolution.
• Dynamo's workflow: A brief introduction to its read and write processes, followed by a focus on the configuration and principles of the three parameters (N, R, W).
• Comparison of different partitioning strategies, primarily contrasting consistent hashing with fixed partitioning strategies.

Introduction

This blog post documents two issues I encountered in recent development:

• Route matching conflicts in FastAPI
• Nginx DNS resolution in Docker environments

Additionally, it includes some reflections after using Vue.

FastAPI Route Matching Conflicts

Problem Investigation

This project uses RBAC for permission control on each endpoint via fastapi.Depends. One endpoint required the roles [A, B]. When I accessed it with a user having role A, it returned a 403 error. I started debugging, only to find this endpoint actually only required [B]. No matter how I debugged, even removing the Depends method from the endpoint, the debugger still jumped to the role-checking method (requiring [B]), and the endpoint consistently returned 403.

Introduction

I recently studied the course "etcd 实战课", or "Practical etcd" in English. The author is truly an expert, getting straight to the point on many issues, which has been very insightful for me. etcd is a vast and complex system; explaining the principles of just one component like MVCC, Watch, or Lease would be impossible in a single blog post, let alone the entire etcd. Therefore, this post focuses on etcd's improvements to Raft. Specifically, we'll start from the algorithm presented in the Raft paper, using the MIT 6.824 lab 3 project (a simple K/V service built atop Raft) as a baseline, to see what optimizations etcd has made. This article covers the ReadIndex optimization for read operations, the PreCandidate mechanism to avoid unnecessary elections, and some other minor optimizations.

ReadIndex: Optimizing Read Operations

The approach of MIT 6.824's K/V service for handling read operations is the same as for writes: only the Leader can process the request. When a request is received, the Leader replicates the new log entry to Followers over the network. Once a majority of Followers have persisted this log entry, the Leader commits it, applies it to the state machine, and then returns the result to the client.

Introduction

Eric Brewer proposed the famous CAP theorem: it is impossible for a web service to simultaneously maintain all three of the following properties: Consistency, Availability, and Partition tolerance. The paper "Brewer's Conjecture and the Feasibility of Consistent, Available, Partition-Tolerant Web Services" provides a proof for this theorem.

As the theory spread, it seemingly morphed into a dogmatic conclusion, deviating from Eric Brewer's original intent. Later, he authored the article "CAP Twelve Years Later: How the 'Rules' Have Changed," which clarifies the misconceptions, applications, and handling of partition related to the CAP theorem.

I read these two papers earlier and gained significant insights. Recently, with continuous learning and practice, I found that these papers resolved many of my questions (some old, some new). Therefore, I wanted to promptly document these reflections. This article will primarily cover the proof of the CAP theorem, CP vs. AP, and the process for handling network partitions.

Introduction

The production server for a project I am working on is an Azure VM that's only accessible via an IP address, which couldn't establish HTTPS connections. This is a situation I found unsatisfactory. To enable HTTPS, we first need to register a domain name, then configure DNS resolution, and finally apply for an SSL certificate. The key to achieving this for free lies in obtaining a domain name at no cost, since Cloudflare offers free DNS resolution services and Let's Encrypt provides free SSL certificates. However, acquiring a free domain name is challenging, and Cloudflare might not recognize such free domains. Consequently, I kept postponing this task.

Recently, while conversing with an LLM again, I discovered that Azure allows us to claim a free domain name ([your-chosen-prefix].[vm-region].cloudapp.azure.com) for a VM's public IP address, and it also provides free DNS resolution services. This significantly simplified my task, as Azure had handled the most difficult part. I only needed to apply for the SSL certificate.

Thus, this blog post was born: how to apply for a free SSL certificate from Let's Encrypt. Note that the vast majority of this post's content was generated by an LLM. I successfully obtained the SSL certificate by following these instructions, so I decided to document them.