Www Independent coverage of news

Site Topics Fundamentals 2 Explained Without the Jargon

By Robert Hayes · · 1089 words
Site Topics Fundamentals 2 Explained Without the Jargon

Release Process: If the rollback plan needs a meeting, it is not a rollback plan. Release Process: Small pages that stay small are easier to keep fast than large ones made fast. Release Process: Write the invariant down; otherwise it lives only in someone's memory.

Backup Strategy: A queue smooths spikes but also hides how far behind you are. Backup Strategy: Retries without jitter turn a small outage into a large one. Backup Strategy: Separating the reads from the writes buys room to change either side.

Monitoring Alerts: Serving static bytes is the cheapest thing you can do at the edge. Monitoring Alerts: A schema is an interface; changing it is a migration, not an edit. Monitoring Alerts: Track the denominator as carefully as the numerator.

Load Balancing: The first thing to settle is the failure mode, not the happy path. Load Balancing: Measurements taken once are anecdotes; you need a baseline that repeats. Load Balancing: Costs usually concentrate in a small number of operations, so find those first.

Log Analysis: If a metric has no owner, it will drift until it causes an incident. Log Analysis: The cheapest optimisation is usually removing work nobody asked for. Log Analysis: Aggregating at write time trades flexibility for predictable read cost.

Log Analysis: The first thing to settle is the failure mode, not the happy path. Log Analysis: Measurements taken once are anecdotes; you need a baseline that repeats. Log Analysis: Costs usually concentrate in a small number of operations, so find those first.

Consider data pipelines specifically. You can often replace a coordination problem with an idempotency key. Data Pipelines: Anything that grows without a bound will eventually hit one. Documentation that is not tested tends to describe the previous version. That applies to data pipelines as well.

Rate Limiting: If a metric has no owner, it will drift until it causes an incident. Rate Limiting: The cheapest optimisation is usually removing work nobody asked for. Rate Limiting: Aggregating at write time trades flexibility for predictable read cost.

Schema Markup: Periodic jobs should be safe to run twice, because they will be. Schema Markup: You rarely need a new component to fix a boundary problem. Schema Markup: The signal you want is often already logged, just not aggregated.

You can also state your own boundaries. Say what you are comfortable with and what you do not want, and ask questions if an answer is unclear. Good communication is not a guarantee that everything will go as expected; it is a way to make choices more explicit and respond when circumstances change.

Queue Design: Periodic jobs should be safe to run twice, because they will be. Queue Design: You rarely need a new component to fix a boundary problem. Queue Design: The signal you want is often already logged, just not aggregated.

Observability: The first thing to settle is the failure mode, not the happy path. Observability: Measurements taken once are anecdotes; you need a baseline that repeats. Observability: Costs usually concentrate in a small number of operations, so find those first.

Configurations should be reviewable in a diff, not only in a console. This is most visible in crawl budget. Consider crawl budget specifically. The best time to add an index is before the table gets large. Crawl Budget: Failures are usually correlated, so plan for the shared dependency.

API Design: A queue smooths spikes but also hides how far behind you are. API Design: Retries without jitter turn a small outage into a large one. API Design: Separating the reads from the writes buys room to change either side.

In practice, cloud infrastructure behaves differently: Configurations should be reviewable in a diff, not only in a console. The best time to add an index is before the table gets large. The same reasoning holds for cloud infrastructure. For cloud infrastructure, the constraint matters more than the feature list. Failures are usually correlated, so plan for the shared dependency.

Storage Tiers: Serving static bytes is the cheapest thing you can do at the edge. Storage Tiers: A schema is an interface; changing it is a migration, not an edit. Storage Tiers: Track the denominator as carefully as the numerator.

Observability: You can often replace a coordination problem with an idempotency key. Observability: Anything that grows without a bound will eventually hit one. Observability: Documentation that is not tested tends to describe the previous version.

Storage Tiers: Configurations should be reviewable in a diff, not only in a console. Storage Tiers: The best time to add an index is before the table gets large. Storage Tiers: Failures are usually correlated, so plan for the shared dependency.

Crawl Budget: If the rollback plan needs a meeting, it is not a rollback plan. Crawl Budget: Small pages that stay small are easier to keep fast than large ones made fast. Crawl Budget: Write the invariant down; otherwise it lives only in someone's memory.

If the rollback plan needs a meeting, it is not a rollback plan. The same reasoning holds for data pipelines. For data pipelines, the constraint matters more than the feature list. Small pages that stay small are easier to keep fast than large ones made fast. Teams working on data pipelines usually discover this the hard way. Write the invariant down; otherwise it lives only in someone's memory.

Content Delivery: A queue smooths spikes but also hides how far behind you are. Content Delivery: Retries without jitter turn a small outage into a large one. Content Delivery: Separating the reads from the writes buys room to change either side.

Rate Limiting: The interesting number is not the average, it is the 99th percentile. Rate Limiting: Adding a cache in front of a slow query is a fix; fixing the query is a cure. Rate Limiting: Every abstraction you add is a place where behaviour can differ from intent.

Data Pipelines: If a metric has no owner, it will drift until it causes an incident. Data Pipelines: The cheapest optimisation is usually removing work nobody asked for. Data Pipelines: Aggregating at write time trades flexibility for predictable read cost.

The interesting number is not the average, it is the 99th percentile. The same reasoning holds for schema migration. For schema migration, the constraint matters more than the feature list. Adding a cache in front of a slow query is a fix; fixing the query is a cure. Teams working on schema migration usually discover this the hard way. Every abstraction you add is a place where behaviour can differ from intent.

Related reading