Prateek Codes - Building Scalable Backend Systems

Stop Pasting Schema Into Your AI: Connect PostgreSQL Directly with MCP

Sun, 05 Apr 2026 00:00:00 +0000

Agentic coding tools have gotten good at reading your codebase. Claude Code will find your schema.rb, Cursor will pick up your Prisma schema, and most tools know how to navigate ORM-based projects well enough to understand your data model structurally.

What they can’t do is reason about your actual data - and that gap matters more than most developers realize.

When you ask an AI to help design a new feature, it’s working from structure alone. It knows what columns exist, not how they’re used. It doesn’t know that your notifications table has 400M rows and any fan-out design will be a problem. It doesn’t know that 80% of your users have never set a preferences value, which changes how you’d model the feature. It doesn’t know whether a background job is necessary or whether the data volume makes a synchronous approach fine. These are the tradeoffs that determine whether a feature ships well or causes incidents - and without live data access, the AI is guessing.

The same gap shows up in debugging. When a query is slow, the AI has to ask you to run EXPLAIN ANALYZE and paste the results. When it needs row counts or value distributions, it asks you to run those too. You become the relay between the AI and your database.

Model Context Protocol (MCP) is an open standard that lets AI tools connect to external systems and query them directly. With a Postgres MCP server configured, your AI can inspect live data volumes, run EXPLAIN ANALYZE, check value distributions, and make recommendations grounded in what’s actually in your database.

The Problem

Here’s what designing a new feature looks like without MCP:

You: “I want to add a real-time notification feed for users.”

AI: “How many users do you have, and roughly how many notifications per user per day?”

You: (checks, pastes numbers)

AI: “And what does the current notifications table look like? How many rows total?”

You: (runs SELECT COUNT(*), pastes result)

AI: “What’s the read pattern - are users mostly reading recent notifications or querying by type?”

You: (runs a distribution query, pastes result)

The AI knows your schema. It has no idea what’s in it. Every data-dependent decision becomes a round-trip where you run the query and report back. For a non-trivial feature this happens five or ten times before you get a recommendation you can act on.

With a Postgres MCP Server

Once connected, the same conversation looks like this:

You: “I want to add a real-time notification feed for users.”

AI: (checks notifications table row count, queries user and notification distribution, inspects existing indexes)

AI: “You have 400M notification rows across 2.1M users, with a long tail - the top 5% of users account for 60% of rows. A naive fan-out on write will be a problem at this scale. I’d recommend a pull-based feed with a covering index on (user_id, created_at DESC) rather than pre-computing feeds. Here’s the schema and the background job design.”

No relay. The AI queries your database when it needs data and gives you a recommendation grounded in actual numbers. The same applies to debugging - understanding query plans becomes something the AI does directly rather than something you transcribe into the conversation.

Setting Up @modelcontextprotocol/server-postgres

The official Postgres MCP server is @modelcontextprotocol/server-postgres. The source repository was archived in May 2025, but the npm package remains functional and is the most straightforward way to get started.

Step 1: Create a Read-Only Database User

Never connect your AI tool using the same credentials your application uses. Create a dedicated user with read-only access:

-- Create a dedicated user for AI access
CREATE USER ai_readonly WITH PASSWORD 'your-secure-password';

-- Grant connect on the database
GRANT CONNECT ON DATABASE your_database TO ai_readonly;

-- Grant schema usage
GRANT USAGE ON SCHEMA public TO ai_readonly;

-- Grant read-only access to all current tables
GRANT SELECT ON ALL TABLES IN SCHEMA public TO ai_readonly;

-- Ensure future tables are also covered
ALTER DEFAULT PRIVILEGES IN SCHEMA public
  GRANT SELECT ON TABLES TO ai_readonly;

This limits blast radius significantly. Worth noting: the MCP server also runs all queries inside a READ ONLY transaction - the README states this explicitly - so it refuses mutations at the server level regardless of user permissions. The read-only DB user is a second, independent layer of protection. Both should be in place.

Step 2: Configure Claude Desktop or Claude Code

For Claude Desktop, edit ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "postgres": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-postgres",
        "postgresql://ai_readonly:your-secure-password@localhost:5432/your_database"
      ]
    }
  }
}

For Claude Code, open ~/.claude/settings.json and add the same mcpServers block:

{
  "mcpServers": {
    "postgres": {
      "command": "npx",
      "args": [
        "-y",
        "@modelcontextprotocol/server-postgres",
        "postgresql://ai_readonly:your-secure-password@localhost:5432/your_database"
      ]
    }
  }
}

Restart Claude after saving.

Step 3: Verify the Connection

Ask Claude: “What tables are in my database?”

If it responds with your actual schema, you’re connected. The server exposes two capabilities to the AI:

Schema inspection - lists tables, columns, data types, constraints, and indexes
Query execution - runs SQL and returns results into the conversation context

That’s enough for the AI to write accurate migrations, suggest indexes based on actual table sizes, and debug slow queries by running EXPLAIN ANALYZE itself.

Other MCP Database Options

The official Postgres server works well for most local setups, but depending on your database host or what you need the AI to do, there are more capable alternatives.

Neon MCP is worth using if you’re on Neon’s serverless Postgres. It’s actively maintained, supports both read and write operations through proper authorization, and integrates branch management. This makes it practical for letting the AI help apply migrations against a staging branch without touching production.

Supabase MCP ships as part of Supabase’s tooling and gives the AI access to your project’s tables, schema, and Row Level Security policies. Useful if your authorization logic lives in the database.

SQLite MCP (@modelcontextprotocol/server-sqlite) is the equivalent for local SQLite databases.

For cloud-hosted databases - RDS, Azure Database, Cloud SQL - the configuration is identical to the local example above. If you’re on a primary-replica setup, point the connection string at the replica rather than the primary.

Security

Use a Read-Only Connection

Use a dedicated database user with SELECT-only privileges, as shown in Step 1. The MCP server’s built-in transaction protection is a safety net, not a substitute for database-level access control.

Zero Data Retention for Sensitive Databases

If your database contains PII, financial records, or anything you’d classify as sensitive, pay attention to your AI provider’s data retention policy. When the AI queries your database through MCP, the results - including actual row data - pass through the conversation context. That context may be retained by default.

For sensitive workloads:

Enable zero data retention (ZDR) through Anthropic’s API if it’s available on your plan. With ZDR enabled, prompts and outputs aren’t stored or used for model training.
Avoid querying raw PII through the AI - have the AI write the query, review it yourself, then run it outside the AI context.
Mask sensitive columns using a view that exposes only what the AI needs:

-- Expose a masked version of the users table
CREATE VIEW public.users_masked AS
SELECT
  id,
  created_at,
  updated_at,
  role,
  subscription_plan,
  '***' AS email,
  '***' AS phone_number
FROM users;

-- Grant access to the masked view only
GRANT SELECT ON public.users_masked TO ai_readonly;
REVOKE SELECT ON public.users FROM ai_readonly;

The AI still has enough context to answer questions about user behavior, subscription distribution, and counts - without seeing the actual values.

What Changes Day to Day

Once the MCP server is running, the AI can:

Understand your full schema without you explaining it
Suggest indexes based on actual table sizes and existing constraints
Write migrations that account for real foreign keys and default values
Debug slow queries by running EXPLAIN ANALYZE itself
Answer data questions like “how many orders shipped last week?” directly

The manual loop of pasting schema, waiting for a clarifying question, then pasting more schema mostly disappears.

Conclusion

The Postgres MCP server takes under ten minutes to set up, runs all queries in read-only transactions, and removes a class of manual work that compounds fast. Start with a dedicated read-only DB user, be deliberate about what data flows through the AI’s context, and you have a setup that’s practical and secure.

References

Rails 8.2 adds this_week?, this_month?, and this_year? to Date and Time

Sat, 04 Apr 2026 00:00:00 +0000

ActiveSupport already has today?, yesterday?, and tomorrow? on Date and Time. Rails 8.2 adds the next logical set: this_week?, this_month?, and this_year?.

Before

Checking whether a date falls within the current week, month, or year required range comparisons:

# Is this order from the current week?
order.placed_at.between?(Time.current.beginning_of_week, Time.current.end_of_week)

# Is this subscription expiring this month?
subscription.expires_on.between?(Date.current.beginning_of_month, Date.current.end_of_month)

# Is this event happening this year?
event.date.year == Date.current.year

Each check is readable enough on its own, but they add up quickly in controllers and views where you are branching on date ranges.

Rails 8.2

PR #55770 introduces three new predicate methods on Date and Time:

order.placed_at.this_week?           # true if within the current week
subscription.expires_on.this_month?  # true if within the current month
event.date.this_year?                # true if within the current year

They follow the same pattern as the existing predicates:

Date.current.this_week?   # => true
Date.current.this_month?  # => true
Date.current.this_year?   # => true

Date.yesterday.this_week? # => true (yesterday is still this week, usually)
Date.current.next_month.this_month? # => false

In controllers

def index
  @orders = Order.all
  @this_week_orders = @orders.select { |o| o.placed_at.this_week? }
  @this_month_orders = @orders.select { |o| o.placed_at.this_month? }
end

In views

<% if subscription.expires_on.this_month? %>
  <div class="warning">Your subscription expires this month.</div>
<% end %>

<% if report.generated_at.this_week? %>
  <span class="badge">Recent</span>
<% end %>

In scopes

class Order < ApplicationRecord
  scope :placed_this_week, -> { where(placed_at: Time.current.beginning_of_week..Time.current.end_of_week) }
  scope :placed_this_month, -> { where(placed_at: Time.current.beginning_of_month..Time.current.end_of_month) }
end

The predicate methods on instances complement these scopes when working with already-loaded records.

How to change the week boundary

this_week? uses Monday as the start of the week by default, consistent with ActiveSupport’s beginning_of_week. If your application configures a different week start, that is respected:

Date.beginning_of_week = :sunday
Date.current.beginning_of_week  # => last Sunday

Conclusion

this_week?, this_month?, and this_year? are small additions that remove a common category of boilerplate. They complete the set of readable date predicates that ActiveSupport has offered since Rails 3.

References

Rails 8.2 lets retry_on read the error when calculating wait time

Wed, 01 Apr 2026 00:00:00 +0000

Active Job’s retry_on accepts a wait: proc for custom backoff logic. Before Rails 8.2, that proc only received the execution count. When a remote API returns a Retry-After header, there was no way to use that value inside the proc. Rails 8.2 fixes this by passing the exception as a second argument.

Before

The wait: proc only knew how many times the job had been attempted:

class PaymentSyncJob < ApplicationJob
  retry_on Stripe::RateLimitError,
           attempts: 5,
           wait: ->(executions) { executions * 10 }

  def perform(order_id)
    Stripe::Charge.retrieve(order_id)
  end
end

If Stripe responded with a Retry-After: 30 header, the job ignored it. The wait time was always based on the execution count, regardless of what the API actually asked for.

To work around this, teams typically stored retry delay information on the exception class itself and then retrieved it through other means, which added boilerplate and coupling.

Rails 8.2

PR #56601 allows the wait: proc to accept the exception as a second argument. Rails checks the proc’s arity, so existing one-argument procs continue to work without any changes.

class PaymentSyncJob < ApplicationJob
  retry_on Stripe::RateLimitError,
           attempts: 5,
           wait: ->(executions, error) { error.retry_after || executions * 10 }

  def perform(order_id)
    Stripe::Charge.retrieve(order_id)
  end
end

When the job retries, it calls the proc with both the execution count and the exception. If the error has a retry_after value, that gets used. Otherwise, it falls back to the execution-based formula.

This works for any error class that exposes delay information:

class ExternalApiJob < ApplicationJob
  retry_on ApiRateLimitError,
           attempts: 10,
           wait: ->(executions, error) do
             # Use the header value if available, cap at 5 minutes
             [error.retry_after || executions ** 2, 300].min
           end

  def perform(resource_id)
    ExternalApi.fetch(resource_id)
  end
end

Backward Compatibility

The change is fully backward compatible. A proc with one argument behaves exactly as before:

# Still works, receives only executions
retry_on SomeError, wait: ->(executions) { executions * 5 }

# New behavior, receives both
retry_on SomeError, wait: ->(executions, error) { error.retry_after || executions * 5 }

Rails uses Ruby’s arity to determine which form the proc uses and calls it accordingly.

When to Use This

Use the two-argument form when:

The API you call returns a Retry-After header or equivalent
Your error class already captures the suggested wait time
You want backoff logic that adapts to what the remote service requests rather than using a fixed formula

Conclusion

Rails 8.2 makes retry logic more accurate for jobs that talk to rate-limited APIs. By exposing the exception to the wait: proc, jobs can respect what the remote service actually asks for instead of guessing.

References

Ruby::Box Practical Guide: Use Cases and Integration Patterns (Part 2)

Thu, 15 Jan 2026 00:00:00 +0000

In Part 1, we covered what Ruby::Box is and how it provides namespace isolation. Now let’s explore practical patterns for integrating it into real applications.

Use Case: Plugin Systems

Plugin systems benefit significantly from Ruby::Box. Each plugin runs in its own isolated environment, preventing plugins from interfering with each other or the host application.

class PluginManager
  def initialize
    @plugins = {}
  end

  def load_plugin(name, path)
    box = Ruby::Box.new
    box.require(path)

    # Access the plugin class from within the box
    plugin_class = box.eval('Plugin')
    @plugins[name] = {
      box: box,
      instance: plugin_class.new
    }
  end

  def run(name, method, *args)
    plugin = @plugins[name]
    plugin[:instance].public_send(method, *args)
  end

  def unload(name)
    @plugins.delete(name)
    # Box becomes eligible for garbage collection
  end
end

# Usage
manager = PluginManager.new
manager.load_plugin(:markdown, './plugins/markdown_plugin')
manager.load_plugin(:syntax_highlight, './plugins/syntax_plugin')

# Each plugin has its own isolated environment
# If markdown_plugin patches String, syntax_plugin won't see it
manager.run(:markdown, :process, content)

This pattern ensures that a misbehaving plugin cannot corrupt the global namespace or break other plugins.

Use Case: Multi-Tenant Configuration

Applications serving multiple tenants often need per-tenant configurations. Ruby::Box provides clean isolation without complex scoping logic.

class TenantContext
  def initialize(tenant_id, config_path)
    @tenant_id = tenant_id
    @box = Ruby::Box.new
    @box.require(config_path)
  end

  def config
    @box.eval('TenantConfig')
  end

  def execute(code)
    @box.eval(code)
  end
end

# Each tenant gets isolated configuration
tenant_a = TenantContext.new('acme', './tenants/acme/config')
tenant_b = TenantContext.new('globex', './tenants/globex/config')

tenant_a.config.theme      # => "dark"
tenant_b.config.theme      # => "light"

# Global variables are isolated too
tenant_a.execute('$rate_limit = 100')
tenant_b.execute('$rate_limit = 500')

tenant_a.execute('$rate_limit')  # => 100
tenant_b.execute('$rate_limit')  # => 500

Use Case: Running Multiple Gem Versions

During migrations, you might need to run two versions of the same gem simultaneously. Ruby::Box makes this possible without separate processes.

# Load v1 API client in one box
v1_box = Ruby::Box.new
v1_box.eval <<~RUBY
  $LOAD_PATH.unshift('./vendor/api_client_v1/lib')
  require 'api_client'
RUBY

# Load v2 API client in another box
v2_box = Ruby::Box.new
v2_box.eval <<~RUBY
  $LOAD_PATH.unshift('./vendor/api_client_v2/lib')
  require 'api_client'
RUBY

# Compare behavior during migration
def compare_responses(endpoint, params)
  code = "ApiClient.get('#{endpoint}', #{params.inspect})"
  v1_response = v1_box.eval(code)
  v2_response = v2_box.eval(code)

  if v1_response != v2_response
    log_difference(endpoint, v1_response, v2_response)
  end

  v1_response  # Return v1 for now, switch to v2 when ready
end

Use Case: Isolated Monkey Patches for Testing

Some tests require monkey patches that would pollute the global namespace. Ruby::Box keeps these contained.

# test_helper.rb
def create_time_frozen_box(frozen_time)
  box = Ruby::Box.new
  box.eval <<~RUBY
    class Time
      def self.now
        Time.new(#{frozen_time.year}, #{frozen_time.month}, #{frozen_time.day})
      end
    end
  RUBY
  box
end

# In your test
def test_subscription_expiry
  box = create_time_frozen_box(Time.new(2026, 1, 1))

  # Load and test code within the frozen-time box
  box.eval <<~RUBY
    expiry_date = Time.new(2025, 12, 31)
    subscription = Subscription.new(expires_at: expiry_date)
    raise "Expected expired" unless subscription.expired?
  RUBY

  # Time.now is unchanged outside the box
  Time.now  # => Current actual time
end

Use Case: Shadow Testing

Run new code paths alongside production code to compare results without affecting users. This pattern is useful for validating refactors or new implementations.

class ShadowRunner
  def initialize(production_box, shadow_box)
    @production = production_box
    @shadow = shadow_box
  end

  def run(method, *args)
    code = "#{method}(#{args.map(&:inspect).join(', ')})"

    # Production path returns the result
    production_result = @production.eval(code)

    # Shadow path runs asynchronously, logs differences
    Thread.new do
      shadow_result = @shadow.eval(code)

      unless production_result == shadow_result
        Logger.warn("Shadow mismatch for #{method}",
          production: production_result,
          shadow: shadow_result
        )
      end
    end

    production_result
  end
end

Working Around Native Extension Issues

Native extensions may fail to install with RUBY_BOX=1 enabled. The solution is to separate installation from execution:

# Gemfile installation without Boxing
bundle install

# Application execution with Boxing
RUBY_BOX=1 bundle exec ruby app.rb

For CI/CD pipelines:

# .github/workflows/test.yml
jobs:
  test:
    steps:
      - name: Install dependencies
        run: bundle install

      - name: Run tests with Ruby::Box
        run: RUBY_BOX=1 bundle exec rspec
        env:
          RUBY_BOX: "1"

Working Around ActiveSupport Issues

Some ActiveSupport core extensions have compatibility issues. Load them in your main context before creating boxes:

# At application startup, before creating any boxes
require 'active_support/core_ext/string/inflections'
require 'active_support/core_ext/hash/keys'

# Now create boxes for isolated code
plugin_box = Ruby::Box.new
# Plugins can use the already-loaded extensions

Alternatively, selectively load only what you need inside boxes:

box = Ruby::Box.new
box.eval <<~RUBY
  # Load specific extensions that are known to work
  require 'active_support/core_ext/object/blank'
RUBY

Performance Considerations

Ruby::Box adds minimal overhead for most operations:

Method dispatch: Slightly more indirection through separate method tables
Object creation: Unaffected, objects pass freely between boxes
Memory: Each box maintains its own class/module definitions

For performance-critical paths, cache class references:

class OptimizedPluginRunner
  def initialize(box)
    @box = box
    # Cache the class reference once
    @processor_class = box.eval('DataProcessor')
  end

  def process(data)
    # Use cached reference instead of evaluating each time
    @processor_class.new.process(data)
  end
end

When to Use `Ruby::Box`

Good candidates:

Plugin or extension systems where isolation is critical
Multi-tenant applications with per-tenant customizations
Testing scenarios requiring invasive monkey patches
Gradual migration between gem versions
Applications loading third-party code that might conflict

Poor candidates:

Running untrusted or potentially malicious code (use OS-level sandboxing)
Production systems until the feature stabilizes
Applications heavily dependent on native extensions
Simple applications without isolation requirements

Migration Strategy

If you’re considering Ruby::Box for an existing application:

Step 1: Test compatibility

# Run your test suite with Boxing enabled
RUBY_BOX=1 bundle exec rspec

Step 2: Identify issues

Look for failures related to:

Shared global state across files
Assumptions about class modifications being visible everywhere
Native extension loading errors

Step 3: Refactor incrementally

Start with isolated subsystems that don’t share state with the rest of your application. Move more code into boxes as you gain confidence.

Step 4: Monitor in staging

Run your staging environment with RUBY_BOX=1 before considering production deployment.

What’s Next for `Ruby::Box`

The Ruby core team has discussed building a higher-level “packages” API on top of Ruby::Box. This would provide more ergonomic ways to manage gem isolation without manual box management. Track progress in Ruby Issue #21681.

Ruby::Box solves real problems around namespace pollution and gem conflicts. While still experimental, it’s worth exploring for applications where isolation matters. Start with non-critical paths, understand the limitations, and provide feedback to the Ruby core team as you experiment.

References

Ruby 4.0 Introduces Ruby::Box for In-Process Isolation (Part 1)

Wed, 14 Jan 2026 00:00:00 +0000

Ruby 4.0 introduces Ruby::Box, a feature that provides isolated namespaces within a single Ruby process. This solves a long-standing problem: monkey patches and global modifications from one gem affecting all other code in your application.

The Problem with Shared Namespaces

When you load a gem that modifies core classes, those changes affect everything in your Ruby process:

# Gem A adds a titleize method to String
class String
  def titleize
    split.map(&:capitalize).join(' ')
  end
end

# Now EVERY piece of code in your process sees this method
# Including Gem B, which might have its own expectations

"hello world".titleize  # => "Hello World"

This becomes problematic when:

Two gems define conflicting methods on the same class
A gem’s monkey patch breaks another library’s assumptions
You want to test code in isolation from invasive patches
You need to run multiple versions of a gem simultaneously

Before Ruby 4.0, the only solutions were separate Ruby processes (with IPC overhead) or containers (with even more overhead).

Ruby 4.0: Enter Ruby::Box

Ruby::Box creates isolated spaces where code runs with its own class definitions, constants, and global variables. Changes made inside a box stay inside that box.

# Enable with environment variable at startup
# RUBY_BOX=1 ruby my_script.rb

# Check if Boxing is available
Ruby::Box.enabled?  # => true

# Create an isolated box
box = Ruby::Box.new

# Load code that patches String
box.eval <<~RUBY
  class String
    def shout
      upcase + "!!!"
    end
  end
RUBY

# The patch exists only inside the box
box.eval('"hello".shout')  # => "HELLO!!!"

# Outside the box, String is unchanged
"hello".shout  # => NoMethodError: undefined method `shout'

Understanding Box Types

Ruby::Box operates with three types of boxes:

Root Box: Contains all built-in Ruby classes and modules. This is established before any user code runs and serves as the template for other boxes.

Main Box: Your application’s default execution context. It’s automatically created from the root box when the process starts. This is where your main script runs.

User Boxes: Custom boxes you create with Ruby::Box.new. Each is copied from the root box, giving it a clean slate of built-in classes without any modifications from the main box or other user boxes.

# Your script runs in the "main" box
Ruby::Box.current  # => #<Ruby::Box main>

# Create isolated boxes
plugin_box = Ruby::Box.new
another_box = Ruby::Box.new

# Each box is independent
plugin_box.object_id != another_box.object_id  # => true

The Ruby::Box API

The API is straightforward with just a few methods:

# Creation
box = Ruby::Box.new

# Loading code
box.require('some_library')        # Respects box's $LOAD_PATH
box.require_relative('./my_file')  # Relative to current file
box.load('script.rb')              # Direct file execution

# Executing code
box.eval('1 + 1')                  # Execute Ruby code as string

# Inspection
Ruby::Box.current    # Returns the currently executing box
Ruby::Box.enabled?   # Check if Boxing is active

What Gets Isolated

Ruby::Box isolates several aspects of the Ruby runtime:

Classes and Constants: Reopening a built-in class in one box doesn’t affect other boxes.

box = Ruby::Box.new
box.eval <<~RUBY
  class Array
    def sum_squares
      map { |n| n ** 2 }.sum
    end
  end
RUBY

box.eval('[1, 2, 3].sum_squares')  # => 14
[1, 2, 3].sum_squares              # => NoMethodError

Global Variables: Changes to globals stay within the box.

box = Ruby::Box.new
box.eval('$my_config = { debug: true }')

box.eval('$my_config')  # => { debug: true }
$my_config              # => nil

Top-Level Methods: Methods defined at the top level become private instance methods of Object within that box only.

box = Ruby::Box.new
box.eval <<~RUBY
  def helper_method
    "I'm only available in this box"
  end
RUBY

box.eval('helper_method')  # => "I'm only available in this box"
helper_method              # => NoMethodError

Enabling Ruby::Box

Ruby::Box is disabled by default. Enable it by setting the RUBY_BOX environment variable before the Ruby process starts:

RUBY_BOX=1 ruby my_application.rb

Important: Setting RUBY_BOX after the process has started has no effect. The boxing infrastructure must be initialized during Ruby’s boot sequence, so the variable must be set before the Ruby process starts.

# This check should be at the top of your application
unless Ruby::Box.enabled?
  warn "Ruby::Box is not enabled. Start with RUBY_BOX=1"
  exit 1
end

Important Limitations

Before adopting Ruby::Box, be aware of these constraints:

Not a Security Sandbox: Ruby::Box provides namespace isolation, not security isolation. Code in a box can still access the filesystem, network, and system resources. Do not use it to run untrusted code.

Native Extensions: Installing gems with native extensions may fail when RUBY_BOX=1 is set. The workaround is to install gems without the flag, then run your application with it enabled.

# Install gems normally
bundle install

# Run with Boxing enabled
RUBY_BOX=1 bundle exec ruby app.rb

ActiveSupport Compatibility: Some parts of active_support/core_ext have compatibility issues with Ruby::Box. Load ActiveSupport in your main context before creating boxes if needed.

Experimental Status: This feature is experimental in Ruby 4.0. Behavior may change in future versions. The Ruby core team recommends experimentation but advises caution in production environments.

File Scope Execution

One important detail: Ruby::Box operates on a file-scope basis. Each .rb file executes entirely within a single box. Once loaded, all methods and procs defined in that file operate within their originating box, regardless of where they’re called from.

# helper.rb
def process(data)
  # This method always runs in the box where helper.rb was loaded
  data.transform
end

# main.rb
box = Ruby::Box.new
box.require_relative('helper')

# Even when called from main, process() runs in box's context
box.eval('process(my_data)')

Ruby::Box brings a long-requested capability to Ruby: proper namespace isolation without process boundaries. In Part 2, we’ll explore practical use cases including plugin systems, multi-tenant configurations, and strategies for gradual adoption.

References

Rails 8.2 makes enqueue_after_transaction_commit the default

Wed, 31 Dec 2025 00:00:00 +0000

Rails 7.2 introduced enqueue_after_transaction_commit to prevent race conditions when jobs are enqueued inside database transactions. However, it required explicit opt-in. Rails 8.2 flips the default. Jobs are now automatically deferred until after the transaction commits.

The Problem with Opt-In

With the opt-in approach in Rails 7.2, teams had to remember to enable the feature:

config.active_job.enqueue_after_transaction_commit = :default

Or configure it per-job:

class WelcomeEmailJob < ApplicationJob
  self.enqueue_after_transaction_commit = :always
end

This created inconsistency. Some jobs would be transaction-aware, others would not. The safer behavior required explicit action.

Rails 8.2 Changes the Default

PR #55788 changes this. When you upgrade to Rails 8.2 and run load_defaults "8.2", jobs are automatically deferred until after the transaction commits.

def create
  User.transaction do
    user = User.create!(params)
    WelcomeEmailJob.perform_later(user)  # Deferred until commit
  end
end

No configuration needed. The job waits for the transaction to complete before being dispatched to the queue.

Opting Out

If you need immediate enqueueing for backward compatibility or specific use cases, you have two options.

Global configuration:

config.active_job.enqueue_after_transaction_commit = false

Per-job configuration:

class TimeStampedJob < ApplicationJob
  self.enqueue_after_transaction_commit = false
end

Why the Global Config Was Restored

The global configuration option has an interesting history. It was deprecated and removed in Rails 8.1. The team initially wanted each job to declare its own preference. However, changing the default behavior without a global opt-out would break existing applications.

The PR restored the global configuration specifically to allow apps upgrading to Rails 8.2 to maintain their existing behavior without modifying every job class.

When This Matters

The new default primarily affects jobs enqueued to external queues like Redis (Sidekiq, Resque). If you use a database-backed queue like Solid Queue or GoodJob with the same database, your jobs are already part of the same transaction.

Jobs that do not depend on transaction data can still be configured for immediate enqueueing if needed.

Conclusion

Rails 8.2 makes the safer behavior the default. Jobs enqueued inside transactions automatically wait for the commit, eliminating a common source of race conditions without requiring explicit configuration.

References

Pull Request #55788 making this the default
Pull Request #51426 introducing the feature in Rails 7.2
Rails 7.2 enqueue_after_transaction_commit - detailed explanation of the feature

Rails 7.2 adds enqueue_after_transaction_commit to prevent job race conditions

Tue, 30 Dec 2025 00:00:00 +0000

Scheduling background jobs inside database transactions is a common anti-pattern which is a source of several production bugs in Rails applications. The job can execute before the transaction commits, leading to RecordNotFound or ActiveJob::DeserializationError because the data it needs does not exist yet. Or worse, the job could run assuming the txn would commit, but it rolls back at a later stage. We don’t need that kind of optimism.

Rails 7.2 addresses this with enqueue_after_transaction_commit, which automatically defers job enqueueing until the transaction completes.

Before

Consider a typical pattern where you create a user and send a welcome email:

class UsersController < ApplicationController
  def create
    User.transaction do
      @user = User.create!(user_params)
      WelcomeEmailJob.perform_later(@user)
    end
  end
end

This code works fine in development where your job queue is slow and transactions commit quickly. In production, with a fast Redis-backed queue like Sidekiq and a busy database, the job can start executing before the transaction commits:

Timeline:
1. Transaction begins
2. User INSERT executes (not committed yet)
3. Job enqueued to Redis
4. Sidekiq picks up job immediately
5. Job tries to find User -> RecordNotFound!
6. Transaction commits (too late)

The same problem occurs with after_create callbacks in models:

class Project < ApplicationRecord
  after_create -> { NotifyParticipantsJob.perform_later(self) }
end

The Workaround

The standard fix was to use after_commit callbacks instead:

class Project < ApplicationRecord
  after_create_commit -> { NotifyParticipantsJob.perform_later(self) }
end

Or wrap job scheduling in explicit after_commit blocks:

class UsersController < ApplicationController
  def create
    User.transaction do
      @user = User.create!(user_params)

      ActiveRecord::Base.connection.after_transaction_commit do
        WelcomeEmailJob.perform_later(@user)
      end
    end
  end
end

This worked but had problems:

Easy to forget: Using after_create instead of after_create_commit is a common mistake
Scattered logic: Job scheduling gets coupled to model callbacks instead of staying in controllers or service objects
Verbose: Wrapping every perform_later call in after_commit blocks adds boilerplate
Testing friction: Transaction callbacks behave differently in test environments using database cleaner with transactions

The after_commit_everywhere gem became popular specifically to address this problem. It lets you use after_commit callbacks anywhere in your application, not just in ActiveRecord models:

class UserRegistrationService
  include AfterCommitEverywhere

  def call(params)
    User.transaction do
      user = User.create!(params)

      after_commit do
        WelcomeEmailJob.perform_later(user)
      end
    end
  end
end

The gem hooks into ActiveRecord’s transaction lifecycle and ensures callbacks only fire after the outermost transaction commits. It handled nested transactions correctly and became a go-to solution for service objects that needed transaction-safe job scheduling.

Some teams built their own lightweight wrappers instead:

# Custom AsyncRecord class that hooks into transaction callbacks
class AsyncRecord
  def initialize(&block)
    @callback = block
  end

  def has_transactional_callbacks?
    true
  end

  def committed!(*)
    @callback.call
  end

  def rolledback!(*)
    # Do nothing if transaction rolled back
  end
end

# Usage
User.transaction do
  user = User.create!(params)
  record = AsyncRecord.new { WelcomeEmailJob.perform_later(user) }
  user.class.connection.add_transaction_record(record)
end

Both approaches worked, but required teams to remember to use them consistently.

Rails 7.2

Rails 7.2 makes Active Job transaction-aware. Jobs are automatically deferred until the transaction commits, and dropped if it rolls back.

Enable it globally in your application:

# config/application.rb
config.active_job.enqueue_after_transaction_commit = :default

Now the original code just works:

class UsersController < ApplicationController
  def create
    User.transaction do
      @user = User.create!(user_params)
      WelcomeEmailJob.perform_later(@user)  # Deferred until commit
    end
  end
end

The job only gets enqueued after the transaction successfully commits. If the transaction rolls back, the job is never enqueued.

Configuration Options

You can control this behavior at three levels:

Global configuration:

# config/application.rb
config.active_job.enqueue_after_transaction_commit = :default

Per-job configuration:

class WelcomeEmailJob < ApplicationJob
  self.enqueue_after_transaction_commit = :always
end

class AuditLogJob < ApplicationJob
  self.enqueue_after_transaction_commit = :never  # Queue immediately
end

The available values are:

:default - Let the queue adapter decide the behavior
:always - Always defer until transaction commits
:never - Queue immediately (pre-7.2 behavior)

Checking Enqueue Status

Since perform_later returns immediately even when the job is deferred, you can check if it was actually enqueued:

User.transaction do
  user = User.create!(user_params)
  job = WelcomeEmailJob.perform_later(user)

  # job.successfully_enqueued? returns false here (still deferred)
end

# After transaction commits, job.successfully_enqueued? returns true

Model Callbacks Simplified

You can now safely use after_create for job scheduling without worrying about transaction timing:

class Project < ApplicationRecord
  # This is now safe with enqueue_after_transaction_commit enabled
  after_create -> { NotifyParticipantsJob.perform_later(self) }
end

The job automatically waits for any enclosing transaction to complete.

When to Disable

Some scenarios require immediate enqueueing:

Database-backed queues: If you use Solid Queue, GoodJob, or Delayed Job with the same database, jobs are part of the same transaction and this deferral is unnecessary
Fire-and-forget jobs: Jobs that do not depend on the transaction data can run immediately
Time-sensitive operations: If you need the job queued at a specific moment regardless of transaction state

class TimeStampedJob < ApplicationJob
  self.enqueue_after_transaction_commit = :never

  def perform
    # This job needs to capture the exact enqueue time
  end
end

Update: Rails 8.2 Makes This the Default

Rails 8.2 makes enqueue_after_transaction_commit the default behavior. Jobs are now automatically deferred until after the transaction commits without requiring explicit configuration.

See Rails 8.2 makes enqueue_after_transaction_commit the default for details on the change, opting out, and the deprecation history.

Conclusion

enqueue_after_transaction_commit eliminates a common source of race conditions in Rails applications. Instead of remembering to use after_commit callbacks or building custom workarounds, jobs are automatically deferred until transactions complete.

References

Pull Request #51426 introducing the feature
Pull Request #55788 making this the default in Rails 8.2
Original Issue #26045 by DHH describing the problem
Active Job Basics Guide

Rails 8.2 introduces Rails.app.creds for unified credential management

Mon, 29 Dec 2025 00:00:00 +0000

Applications often store secrets in both environment variables and encrypted credential files. Migrating between these storage methods or using both simultaneously has traditionally required code changes. Rails 8.2 solves this with Rails.app.creds, a unified API that checks ENV first, then falls back to encrypted credentials.

Before

Managing credentials from multiple sources meant mixing different APIs:

class StripeService
  def initialize
    # Check ENV first, fallback to credentials
    @api_key = ENV["STRIPE_API_KEY"] || Rails.application.credentials.dig(:stripe, :api_key)
    @webhook_secret = ENV.fetch("STRIPE_WEBHOOK_SECRET") {
      Rails.application.credentials.stripe&.webhook_secret
    }

    raise "Missing Stripe API key!" unless @api_key
  end
end

class DatabaseConfig
  def connection_url
    # Different syntax for each source
    ENV["DATABASE_URL"] || Rails.application.credentials.database_url
  end

  def redis_url
    ENV.fetch("REDIS_URL", Rails.application.credentials.dig(:redis, :url) || "redis://localhost:6379")
  end
end

This approach has several problems:

Inconsistent APIs between ENV.fetch() and credentials.dig()
Manual fallback logic scattered throughout the codebase
Code changes required when moving secrets between storage methods
Easy to forget nil checks on nested credentials

Rails 8.2

The new Rails.app.creds provides a consistent interface:

class StripeService
  def initialize
    @api_key = Rails.app.creds.require(:stripe_api_key)
    @webhook_secret = Rails.app.creds.require(:stripe_webhook_secret)
  end
end

class DatabaseConfig
  def connection_url
    Rails.app.creds.require(:database_url)
  end

  def redis_url
    Rails.app.creds.option(:redis_url, default: "redis://localhost:6379")
  end
end

The require method mandates a value exists and raises KeyError if missing from both ENV and encrypted credentials. The option method returns nil or a default value gracefully.

Nested Keys

For nested credentials, pass multiple keys. Rails automatically converts them to the appropriate format for each source:

# Checks ENV["AWS__ACCESS_KEY_ID"] first, then credentials.dig(:aws, :access_key_id)
Rails.app.creds.require(:aws, :access_key_id)

# Multi-level nesting
# ENV["REDIS__CACHE__TTL"] || credentials.dig(:redis, :cache, :ttl)
Rails.app.creds.option(:redis, :cache, :ttl, default: 3600)

The ENV lookup uses double underscores (__) as separators for nested keys:

:database_url → ENV["DATABASE_URL"]
[:aws, :region] → ENV["AWS__REGION"]
[:redis, :cache, :ttl] → ENV["REDIS__CACHE__TTL"]

Dynamic Defaults

The option method accepts callable defaults, evaluated only when needed:

Rails.app.creds.option(:cache_ttl, default: -> { 1.hour })
Rails.app.creds.option(:max_connections, default: -> { calculate_pool_size })

ENV-Only Access

Access environment variables directly using the same API via Rails.app.envs:

# Only checks ENV, no encrypted credentials fallback
Rails.app.envs.require(:port)
Rails.app.envs.option(:log_level, default: "info")

Custom Credential Sources

Under the hood, Rails.app.creds is powered by ActiveSupport::CombinedConfiguration, which checks multiple credential sources (called backends) in order. By default, it checks ENV first, then encrypted credentials. You can customize this chain to include external secret managers:

# config/initializers/credentials.rb
Rails.app.creds = ActiveSupport::CombinedConfiguration.new(
  Rails.app.envs,                   # Check ENV first
  VaultConfiguration.new,           # Then HashiCorp Vault
  OnePasswordConfiguration.new,     # Then 1Password
  Rails.app.credentials             # Finally, encrypted credentials
)

Each credential source needs to implement require and option methods matching the API.

Rails.app Alias

This feature comes alongside a new Rails.app alias for Rails.application:

# Before
Rails.application.credentials.aws.access_key_id

# After
Rails.app.credentials.aws.access_key_id

The shorter alias makes chained method calls more pleasant to read and write.

Conclusion

Rails.app.creds eliminates the friction of managing credentials across multiple sources. Secrets can move between ENV and encrypted files without touching application code.

References

PR #56404 - Add Rails.app.creds for combined credentials lookup
PR #56403 - Add Rails.app alias for Rails.application

Understanding PostgreSQL Checkpoints: From WAL to Disk

Fri, 17 Oct 2025 00:00:00 +0000

PostgreSQL relies on checkpoints to ensure data durability while maintaining performance. Understanding how checkpoints work and their relationship with Write-Ahead Logging is essential for database performance tuning and troubleshooting.

This post builds on fundamental concepts covered in our PostgreSQL internals series:

Write-Ahead Logging: The Foundation

Checkpoints work hand-in-hand with Write-Ahead Logging (WAL). When PostgreSQL modifies data, changes are written to WAL files first (sequential, fast) before updating data pages in memory. Modified pages (called “dirty pages”) accumulate in shared_buffers, and eventually these changes need to be written to the actual data files. That’s where checkpoints come in.

What Happens During a Checkpoint

A checkpoint is PostgreSQL’s process of writing all dirty pages from shared buffers to disk. It creates a known recovery point and ensures data durability.

The Checkpoint Process

When a checkpoint occurs:

Checkpoint starts
- PostgreSQL marks the current WAL position as the checkpoint location
- This position is the recovery starting point if a crash occurs
Dirty pages are written
- All modified data pages in shared_buffers are flushed to disk
- This happens gradually to avoid I/O spikes (controlled by checkpoint_completion_target)
- Pages are written in order to minimize random I/O
Checkpoint completes
- A checkpoint record is written to WAL
- The pg_control file is updated with the new checkpoint location
- Old WAL files (before the checkpoint) can now be recycled or archived

What Triggers a Checkpoint?

PostgreSQL creates checkpoints based on:

Time: checkpoint_timeout parameter (default: 5 minutes)
WAL volume: max_wal_size parameter (default: 1GB)
Manual trigger: CHECKPOINT command
Shutdown: Always creates a checkpoint during clean shutdown

-- Force an immediate checkpoint
CHECKPOINT;

Checkpoint Impact on Performance

Checkpoints involve heavy I/O (writing potentially gigabytes of dirty pages), which can cause temporary performance degradation. Understanding performance trade-offs helps you balance durability with speed. PostgreSQL spreads checkpoint writes over time using checkpoint_completion_target (default: 0.9) to minimize I/O spikes.

Monitoring Checkpoint Activity

For detailed monitoring techniques, see Part 6: Monitoring and Administration. Key metrics to track:

Check Checkpoint Statistics

-- PostgreSQL 17+
SELECT * FROM pg_stat_checkpointer;

-- PostgreSQL 16 and earlier
SELECT * FROM pg_stat_bgwriter;

What to look for:

High checkpoints_req (or num_requested in v17+) means checkpoints are happening too frequently
Large checkpoint_write_time (or write_time in v17+) indicates heavy I/O load

Monitor WAL Generation Rate

High WAL generation can trigger frequent checkpoints. See Part 6 for detailed WAL monitoring queries and interpretation.

Tuning Checkpoint Behavior

Key parameters to adjust (see Part 4: Performance Patterns for trade-offs):

checkpoint_timeout = 15min          # Default: 5min
max_wal_size = 4GB                  # Default: 1GB
checkpoint_completion_target = 0.9  # Default: 0.9
log_checkpoints = on

Guidelines:

Increase max_wal_size if checkpoints_req is high
Increase checkpoint_timeout for write-heavy workloads
Keep checkpoint_completion_target at 0.9 to avoid I/O spikes

For broader PostgreSQL performance optimization, see PostgreSQL query optimization guide.

Conclusion

Checkpoints are PostgreSQL’s mechanism for persisting in-memory changes to disk, creating recovery points, and managing WAL files. They balance performance with durability by batching writes and spreading I/O over time. Watch for frequent requested checkpoints and long write times as signals for tuning opportunities.

For deeper understanding, explore the PostgreSQL internals series, or dive into PostgreSQL EXPLAIN ANALYZE for query optimization.

References

PostgreSQL Fundamentals: Monitoring and Administration Tools (Part 6)

Thu, 16 Oct 2025 00:00:00 +0000

In Part 5, we learned how Write-Ahead Logging works internally. Now let’s explore the tools PostgreSQL provides for monitoring and administering these systems.

This is Part 6 (final part) of a series on PostgreSQL internals.

System Views: Your Window Into PostgreSQL

PostgreSQL exposes extensive information through system views. These views are your primary tool for understanding what’s happening inside the database.

pg_stat_checkpointer: Checkpoint Statistics (PostgreSQL 17+)

The most important view for checkpoint monitoring:

SELECT * FROM pg_stat_checkpointer;

Key columns:

SELECT
    num_timed,              -- Scheduled checkpoints (time-based)
    num_requested,          -- Requested checkpoints (WAL-based or manual)
    write_time,             -- Milliseconds spent writing files
    sync_time,              -- Milliseconds spent syncing files
    buffers_written,        -- Buffers written during checkpoints
    stats_reset            -- When stats were last reset
FROM pg_stat_checkpointer;

Note: Before PostgreSQL 17, checkpoint statistics were in pg_stat_bgwriter with column names checkpoints_timed, checkpoints_req, checkpoint_write_time, checkpoint_sync_time, and buffers_checkpoint.

Interpreting pg_stat_checkpointer

-- Example output:
num_timed:       1250    -- Mostly time-based (good)
num_requested:   45      -- Few requested (good)
write_time:      450000  -- 450 seconds total writing
sync_time:       2500    -- 2.5 seconds total syncing
buffers_written: 500000  -- 500k buffers written at checkpoints

What to look for:

High num_requested: Checkpoints happening too frequently
- Solution: Increase max_wal_size
High write_time: Checkpoint I/O is slow
- Solution: Increase checkpoint_completion_target
- Or: Improve disk I/O performance
High buffers_written relative to checkpoint frequency: Large checkpoints
- Solution: More frequent checkpoints or increase shared_buffers

pg_stat_wal: WAL Activity

Monitor WAL generation and flush activity:

SELECT
    wal_records,        -- Total WAL records generated
    wal_fpi,           -- Full page images written
    wal_bytes,         -- Total bytes written to WAL
    wal_buffers_full,  -- Times WAL buffer was full
    wal_write,         -- Number of WAL writes
    wal_sync,          -- Number of WAL syncs (fsync)
    wal_write_time,    -- Time spent writing WAL (ms)
    wal_sync_time,     -- Time spent syncing WAL (ms)
    stats_reset
FROM pg_stat_wal;

Calculating WAL Generation Rate

-- Record current WAL stats
CREATE TEMP TABLE wal_baseline AS
SELECT
    now() AS measured_at,
    pg_current_wal_lsn() AS wal_lsn,
    wal_bytes
FROM pg_stat_wal;

-- Wait 60 seconds...
SELECT pg_sleep(60);

-- Calculate rate
SELECT
    pg_size_pretty(
        w.wal_bytes - b.wal_bytes
    ) AS wal_generated,
    EXTRACT(EPOCH FROM (now() - b.measured_at)) AS seconds,
    pg_size_pretty(
        (w.wal_bytes - b.wal_bytes) /
        EXTRACT(EPOCH FROM (now() - b.measured_at))
    ) || '/s' AS wal_rate
FROM pg_stat_wal w, wal_baseline b;

-- Example result:
-- wal_generated: 25 MB
-- seconds: 60
-- wal_rate: 427 kB/s

pg_stat_database: Database-Level Stats

SELECT
    datname,
    xact_commit,           -- Transactions committed
    xact_rollback,         -- Transactions rolled back
    blks_read,            -- Disk blocks read
    blks_hit,             -- Disk blocks found in cache
    tup_inserted,         -- Rows inserted
    tup_updated,          -- Rows updated
    tup_deleted,          -- Rows deleted
    temp_files,           -- Temp files created
    temp_bytes            -- Temp file bytes
FROM pg_stat_database
WHERE datname = current_database();

Calculate cache hit ratio:

SELECT
    datname,
    round(
        100.0 * blks_hit / nullif(blks_hit + blks_read, 0),
        2
    ) AS cache_hit_ratio
FROM pg_stat_database
WHERE datname = current_database();

-- Healthy databases: 95%+
-- Low ratio: Need more shared_buffers or working set too large

pg_stat_statements: Query-Level WAL Generation

Track which queries generate the most WAL:

-- Enable extension
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;

-- Top WAL generators (PostgreSQL 13+)
SELECT
    substring(query, 1, 60) AS query_preview,
    calls,
    pg_size_pretty(wal_bytes) AS wal_generated,
    pg_size_pretty(wal_bytes / calls) AS wal_per_call,
    round(100.0 * wal_bytes / sum(wal_bytes) OVER (), 2) AS wal_percent
FROM pg_stat_statements
ORDER BY wal_bytes DESC
LIMIT 10;

Once you identify high WAL-generating queries, you can optimize write operations: batch INSERT/UPDATE operations, use COPY for bulk loads, and consider whether all indexes are necessary.

pg_stat_activity: Live Connections

SELECT
    pid,
    usename,
    application_name,
    state,
    query_start,
    state_change,
    wait_event_type,
    wait_event,
    substring(query, 1, 50) AS query_preview
FROM pg_stat_activity
WHERE state != 'idle'
ORDER BY query_start;

Find long-running queries:

SELECT
    pid,
    now() - query_start AS duration,
    query
FROM pg_stat_activity
WHERE state = 'active'
  AND now() - query_start > interval '5 minutes'
ORDER BY duration DESC;

For long-running queries, you can:

Analyze execution plans: Use EXPLAIN ANALYZE to understand why queries are slow. See PostgreSQL EXPLAIN ANALYZE Deep Dive for detailed analysis techniques.
Offload reads to replicas: Move long-running SELECT queries to read replicas to reduce contention on the primary database. See Rails Read Replicas Part 1 for implementation patterns.

PostgreSQL Logs: Checkpoint and WAL Messages

Enable detailed logging in postgresql.conf:

# Log checkpoints
log_checkpoints = on

# Log long-running statements
log_min_duration_statement = 1000  # Log queries > 1 second

# Log connections and disconnections
log_connections = on
log_disconnections = on

# Set log destination
logging_collector = on
log_directory = 'log'
log_filename = 'postgresql-%Y-%m-%d_%H%M%S.log'

Reading Checkpoint Logs

With log_checkpoints = on, you’ll see:

2025-10-17 10:15:42.123 UTC [12345] LOG: checkpoint starting: time
2025-10-17 10:16:11.456 UTC [12345] LOG: checkpoint complete: wrote 2435 buffers (14.9%); 0 WAL file(s) added, 0 removed, 3 recycled; write=29.725 s, sync=0.004 s, total=29.780 s; sync files=7, longest=0.003 s, average=0.001 s; distance=49142 kB, estimate=49142 kB

Breakdown:

wrote 2435 buffers (14.9%)    # Dirty pages written (14.9% of shared_buffers)
0 WAL file(s) added           # New WAL segments created
0 removed                     # Old WAL segments deleted
3 recycled                    # WAL segments renamed for reuse
write=29.725 s                # Time spent writing buffers
sync=0.004 s                  # Time spent fsync'ing
total=29.780 s                # Total checkpoint duration
distance=49142 kB             # WAL generated since last checkpoint
estimate=49142 kB             # Estimated WAL to next checkpoint

What to watch:

High write time: Checkpoint taking too long
- Check disk I/O performance
- Consider spreading checkpoint over more time
Frequent checkpoints: If you see many “checkpoint starting: xlog” instead of “checkpoint starting: time”
- Increase max_wal_size
Large distance: Generating lots of WAL
- Normal for write-heavy workloads
- Ensure max_wal_size is adequate

pg_waldump: Inspecting WAL Records

pg_waldump lets you read WAL files directly:

# Find WAL files
ls $PGDATA/pg_wal/

# If this doesn't work, replace PGDATA with output from SHOW data_directory;

# Dump a WAL segment
pg_waldump $PGDATA/pg_wal/000000010000000000000001

# Output shows each WAL record:
rmgr: Heap        len: 54   rec: INSERT off 1 flags 0x00
rmgr: Btree       len: 72   rec: INSERT_LEAF off 5
rmgr: Transaction len: 34   rec: COMMIT 2025-10-17 10:15:42.123456 UTC

Filtering WAL Records

# Show only specific resource manager (Heap = table data)
pg_waldump -r Heap $PGDATA/pg_wal/000000010000000000000001

# Show records from specific LSN range
pg_waldump -s 0/1500000 -e 0/1600000 $PGDATA/pg_wal/000000010000000000000001

# Show statistics summary
pg_waldump --stats $PGDATA/pg_wal/000000010000000000000001

Understanding WAL Record Output

rmgr: Heap        len: 54   rec: INSERT off 1
    lsn: 0/01500028, prev: 0/01500000, desc: INSERT off 1 flags 0x00
    blkref #0: rel 1663/16384/16385 blk 0

Breaking this down:

rmgr: Heap              # Resource manager (table data)
len: 54                 # Record length in bytes
rec: INSERT             # Operation type
lsn: 0/01500028        # Log Sequence Number
prev: 0/01500000       # Previous record LSN
blkref #0:             # Block reference
  rel 1663/16384/16385 # Relation OID (tablespace/database/relation)
  blk 0                # Block number

pg_control: Database Cluster State

View control file (requires command-line tool):

pg_controldata $PGDATA

Key information:

pg_control version number:            1300
Catalog version number:               202107181
Database system identifier:           7012345678901234567
Database cluster state:               in production
pg_control last modified:             Thu 17 Oct 2025 10:15:42 AM UTC
Latest checkpoint location:           0/1500000
Latest checkpoint's REDO location:    0/1480000
Latest checkpoint's TimeLineID:       1
Latest checkpoint's full_page_writes: on
Latest checkpoint's NextXID:          0:1000
Latest checkpoint's NextOID:          24576

This shows the last checkpoint LSN, which is crucial for crash recovery.

Monitoring Checkpoint Health

Create a monitoring query (PostgreSQL 17+):

CREATE OR REPLACE VIEW checkpoint_health AS
SELECT
    num_timed,
    num_requested,
    round(100.0 * num_requested /
          nullif(num_timed + num_requested, 0), 2
    ) AS req_checkpoint_pct,
    pg_size_pretty(
        buffers_written * 8192::bigint
    ) AS checkpoint_write_size,
    round(
        write_time::numeric /
        nullif(num_timed + num_requested, 0),
        2
    ) AS avg_checkpoint_write_ms,
    round(
        sync_time::numeric /
        nullif(num_timed + num_requested, 0),
        2
    ) AS avg_checkpoint_sync_ms
FROM pg_stat_checkpointer;

-- Check health
SELECT * FROM checkpoint_health;

-- Example output:
-- num_timed: 1200
-- num_requested: 50
-- req_checkpoint_pct: 4.00           ← Good (< 10%)
-- checkpoint_write_size: 4000 MB
-- avg_checkpoint_write_ms: 375.21
-- avg_checkpoint_sync_ms: 2.08

For PostgreSQL 16 and earlier, use pg_stat_bgwriter with column names checkpoints_timed, checkpoints_req, checkpoint_write_time, checkpoint_sync_time, and buffers_checkpoint.

Healthy checkpoint system:

req_checkpoint_pct < 10%: Most checkpoints are scheduled
Reasonable write times: Not overwhelming the I/O system
Consistent checkpoint sizes

Resetting Statistics

Statistics accumulate since the last reset:

-- Reset all statistics
SELECT pg_stat_reset();

-- Reset bgwriter stats
SELECT pg_stat_reset_shared('bgwriter');

-- Reset WAL stats
SELECT pg_stat_reset_shared('wal');

-- Check when stats were last reset
SELECT stats_reset FROM pg_stat_bgwriter;

Reset stats to measure recent behavior or after configuration changes.

Putting It All Together

A complete checkpoint monitoring query:

WITH wal_rate AS (
    SELECT
        pg_size_pretty(wal_bytes) AS total_wal,
        wal_records AS total_records,
        wal_fpi AS full_page_images
    FROM pg_stat_wal
),
checkpoint_stats AS (
    SELECT
        checkpoints_timed + checkpoints_req AS total_checkpoints,
        checkpoints_req,
        round(100.0 * checkpoints_req /
              nullif(checkpoints_timed + checkpoints_req, 0), 2
        ) AS req_pct,
        pg_size_pretty(buffers_checkpoint * 8192::bigint) AS data_written,
        round(checkpoint_write_time::numeric /
              nullif(checkpoints_timed + checkpoints_req, 0), 2
        ) AS avg_write_ms
    FROM pg_stat_bgwriter
)
SELECT
    c.total_checkpoints,
    c.checkpoints_req,
    c.req_pct || '%' AS req_checkpoint_pct,
    w.total_wal,
    w.total_records,
    w.full_page_images,
    c.data_written AS checkpoint_data_written,
    c.avg_write_ms || ' ms' AS avg_checkpoint_write_time
FROM checkpoint_stats c, wal_rate w;

Conclusion

You now have the foundational knowledge of PostgreSQL internals:

Memory vs disk performance (Part 1)
How data is stored in pages (Part 2)
Transactions and ACID (Part 3)
Performance trade-offs (Part 4)
Write-Ahead Logging (Part 5)
Monitoring tools (Part 6)

Think I missed out on a key topic? Please reach out to me.

Previous: Part 5 - Write-Ahead Logging Deep Dive

Next: Understanding PostgreSQL Checkpoints

Prateek Codes - Building Scalable Backend Systems

Stop Pasting Schema Into Your AI: Connect PostgreSQL Directly with MCP

The Problem

With a Postgres MCP Server

Setting Up @modelcontextprotocol/server-postgres

Step 1: Create a Read-Only Database User

Step 2: Configure Claude Desktop or Claude Code

Step 3: Verify the Connection

Other MCP Database Options

Security

Use a Read-Only Connection

Zero Data Retention for Sensitive Databases

What Changes Day to Day

Conclusion

References

Rails 8.2 adds this_week?, this_month?, and this_year? to Date and Time

Before

Rails 8.2

In controllers

In views

In scopes

How to change the week boundary

Conclusion

References

Rails 8.2 lets retry_on read the error when calculating wait time

Before

Rails 8.2

Backward Compatibility

When to Use This

Conclusion

References

Ruby::Box Practical Guide: Use Cases and Integration Patterns (Part 2)

Use Case: Plugin Systems

Use Case: Multi-Tenant Configuration

Use Case: Running Multiple Gem Versions

Use Case: Isolated Monkey Patches for Testing

Use Case: Shadow Testing

Working Around Native Extension Issues

Working Around ActiveSupport Issues

Performance Considerations

When to Use Ruby::Box

Migration Strategy

What’s Next for Ruby::Box

References

Ruby 4.0 Introduces Ruby::Box for In-Process Isolation (Part 1)

The Problem with Shared Namespaces

Ruby 4.0: Enter Ruby::Box

Understanding Box Types

The Ruby::Box API

What Gets Isolated

Enabling Ruby::Box

Important Limitations

File Scope Execution

References

Rails 8.2 makes enqueue_after_transaction_commit the default

The Problem with Opt-In

Rails 8.2 Changes the Default

Opting Out

Why the Global Config Was Restored

When This Matters

Conclusion

References

Rails 7.2 adds enqueue_after_transaction_commit to prevent job race conditions

Before

The Workaround

Rails 7.2

Configuration Options

Checking Enqueue Status

Model Callbacks Simplified

When to Disable

Update: Rails 8.2 Makes This the Default

Conclusion

References

Rails 8.2 introduces Rails.app.creds for unified credential management

Before

Rails 8.2

Nested Keys

Dynamic Defaults

ENV-Only Access

Custom Credential Sources

When to Use `Ruby::Box`

What’s Next for `Ruby::Box`