Blue-Collar Engineering Dispatch #3: "SQLite: The Database You're Not Using (But Should Be)"

Hi there, and welcome back!

Welcome to another issue of Blue-Collar Engineering Dispatch! If you're new here, this newsletter is all about building reliable, maintainable systems without falling into the trap of over-engineering. Today, we're talking about a database that's probably already on your machine but you're ignoring it in favor of something more "enterprise-y": SQLite.

Let's dive into why SQLite might be the most underrated tool in your engineering toolkit.

The Tale of Alex and His Expensive Empty Database

Alex had a big idea: an AI-powered app that was going to change everything. He didn’t have any users yet, but that was just a detail. What mattered was being ready to scale.

So Alex built what he thought a “real” startup should look like: PostgreSQL running on RDS with multi-AZ failover, PgBouncer for connection pooling, Redis for caching, HAProxy load balancers, Terraform configs for every subnet, and enough YAML to be inducted into the DevOps hall of fame. He even sketched out a disaster recovery plan. For his app with zero users. Unless you count his mom.

When the first AWS bill arrived, Alex nearly choked on his Swiss water processed decaf. Hundreds of dollars burned on infrastructure that mostly sat idle. The only person stressing his system was Alex himself, furiously refreshing the homepage during late-night coding sessions.

Panicked, he turned to his AI chatbot for advice: “How do I lower my cloud costs without rewriting everything?”

The reply was blunt: “Try SQLite. Until you actually have users, you don’t need anything more.”

Skeptical but desperate, Alex gave it a shot. He deleted the RDS cluster, removed the Terraform, and swapped one line of code for sqlite3 myapp.db. Suddenly his stack shrank to a single file. His app shipped faster, his bills dropped significantly, and his brain finally had room to think about features instead of failover.

Alex learned the hard way that sometimes the “enterprise-grade” choice is just expensive theater. What your first hundred users really need isn’t replicas, sharding, or high availability—it’s for the app to simply work.

The Reality Check: Why We Default to the Wrong Database

Here's the uncomfortable truth: most of us reach for PostgreSQL (or MySQL) by default, not because we need it, but because it feels "professional." We've been conditioned to think that serious applications require serious databases, but this assumption costs us time, money, and sanity when we're just getting started.

SQLite isn't just a toy database or a stepping stone to something "real". It's often the best first choice for the job, and you can always graduate to PostgreSQL when you have real data proving you need its complexity.

Why SQLite Makes Sense as Your Starting Point

Zero Configuration Hell

Remember the last time you set up your scale-ready database for a new project? Install the server, create users, configure authentication, set up connection pooling, tune memory settings... SQLite? You literally just open a file. That's it.

import sqlite3
conn = sqlite3.connect('myapp.db')

Want even less setup? In Rust you can even bundle SQLite directly in your application:

// Cargo.toml
[dependencies]
rusqlite = { version = "0.29", features = ["bundled"] }

// main.rs
use rusqlite::Connection;

fn main() {
    let conn = Connection::open("myapp.db").unwrap();
    // Zero external dependencies. Ships everywhere.
}

It’s easy to go from zero to prototype, but there has to be other benefits, right? Glad you asked, here is a short list of other benefits for SQLite as your database choice:

No Process Management

PostgreSQL requires a separate server process that needs monitoring, restarting, and resource management. SQLite is a library that runs in your application process. One less thing to monitor, one less thing to fail, and more time to focus on building features.

Backup Simplicity That Actually Works

PostgreSQL uses pg_dump, WAL archiving, point-in-time recovery, backup validation... SQLite backups just copy the file. Want continuous backups? Use a tool like Litestream to stream changes to S3. Want to restore? Copy the file back. More on Litestream later.

ACID Compliance Without the Complexity

SQLite gives you full ACID transactions, just like PostgreSQL. The difference? You don't need to configure anything to get them. No tuning checkpoint intervals or managing transaction log files.

Surprisingly Good Performance for Early-Stage Apps

This might shock you, but SQLite performs surprisingly well for typical web application workloads, especially in read-heavy scenarios. Why? No network overhead, no process context switching, and extremely efficient B-tree implementations.

One of the knocks I've always heard about SQLite is that it doesn't scale. So when I was researching the numbers, I was surprised about what it can actually handle before you need to consider PostgreSQL. Here are some facts that might change your assumptions:

• Database Size: SQLite handles databases up to 281 TB (theoretical limit, though practical limits are much lower)

• Concurrent Connections: Hundreds of concurrent readers with no issues. Single writer limitation? For most early-stage applications, this isn't actually a problem.

• Query Performance: Simple queries are often faster due to zero network latency

• User Scale: Can comfortably serve most applications with thousands of concurrent users, depending on usage patterns

When SQLite Is Perfect for Getting Started

Internal Tools and Admin Panels

That admin interface for managing customer data? SQLite is perfect. Zero operational overhead, easy to backup, simple to deploy.

Developer Tools and CLI Applications

Building a CLI tool that needs to persist data? SQLite gives you a full SQL database without requiring users to install anything.

Small to Medium Web Applications

Most web applications start with traffic patterns that SQLite can handle easily. You can always migrate to PostgreSQL when you have real user data proving you need more complexity

Edge Computing and Mobile

Deploying to edge locations or mobile devices? SQLite gives you full database capabilities without network dependencies.

Application File Formats

Instead of inventing custom file formats, use SQLite. It's literally designed for this use case and gives you SQL querying for free.

Hands-On: Setting Up SQLite with Litestream for High Availability

Here's something that might surprise you: SQLite can actually provide high availability with the right tooling. Let's build a simple high-availability SQLite setup that's inexpensive and easier than PostgreSQL clustering:

1. Install Litestream

# On Ubuntu/Debian
$ curl -fsSL https://repo.litestream.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/litestream-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/litestream-keyring.gpg] https://repo.litestream.io/debian bullseye main" | sudo tee /etc/apt/sources.list.d/litestream.list
sudo apt update && sudo apt install litestream

2. Configure Litestream Create /etc/litestream.yml (note: you'll need AWS credentials configured and an S3 bucket created):

dbs:
  - path: /var/lib/myapp/database.db
    replicas:
      - type: s3
        bucket: myapp-database-backups
        path: database
        region: us-east-1
        sync-interval: 1s

3. Start Continuous Replication

$ sudo systemctl enable litestream
$ sudo systemctl start litestream

4. Restore on Failure

$ litestream restore -o /var/lib/myapp/database.db s3://myapp-database-backups/database

That's it. You now have continuous database replication that's simpler and often more reliable than PostgreSQL streaming replication. This setup costs pennies compared to database clustering and can provide excellent uptime for most applications. The secret? Simplicity reduces failure modes. And when you do need to graduate to PostgreSQL, this gives you a clear migration path.

When to Graduate to PostgreSQL

Here's the part where I tell you when it's time to consider more complexity. You should migrate to PostgreSQL when you actually need its features, not when you think you might:

• Multiple concurrent writers across different application instances

• Very high concurrency (thousands of simultaneous database connections)

• Complex replication setups with read replicas in multiple regions

• Specialized database features like custom extensions, advanced indexing, or complex data types

• Regulatory requirements that specifically mandate certain database features

• Real performance bottlenecks that you've measured and identified

The key word here is "actually." Not "might someday" or "just in case we scale," but actually need right now, with real user traffic and measured performance data proving the necessity.

The Migration Path: SQLite to PostgreSQL

When you do need to graduate, the transition is usually straightforward:

• Most SQLite SQL works in PostgreSQL with minimal changes

• Export data using .dump or tools like sqlite3-to-postgres

• Update your connection string

• Your application logic largely stays the same

This lets you start simple and add complexity only when justified.

The Takeaway

The best database for getting started is the one you don't have to think about. For most applications, that's SQLite.

Your action plan: Start with SQLite. Build your features. Get real users. Measure actual performance bottlenecks. Then migrate to PostgreSQL when you have data proving you need its complexity.

Your future self (and your AWS bill) will thank you for starting simple.

Reader Challenge

What's the simplest technical solution you've implemented that worked better than a complex alternative? Reply and share your story—I might feature it in a future issue!

Until next time, keep it simple,

Bradley

Chief Advocate for Boring Technology That Works

P.S. If you enjoyed this issue, forward it to a fellow engineer who might want to read about simplifying infrastructure and architecture. And if you're new here, you can catch up on our previous issues about avoiding Kubernetes complexity and the myths of static typing that lead to over-engineered solutions.