Is your Postgres migration safe or not safe?

(safenotsafe.dev)

22 points | by vira28 2 hours ago

4 comments

  • vira28 29 minutes ago
    Author here: Adding some context. I led the Postgres platform team (2019-23) at Cloudflare and we were supporting 170+ growing product teams. One of the constant asks is schema migration review. We published a lot of best practices, added CI checks however, it was still hard to catch. Also, I tried to explain the internals of how the locking (rewrite) works, but I realized most of the devs just want the answer - Is it safe or not safe to run?

    Not sure if it rings a bell, the name is a reference to the Silicon Valley Jian Yang's hot dog or not hot dog app.

    Also, I understand the decision of safe vs not-safe depends heavily on data/histogram and edge cases, but still quite a lot of low-hanging issues can be easily caught with a deterministic rule engine. So I ported pg_savior[1] and used sql parser from libpg-query-node[2] which compiles as WASM, so it entirely runs on the browser. No telemetry, no login. Source attached [3]

    [1] https://github.com/viggy28/pg_savior [2] https://github.com/constructive-io/libpg-query-node [3] https://github.com/viggy28/safe-not-safe

    • necovek 21 minutes ago
      Wow, great idea!

      It's not immediately clear from the README, but is it easy to run with multiple profiles like "backwards-compatible", "revertable" (both data and schema) and "destructive" for that final clean-up in multi-staged no-downtime migrations? Basically common subsets of "safe-ness" of the schema migration queries.

      I imagine it can be tuned, but I'd love this for all my projects.

      And since I am currently on a project doing MS SQL (gasp), that'd be cool too ;)

      I am familiar with an "is it a hot dog" app from back in the day, bit would have never made the connection :)

  • jokull 46 minutes ago
    Very cool! I wrote a go library for this https://onwardpg.solberg.is
    • vira28 7 minutes ago
      Nice. Integrating it on CI and catching there is still the best way/place. Having said, it doesn't work like that in practice.
  • rohansx 25 minutes ago
    really like the browser-only approach here - catching the obvious migration risks locally before they ever reach CI feels super useful
  • kettlecrisp99 1 hour ago
    Thing that bit me most wasn't the DDL itself, it was lock queuing. An ADD COLUMN is instant but if it waits behind a long read, every query behind it piles up too. Lock_timeout plus retry saved us more than any clever migration tool.
    • vira28 9 minutes ago
      Agree. `lock_timeout` will go a long way in terms of damage control.