After fighting crash after crash with pgcopydb v17 and v18 I decided to bite the bullet and write my own in Golang.
I picked Golang because most issues with pgcopydb were related to concurrency bugs and memory leaks. There is nothing in this tool that requires C, Golang is totally fine and comes with a good concurrency built-in and a garbage collector.
A few more notable improvements:
- I use binary format instead wal2json
- Optional control plane with web interface
- CDC replay runs concurrently (partition by fk)
I had a very similar story back pre-Go-v1, was frustrated trying to debug a C memory leak in concurrent code. Switched to Go in a few weeks, less code, no bugs in this realm, more features. I was hooked and have been a proud gopher since
How are you guys running it? Is it via RunsOn, Ubicloud? We just moved ours to Blacksmith since I still don't want to manage the worker ourselves yet. But with this change, we might be looking into cheaper, better alternatives if there's any.
Plenty of engineers on the market can work professionally in C. Only a small amount of people can write Cobol (or is willing to given that is almost useless). That alone is a good reason to consider Cobol a legacy language and throw away a codebase written in Cobol.
Throw away the codebase: no, until you have a well-working, battle-tested replacement. I mean, yes, it's possible to do, and you can even run e.g. Java under z/OS on the same in-house mainframe hardware because you can't trust the public cloud. But you still have to do the massive work of reverse-engineering the ancient, scantly documented Cobol codebase, write a modern replacement, cover a ton of corner cases, run in in prod as a shadow that does all the same work, and comparing the results with the load-bearing Cobol codebase, and switch over very carefully.
Depending on the size of your codebase, the complexity of your processes, the strength of your need for change, and the quality of your engineering org, the above may be a very costly process. Few managers are comfortable to approve such a large cost without a very clear return on this investment. Hence we'll see Cobol running for a few decades more.
I picked Golang because most issues with pgcopydb were related to concurrency bugs and memory leaks. There is nothing in this tool that requires C, Golang is totally fine and comes with a good concurrency built-in and a garbage collector.
A few more notable improvements:
- I use binary format instead wal2json - Optional control plane with web interface - CDC replay runs concurrently (partition by fk)
reply