It has not been my experience that Go is good for almost everything. On the contrary, it seems good at a couple very specific (though very common) niches: network services and cli utilities. But for most of what I do right now - data heavy work - it has not turned out to be very good (IMO). It really is just not better in any way to have to constantly write manual loops to do anything.
I think Go is pretty OK as a language for building data pipelines (I’m assuming you meant statistical ones, but the same argument applies to more data transform-y ones). What it is not good for is doing exploratory analysis (which is where Python shines).
Manual loops are pretty annoying when the focus is on figuring out which loops to write (exploratory phase). However, they are pretty nice once you’ve figured it out and need write a durable bit of code where your prioritise readability over conciseness (productionisation).
Going from Python to <any language> between the exploratory phase and the productionised pipeline is going to be a pain, I don’t think Go is particularly worse than others. At that point it’s all about the classic software tradeoffs (performance vs velocity vs maintainability) and I didn’t think Go is a good choice in many situations.
Well I totally disagree that writing manual loops is ever "pretty nice", but I agree that it's not as big an issue in final-version code as it is in exploration.
And I'm also in strong agreement that making any language transition between exploration and implementation is problematic. I do think go is worse than most, because I just think it has a mostly cultural allergy to manipulating collections of data as collections rather than element-by-element, but I agree that this is mostly lost in the noise of doing any re-write into a new language.
But this is why Python is best in this space. It simply has the best promotion path from experimentation to production. It is better than other "real" languages like go, because it thrives in the exploratory phase, and it is better than purpose-specific languages, like R, because it is also a great general-purpose language.
The other contender I see is Julia, which comes more from the experimentation-focused side, while trying to become a good general purpose language, but unfortunately I think it still needs to mature a lot on that side, and it's not clear that it has the community to push it far enough fast enough in that direction (IMO).
Even very performance-critical use cases work with python, because the iteration process can follow experimentation -> productionization -> performance analysis -> fixing low-hanging bottlenecks by offloading to existing native extensions -> writing custom native extensions for the real bottlenecks.
Yeah, but Go is also worse (in my experience) than most, if not all, of the other general-purpose languages I've used, for this niche.
For instance, Rust is actually pretty great in this space, despite being very ... not-python, and Java also has decent libraries. Then C++ (and Fortran!) are good for a very different reason than python, "on the metal" performance, which Go also isn't a great fit for.
