I think there is more to learn here about engineering mistakes than about Go:
- Why pick Go for this project in the first place? Those days the biggest productivity difference that arises from language choice comes from the availability of libraries, and it does not take much research to see that Go wasn't particularly targeted at numeric computing and that library support for those kinds of things is very poor. There is a number of mature platforms for this class of problems, SciPy, MatLab/Octave, C/C++ with various BLAS-derived libraries etc.
- The bit on poor performance is unconvincing because the source of the difference has not been identified. The speculations about Go and Java that follow are so poorly backed by any evidence and so wild they should have been cut out from the article. It's clear that the author has no clue what really happens in either case, and he proceeds to draw conclusions anyway. This kind of "magical" thinking about black boxes one doesn't understand is unfortunately all too common across software engineering.
- One place where I agree with the author is that Go's zoo of builtin data structures is really, really poor compared to Java. I mean compare:
>I think there is more to learn here about engineering mistakes than about Go: Why pick Go for this project in the first place?
I hate this type of non-productive comment.
"Why he picked Go" is beside the point. That was his personal choice, and it's of no concern to us if it was a good or bad choice when he made it.
What IS of interest to us, is his findings, ie. about how (non) fitting was Go for the kind of work he describes, and for what reasons. Because that can helps us access the language ourselves for such uses.
>Those days the biggest productivity difference that arises from language choice comes from the availability of libraries
Again, not true for his needs. For what he needed, core library built-ins have sufficed in Java (Set etc). As for Go, he did find 2 libraries implementing them, but he still wished for a good core implementation.
>The bit on poor performance is unconvincing because the source of the difference has not been identified.
No, but it's well known that Go lags behind the JVM, something even the Go team aknowledges and attributes to maturity of the compiler/GC.
Plus, from the writeup, he sounds like a thoroughly decent programer to not be able to code his way out of Go performance bottlenecks. Doesn't sound like the guy who can't use a profiler (and in fact, he points in his article that he DID use one).
>This kind of "magical" thinking about black boxes one doesn't understand is unfortunately all too common across software engineering.
This reminds me of that one guy on the go-nuts irc channel who asked for help with his Ruby port over to Go. He was scraping a set of web pages in Ruby and migrated that over to Go for its easy parallelization. His claim was that Go was 3x slower in speed. He pasted some code that seemed fine, but left out the context of what he was doing. At the end of the day, it as just an attempt to claim that Ruby is faster than Go. If he doesn't provide the entire context with benchmarks included, I'm just taking his word for it.
On another note, I built an algorithm using a bitset. At first, I used one of the libraries the author linked, but then I saw that it was easier to just do the bit-bashing myself for my particular use case. It's not that difficult if you are familiar with C and C++ programming.
If, however, you are coming over from a language like Python, Ruby, and even Java to criticize Go is an easy thing to do. Those are very high level languages that protect you from needing to understand types. The object oriented parts of Java languages give you the ability to use polymorphism, which is not as straight forward in Go. You don't have a common base class. You only have interfaces. Trying to recreate generic structures and then blame the language for not having them is foolish. If you can't program without generics, stick with Java from the beginning. Or better yet, don't do a project in Go until you understand how to approach problems the Go way.
I happen to like Go for the very reason that you don't deal with generics. Seeing the types make it easier for me to follow as opposed to following a chain of inheritance. Getting rid of all the factory code trims my code down. This makes things easier to read when you are not the author.
Comparing Java's compiler to any other compiler is going to have predictable results. Java may have the best compiler out there.
Generics, inheritance, and the factory pattern are completely orthogonal features. Adding generics would not entail adding inheritance, mandating the factory pattern, or any other slippery slope feature.
> Inheritance is overrated, and the factory pattern is often overused. Both of those happen with generics.
Generics have nothing to do with inheritance and the factory pattern. There are lots of languages that have generics but neither inheritance neither factories: SML, OCaml, Haskell, etc.
Inheritance makes generics harder, in fact, because type inference becomes undecidable in the general case.
Factories are basically a workaround for functions not being able to be freestanding (unattached to a class) in Java (the "kingdom of nouns"), a problem that Go doesn't have.
Generics have nothing to do with inheritance and the factory pattern.
Where do I say such a thing? I'm saying that generics are both overrated and overused.
Factories are basically a workaround for functions not being able to be freestanding (unattached to a class) in Java (the "kingdom of nouns"), a problem that Go doesn't have.
Being the "Kingdom of Nouns" is kinda the point of Smalltalk/Objective-C style OO. (Though Alan Kay once famously said that when he thought of OO, he was not thinking of Java.) That said, while it is nice to have someplace to put functions, there are times when it's also nice not to be forced to attach them to a class. Not so sure it's a problem as much as it's just a particular approach with its advantages and drawbacks.
Where do you get that Factories are "a workaround for functions not being able to be freestanding?"
>> Generics have nothing to do with inheritance and the factory pattern.
> Where do I say such a thing? I'm saying that generics are both overrated and overused.
It looks like that's what you're suggesting here:
> Inheritance is overrated, and the factory pattern is often overused. Both of those happen with generics.
---
> Where do you get that Factories are "a workaround for functions not being able to be freestanding?"
A factory is, conceptually, a function of some set of inputs that returns a new instance - which is, of course, exactly what a constructor is. Now, it's not always appropriate for a class to know every single way in which it might be assembled, so such responsibility is usually lifted out - the question is where to put it. We already have constructors, and those are functions, so it would seem reasonable that we could just write method similar to a constructor, place it in some other namespace (perhaps in a completely different package/module), and then pass that along to the component that needs to create these new instances.
For languages like Java, the problem is that functions are not first class - you can't directly pass along a method/function like (Conf -> SomeService), so instead, it's common to create some special type, like SomeServiceFactory. If, for comparison, you take a look at Scala/Haskell/C#/etc, you'll wont often spot /.*Factory/ anywhere, because a function (with the ability to close over free variables) is much more concise, and doesn't require defining new types (e.g. SomeServiceFactory, versus Function<Conf,SomeService>).
I think that, seen in this light, the practice of defining dedicated factory classes is a rather clumsy approach necessitated by the design of the underlying language.
> Inheritance is overrated, and the factory pattern is often overused. Both of those happen with generics.
Non-sequitur here. To me, it's clear that I'm saying that generics are overrated and overused. Your interpretation makes no sense!
For languages like Java, the problem is that functions are not first class - you can't directly pass along a method/function like (Conf -> SomeService), so instead, it's common to create some special type, like SomeServiceFactory.
Another non-sequitur here. Factory pattern was originally popularized in the GoF book as a way of providing an opportunity for polymorphism.
If, for comparison, you take a look at Scala/Haskell/C#/etc, you'll wont often spot /.Factory/ anywhere, because a function (with the ability to close over free variables) is much more concise*
Old hand at Smalltalk here -- over 10 years. You don't often see Factory classes in Smalltalk. We also have very easy and nimble closures over variables in the context they're defined. I still don't get what you're on about.
>> Inheritance is overrated, and the factory pattern is often overused. Both of those happen with generics.
> Non-sequitur here. To me, it's clear that I'm saying that generics are overrated and overused. Your interpretation makes no sense!
Ok, now I understand what you're saying. As an aside, I would suggest that most people (as is evident in this comment thread) would not interpret your statement the way you intended.
>> For languages like Java, the problem is that functions are not first class - you can't directly pass along a method/function like (Conf -> SomeService), so instead, it's common to create some special type, like SomeServiceFactory.
> Another non-sequitur here. Factory pattern was originally popularized in the GoF book as a way of providing an opportunity for polymorphism.
I agree (re: polymorphism), but I don't think that contradicts what I'm saying; functions with parametric polymorphism (with support for {co,contra,in}variance) provide the same benefits of class polymorphism in this context.
> Old hand at Smalltalk here -- over 10 years. You don't often see Factory classes in Smalltalk. We also have very easy and nimble closures over variables in the context they're defined. I still don't get what you're on about.
You're absolutely correct. However, my point was that in statically typed languages, generics are conducive to favoring closures over factories classes and inheritance. But, now I realize that we were arguing two different things.
>Non-sequitur here. To me, it's clear that I'm saying that generics are overrated and overused. Your interpretation makes no sense!
You are making the non-sequitur. His interpretation is the only one that makes sense. It is not clear at all that you are saying what you claim you are saying. You are in fact not saying anything like it. You are saying two things you don't like happen with generics, but that is completely false.
What do you mean "now"? That was the first thing I said to you. You may wish to consider that when many people are telling you that what you said was nonsense, it may not be that all of them are crazy.
Those are all object oriented features. I was trying to compare O.O. to golang's approach.
I like the fact that you are forced to look at the algorithm vs the class hierarchy. I like that you don't have to read through factories.
I do think that generics would be great if they found a way of implementing them, however, I do not think that you need generics in order to write this kind of project.
If you generate a lot of goroutines, the overhead for maintaining and scheduling them is fairly significant, particularly when compared to constructs like Python generators.
I've demonstrated this myself by comparing the goroutines example prime number generator to a similar version written using Python generators.
The python generator code was faster, but there was a hard limit to how many primes could be calculated this way; I hit the stack limit.
>This reminds me of that one guy on the go-nuts irc channel who asked for help with his Ruby port over to Go. He was scraping a set of web pages in Ruby and migrated that over to Go for its easy parallelization. His claim was that Go was 3x slower in speed. He pasted some code that seemed fine, but left out the context of what he was doing.
Yes, with the exception of that being a story about some random idiot with an axe to grind, and this being a totally competent guy, that goes into detail about what he did and how, what trouble spots he found etc.
I guess, it being a hired project, he can't share the actual code. But he provided tons of pain points, well articulated.
This notion that having a huge gaping defect is just a different way of doing things is absurd. Lacking parametric polymorphism is completely unacceptable. This is a solved problem as of 1973. There is no excuse. Inheritance has nothing to do with it, and "all the factory code" has even less to do with it. You are saying "I like go because it isn't java", which is fine, but you are implying that being java is necessary to have parametric polymorphism, which is not at all accurate.
Or the ability to write collections in the first place, as enabled by generics. You can't even write a collection in Go unless you're dealing with interface{} or unsafe.Pointer and casting a lot.
I really like the language, enough to use it and work around the lack of generics, but that's a glaring weakness. They need some way to enable container classes.
With Go, the programmer will typically not be using data structures that are concurrently accessed from multiple goroutines. It is considered more idiomatic to have one goroutine controlling a given data structure, and have the others communicate with it via channels.
That is only one way of doing things. Even though channels facilitate that method, it is not unidiomatic to not use them.
Channels are a way of queueing work, which is great for many use cases.
You can use mutexes and shared memory if your situation calls for it. For example, if I am doing a graph traversal concurrently, I can use channels to send the work to many listening threads as I traverse that graph synchronously.
If for some reason, I need to update that graph's representation asynchronously, I may need to use mutexes to lock the graph since the graph is now shared memory.
Java's concurrency primitives and libraries are really overlooked in these discussions. Is the syntax too off-putting, or is it merely that Java is unfashionable?
Very much unfashionable for the HN crowd. Most of the new research in concurrency is happening in Java and funded by the high performance trading industry - an industry which is very far from the HN crowd. The new Java8 stampedlock is a good example. It's possible to implement it in C++ as well, but because of the guarantees required by the lock it is a very difficult lock to integrate into C++ code. On the other hand, the JRE guarantees the correct constraints for Java code making a stampedlock very easy to use [1]. The performance of a stampedlock also seems to be the best case for any multi-reader environment. [2]
I find your claim that most new research in concurrency happens in Java strange. Perhaps you are unfamiliar with academic research in concurrency and parallelism? A way to get a small taste is to look at recent papers from the conference Practice and Principles of Parallel Programming (PPoPP).
If Java is involved, it's more likely they're interested in cutting developer costs than doing any "new research." That isn't to say they aren't doing any "new research," or that the research quality is poor, or anything negative at all, really. It's just a restatement of the observation that the labor market for Java developers is quite different from C developers or C++ developers (there are a lot more of the former than of the latter--a lot more).
I don't think there is anything wrong with Java as a language, but I have a question for you. Does Java's concurrency engine allow for many threads like Go or is it similar to C# in how your threads are not lightweight and are therefore limited to ixN where N is the number of CPU cores and i is a small integer < 10.
The Java language specification does not define how threads are implemented.
The first set of JVMs did implement green threads, which are what goroutines are. Shortly thereafter most of them switched to red threads, aka real threads.
You can still find a few JVMs that use green threads, like Squawk.
Java doesn't have a "concurrency engine", but a very large set of concurrency primitives: schedulers, locking and lock-free data structures, atomics etc.
To answer your question, yes: my own library, Quasar[1], provides lightweight threads for Java.
- Why pick Go for this project in the first place? Those days the biggest productivity difference that arises from language choice comes from the availability of libraries, and it does not take much research to see that Go wasn't particularly targeted at numeric computing and that library support for those kinds of things is very poor. There is a number of mature platforms for this class of problems, SciPy, MatLab/Octave, C/C++ with various BLAS-derived libraries etc.
- The bit on poor performance is unconvincing because the source of the difference has not been identified. The speculations about Go and Java that follow are so poorly backed by any evidence and so wild they should have been cut out from the article. It's clear that the author has no clue what really happens in either case, and he proceeds to draw conclusions anyway. This kind of "magical" thinking about black boxes one doesn't understand is unfortunately all too common across software engineering.
- One place where I agree with the author is that Go's zoo of builtin data structures is really, really poor compared to Java. I mean compare:
http://golang.org/pkg/container/
http://docs.oracle.com/javase/tutorial/collections/interface...
http://docs.oracle.com/javase/tutorial/collections/implement...