The thing is that "less-than-rigorous exercise" is true in many areas of ML. Take for example neural nets, which are very popular and successful, even among real expert's there a lot of 'magic' behind why they really work. SVMs are loved partially because they work well, but also they are very sound from a theoretical standpoint, if you know the math you can show that it will work, this is not necessarily true with many other successful techniques.
Interesting side note for ensembles: 'averaging' is usually not one of the best methods for blending results. More successful approaches include using either a perceptron or a simply training a linear model to find appropriate weights for predictions from each individual model. I've even had a case where simply picking the MIN of each set of predictions worked surprisingly well for a particular problem.
The above btw is something that I think a "Data Scientist" should know, and is well out of the scope of a software engineer who just plugs values into prepackaged algorithms. A "data scientist" should be able to read papers [1] that explain these things, which is more than many software engineers do.
Now I'm not a data scientist, but while I can't write an SVM from scratch, when I'm working on data mining problems I am reading several academic papers a week. I really think we're looking at two sincerely distinct areas of expertise and it's not too lofty to look at someone who has to read academic papers to do his job as a "scientist".
Interesting side note for ensembles: 'averaging' is usually not one of the best methods for blending results. More successful approaches include using either a perceptron or a simply training a linear model to find appropriate weights for predictions from each individual model. I've even had a case where simply picking the MIN of each set of predictions worked surprisingly well for a particular problem.
The above btw is something that I think a "Data Scientist" should know, and is well out of the scope of a software engineer who just plugs values into prepackaged algorithms. A "data scientist" should be able to read papers [1] that explain these things, which is more than many software engineers do.
Now I'm not a data scientist, but while I can't write an SVM from scratch, when I'm working on data mining problems I am reading several academic papers a week. I really think we're looking at two sincerely distinct areas of expertise and it's not too lofty to look at someone who has to read academic papers to do his job as a "scientist".
[1] http://www.edscave.com/docs/Blending_Methods_AusDM2009.pdf