I think that, to some degree, becoming an effective APL programmer almost required becoming good at data representation. If you want to do things the "APL way" you have to think about structures that work well with array, matrix and tensor processing ideas. When you could represent your data in the most optimal way you could sometimes write a function with only a few operators that could do what needed to be done to that data.
This means that, to some extent, data representation might have several local minima (in terms of being optimal) with each population of possible representations exists around the language or toolset you are going to use to process this data.
What I mean is that the optimal data representation for an embedded system (as an example) written in assembler might be different than that of the same system written in C or Forth.
I don't use APL very much any more. Back when I did I was also programming in Forth, C and Assembler. I know that the decisions I would make --in terms of optimal data representation-- varied from language to language. This is partly true because each language, in my case, was being used to deal with different problem domains. For example, APL could do high-level computations while Forth was great for low-level, real-time stuff.
Interesting comments. I've been working with an APL-derivative and this sounds similar to how I have been thinking about things. It's a matter of massaging the data to get in into a certain representation, e.g. a matrix, then from there it's very, very easy to work with using APL. As you say a few operators is all you need. The code is very terse. Very powerful. The real work seems to be shaping the data into the right representation first, a matrix. I use non-APL-descendent programs, the usual UNIX utilities, to do this for now.
Right. I remember having fun with this doing an APL application to aid in DNA sequencing. There were a number of ways to represent the data provided by the sequencer. At one point it almost became a game to see how small of an expression one could create to process the data by changing the representation.
With APL one has to be careful not to create monsters that cause geometric expansion in memory needs.
If the data set is large and the expressions processing the data cause frequent expansion into matrices or tensors (n-dimensional data structures where n > 2) one could end-up with geometric or exponential memory requirements. This, again, is another case of having to understand and fit data representation to the programming language AND the approach one will use to work with the data.
While languages like APL can be great, they can be disastrous in the hands of a programmer who does not understand what might be going on at a lower level. Sometimes there's nothing better than good-old low-level C.
This means that, to some extent, data representation might have several local minima (in terms of being optimal) with each population of possible representations exists around the language or toolset you are going to use to process this data.
What I mean is that the optimal data representation for an embedded system (as an example) written in assembler might be different than that of the same system written in C or Forth.
I don't use APL very much any more. Back when I did I was also programming in Forth, C and Assembler. I know that the decisions I would make --in terms of optimal data representation-- varied from language to language. This is partly true because each language, in my case, was being used to deal with different problem domains. For example, APL could do high-level computations while Forth was great for low-level, real-time stuff.