In your example, why not select the non-circles with one rectangle select, then invert the selection by Shift-rectangle selecting everything?
As an aside, the lasso is a "direct selection" tool, operating on individual paths rather than shapes, not that this helps in your example. But it'd work well if I wanted to select, say, only the bottom side of your rectangle, for essentially the reason you dislike.
That works only in a simple example like this, where you can make contact with all of the undesired objects without making contact with the desired ones. This is almost never the case in a real-world illustration.
As an aside, the lasso is a "direct selection" tool, operating on individual paths rather than shapes, not that this helps in your example. But it'd work well if I wanted to select, say, only the bottom side of your rectangle, for essentially the reason you dislike.