Like, the iPhone? Or maybe a Lumia 800? They both fit comfortably in all mens' pockets (and most women's, assuming they have pockets).
The problem is Android is such a power hungry OS that you need to be constantly updating the specs as hardware and app demands grow higher, and high-end specs require space to fit everything in (including ever-growing battery demands), so small Android handsets have to make compromises on specs. It feels sort of like PCs did 10 years ago - 3 years of use and your top-of-the-line PC was near unusable. I find Windows Phone and iOS don't suffer from this nearly as much.
It isn't android. It's the manufacturer skins. AOSP Jellybean runs really smooth on my low end phone. (512 MB ram, 600 MHz processor) But HTC Sense 4 runs like crap.
It's a bit of both. The original Galaxy S (which was only a year and a half old at the time, and was a top-of-the-line phone) struggled really hard with stock android ICS for me.
The problem is Android is such a power hungry OS that you need to be constantly updating the specs as hardware and app demands grow higher, and high-end specs require space to fit everything in (including ever-growing battery demands), so small Android handsets have to make compromises on specs. It feels sort of like PCs did 10 years ago - 3 years of use and your top-of-the-line PC was near unusable. I find Windows Phone and iOS don't suffer from this nearly as much.