Tsk, tsk. I see your Canon 40/2.8 and raise you a Perar 28/4 for Leica M. Looks like a slightly bulging body cap:
http://japancamerahunter.com/shop/ms-optical-perar-28mm-f4-super-triplet/
You might also take a look at the 35/4.5 Perar:
http://japancamerahunter.com/2012/07/ms-optical-perar-35mm-f3-5-super-triplet-mk-ii/
It cheats slightly, in that it is a collapsible lens. But it does not collapse by much :)
Actually, a lot of the individual lenses inside a modern camera lens are there to correct the optical imperfections that the other lenses have introduced. If you are willing to make some compromises in absolute optical quality, a lens can be made smaller and lighter than if you have higher demands. Also, the Canon 40mm is one of a family of lenses called "pancakes", in which compact size has been the overriding design requirement. Pentax seems to have made a habit of such lenses and makes them in 21/3.2, 40/2.8 and 70/2.4 variants. Other manufacturers have the odd one as well, but it is a niche product for everybody but Pentax.
Another point is that due to the mirror-box of an SLR, wide-angled SLR lenses generally have to be designed as "retrofocus" ones which basically means that you bolt a tele lens on to the camera side of a wide-angle lens. Which makes it a lot bigger, naturally. Camera systems such as Leica M where the lens is mounted much closed to the film plane can have wide-angles without the retrofocal bits, so they can be made much smaller.
And, as a side note, f/2.8 zooms tend to be large to very large. 2.8 primes can be rather small, but 2.8 is not exactly blazingly fast by prime standards. Slower lenses tend to be smaller and lighter, for the same focal length, but slower primes than f/2.8 are not often seen these days outside of the rangefinder community. (Longer tele lenses excepted, of course, and the odd macro but macros have a whole heap of other design considerations of their own so it is not really fair to compare them to normal primes.)