I always think of kneeling (the knees on the ground kind) as goyish -- something Dennis the Menace does next to his bed before going to sleep or Muslims do on rugs facing Mecca. Of course, it's perfectly Jewish, except for the fact that Jews almost never do it.

Keri'ah is kneeling (or at least bending the knees and bowing); kidah is bowing so that the forehead touches the ground; hishtachavayah is lying prostrate so that the forehead touches the ground and arms and legs are spread. As is clear from the gemara in Megillah 22b, the ordinary form of tachanun is with either kidah or hishtachavayah. The Rambam also rules (Hil. Nesias Kapayim 5:14) that this is how tachanun should be performed.

So why do we not do this? There is a Biblical prohibition against bowing (outside of the beis hamikdash) so that one's forehead is touching the ground, where that ground is hewn stone (even maskis). According to at least one view in the gemara in Megillah this prohibition is restricted to the case of hishtachavayah. One way to avoid this problem is to lean slightly to the side so that the forehead does not make contact with the ground.

One can speculate that as shul architecture evolved, kidah with a slight twist evolved into the pretty lame lean we do today. This is an area ripe for some kind of back-to-authenticity movement a la techeiles. And it doesn't even cost money.

One might wonder why we should fuss about the prohibition in places where there is no stone floor. And moreover, why we should be concerned about the prohibition if we don't do hishtachavayah (though, it is not clear that we shouldn't be doing hishtachavayah on Yom Kippur). The answer goes back to two chumrahs of the Rivash (Resp. 412). First, the Rivash holds that the prohibition holds even for kidah. Second, he holds that the prohibition holds even in a place where there was once a stone floor that has since been built over. This is why the Rema rules that (even on non-stone floors) one should put paper or some such in front of his head (not under his knees) when bowing on Yom Kippur (presumably, even the Rivash would agree that some material that is not attached to the ground would be sufficient interference).

Incidentally, the business about not even doing the lame lean in the absence of a sefer torah is a novel idea of the Roke'ach. The Divrei Yatziv, in a comprehensive responsum about customs of tachanun (OH 74), argues persuasively that the Roke'ach's rule should only apply for those actually doing hishtachavayah. In any case, he says, since tachanun is optional, it isn't worth getting into arguments about it.