Today in the daf (Menachot 33a), Rav Yehudah in the name of Rav states that if a mezuzah is placed like a nail, or bolt, that it is invalid. The Gemara then tells of the mezuzot in Rebbe’s house which placed like an “איסתוירא,” and were valid. To explain these terms, Rashi states that a bolt placement means a horizontal placement – this is invalid. The איסתוירא placement which is valid is either a bent, elbow shape, or simply a vertical shape. For Rashi, the ideal placement is vertical.
Tosafot (s.v. Ha) quotes Rabbenu Tam who disagrees. Rabbenu Tam states that the more respectful placement on scrolls is that of lying down horizontally. This is how, he records, scrolls are placed in the aharon, and this then is how the mezuzah should be placed in the doorpost. He also states that this is how the parchments of tfillin should be placed in their batim. Tosafot concludes thusly: “ואומר ר”ת צא ולמד דהא מעשה בכל יום כשספר תורה עומד על הבימה הכל עומדין וכששליח ציבור מושיבו הכל יושבין – And Rabbenu Tam says: Go and see, that it is a daily practice that when the sefer Torah is standing on the bimah that everyone stands, and when the shaliach tzibbur sets it down, everyone sits.” This phenomenon is presumably evidence that the sefer Torah is considered to be at rest, and in its proper place, when it is laid down horizontally.
For Rabbenu Tam, then, the conceptual model of the mezuzah‘s placement (and the placement of the tfillin klafim) is that of “storage” or “setting in its place.” To store sifrei kodesh means, for him, to lay them down horizontally, and thus to store, or to set the mezuzah in its place means to put it horizontally in the doorpost. Rashi, it would seem, sees it differently. For Rashi, the mitzvah is to affix the mezuzah, not to store or set the klaf. The klaf is being used, not stored, and as such it is held up vertically. At a conceptual level, the mitzvah is even more than affixing – it is “writing” the mezuzah on the doorpost. Thus, if we are to write, we must do this with the standard vertical orientation, and when we affix the mezuzah, the klaf must be in a position that it can be read, that is, vertical.
How do we rule? The Vlina Gaon rules like Rashi (see his commentary to Shulkhan Arukh YD 289:6), and in Yerushalayim, which follows minhag ha’GRA, you will find that the mezuzot are placed in a completely vertical position. This, however, is not standard Ashkenazic practice. Our practice follows Tosafot’s explanation of the phrase איסתוירא. Tosafot states that this means diagonal, and that such an orientation would be good even for Rashi. Our practice, then, is to place the mezuzah diagonally, with the assumption that this is what Rebbe did, and that this would be good according to both Rashi and Rabbenu Tam [a true compromise if I ever saw one!]. Nevertheless, we do fundamentally rule like Rashi, and thus our mezuzot are placed much more vertically than horizontally, and if the doorpost is not wide enough for a diagonal placement, we will place the mezuzah in a fully vertical position.
There is an interesting historical context here as well. Prof. Daniel Sperber, whose main field before his minhagim books was Talmudic realia, has an article in the first volume of his Minhagei Yisrael, where he looks at the doorways from the Tannaitic period. He shows that these doorways were usually made with a solid stone pillar on one side, and two stones with a small gap between them on the left (see illustration, upper left). A door bolt was placed into this gap to lock the door. This, then, is a very reasonable reading of the term נגר and סיכתא – positioned like a bolt used to lock the door, and this would reinforce Rashi’s reading of the Gemara. [The entire article can be downloaded under Resources.]
I have heard another explanation for the Ashkenazic practice of placing the mezuzah diagonally. That is, that originally the mezuzah was placed vertically. However, in Ashkenaz, when Crusaders would either pass through a Jewish town, or when Christians would take over Jewish houses, they would put a horizontal bar of wood across the mezuzah and turn it into a cross. To prevent this, the minhag in Ashkenaz changed to affix the mezuzah diagonally.
I have not seen a source for this theory, and have no idea if there is any evidence to support it. I would be interested in knowing if anyone has seen any relevant scholarship on this issue.