SOMEONE RECENTLY POSED the following question on Mi Yodeya:
There is a known rule that Rishonim cannot argue on the Gemara when it comes to ‘Halachic’ drashos. For example, a Rishon will never give a different answer to a question which the Gemara already answered (Tosafos may ask why the Gemara didn't give that other answer, but he will never give his own answer without more). This is true whether the Gemara's subject is practical Halacha or not.
However, when it comes to Aggadic drashos on Tanach, we find Rishonim ‘disagreeing’ with Midrashim and late Acharonim disagreeing on Rishonim. Why do Rishonim feel freer to disagree on an explanation of Chumash than to disagree with a Gemara?
This was my answer:
The Rambam writes in one of his letters:
We do not pose difficulties with [i.e. from] the Aggadah. Are they words of Tradition or expressions of reason? Rather each individual considers their explanation as it seems fit to him. In this [Aggadah] there are no words of Tradition, no prohibition and no license, and no law among the Laws; therefore we do not pose difficulties with it. Should you say to me as many have said to me, “Can it be that you apply the term Aggadah [as pertains to this argument] to words of the Talmud?” It is so; all of these words and those similar to them are Aggadah in their reckoning, whether they be written in books of Derashos, whether they be written in books of Aggadah.
This position is echoed by various Ge'onim and Rishonim; see for example Otzar HaGeonim to Chagigah (pp. 59-60) from R. Hai Gaon and R. Shereira Gaon; and the Ramban in his famous Disputation.
While I received a few up-votes for this one (phew!), this commenter took issue with the idea:
What troubles me with this answer is that the Ramban will knock out Chazal's interpretation in a Midrash using logic. If it can be knocked out through logic, why didn't Chazal think of it before choosing that interpretation?
The truth is this is a very good question, and the comments section was too small to adequately address it. So I’ll try to do it justice here.
One of the most remarkable stories of the Talmud is that of Rabbi Eliezer and the oven of Achna’i. The Gemara relates as follows:
תנא באותו היום השיב רבי אליעזר כל תשובות שבעולם ולא קיבלו הימנו. אמר להם אם הלכה כמותי חרוב זה יוכיח, נעקר חרוב ממקומו מאה אמה ואמרי לה ארבע מאות אמה. אמרו לו אין מביאין ראיה מן החרוב. חזר ואמר להם אם הלכה כמותי אמת המים יוכיחו, חזרו אמת המים לאחוריהם. אמרו לו אין מביאין ראיה מאמת המים. חזר ואמר להם אם הלכה כמותי כותלי בית המדרש יוכיחו, הטו כותלי בית המדרש ליפול. גער בהם רבי יהושע אמר להם אם תלמידי חכמים מנצחים זה את זה בהלכה אתם מה טיבכם?! לא נפלו מפני כבודו של רבי יהושע ולא זקפו מפני כבודו של ר"א ועדיין מטין ועומדין. חזר ואמר להם אם הלכה כמותי מן השמים יוכיחו, יצאתה בת קול ואמרה מה לכם אצל ר"א, שהלכה כמותו בכ"מ! עמד רבי יהושע על רגליו ואמר (דברים ל:יב) לא בשמים היא! מאי לא בשמים היא, אמר רבי ירמיה שכבר נתנה תורה מהר סיני, אין אנו משגיחין בבת קול, שכבר כתבת בהר סיני בתורה (שמות כג:ב) אחרי רבים להטות. אשכחיה רבי נתן לאליהו א"ל מאי עביד קוב"ה בההיא שעתא, א"ל קא חייך ואמר נצחוני בני נצחוני בני.
…R. Eliezer brought forward every imaginable argument [to prove that the oven of Achna’i was ritually pure], but they did not accept them. He said to them: “If the Halacha agrees with me, let this carob tree prove it!” Thereupon the carob tree was torn a hundred cubits out of its place – and others say four hundred cubits. “No proof can be brought from a carob tree,” they retorted. Again he said to them: “If the Halacha agrees with me, let the stream of water prove it!” whereupon the stream of water flowed backwards. “No proof can be brought from a stream of water,” they rejoined. Again he urged: “If the Halacha agrees with me, let the walls of the schoolhouse prove it,” whereupon the walls inclined to fall. But R. Joshua rebuked them, saying: “when scholars are engaged in a halachic dispute, what right do you have to interfere?” Hence they did not fall, in honor of R. Joshua, nor did they resume the upright, in honor of R. Eliezer; and they are still standing thus inclined. Again he said to them: “If the Halacha agrees with me, let it be proved from Heaven!” whereupon a Heavenly Voice cried out: “why do you dispute with R. Eliezer, seeing that in all matters the Halacha agrees with him?!” But R. Joshua arose and exclaimed: “It is not in heaven!” What did he mean by this? Said R. Jeremiah: “Since the Torah has already been given at Mount Sinai we pay no attention to a Heavenly Voice, because You have long since written in the Torah at Mount Sinai, “After the majority must one incline.” R. Nathan met Elijah and asked him: What did the Holy One, Blessed is He, do in that moment? [Answered Elijah:] “He laughed [with joy], he replied, saying “My sons have defeated Me, My sons have defeated Me.”
I have always interpreted this Gemara as firmly and unequivocally advocating a certain fundamental concept in Halacha, which I have recently found to be virtually identical to the philosophical theory of legal positivism:
Legal positivism is the thesis that the existence and content of law depends on social facts and not on its merits. The English jurist John Austin (1790-1859) formulated it thus: “The existence of law is one thing; its merit and demerit another. Whether it be or be not is one enquiry; whether it be or be not conformable to an assumed standard, is a different enquiry.” (1832, p. 157) The positivist thesis does not say that law's merits are unintelligible, unimportant, or peripheral to the philosophy of law. It says that they do not determine whether laws or legal systems exist. Whether a society has a legal system depends on the presence of certain structures of governance, not on the extent to which it satisfies ideals of justice, democracy, or the rule of law. What laws are in force in that system depends on what social standards its officials recognize as authoritative; for example, legislative enactments, judicial decisions, or social customs. The fact that a policy would be just, wise, efficient, or prudent is never sufficient reason for thinking that it is actually the law, and the fact that it is unjust, unwise, inefficient or imprudent is never sufficient reason for doubting it. According to positivism, law is a matter of what has been posited (ordered, decided, practiced, tolerated, etc.); as we might say in a more modern idiom, positivism is the view that law is a social construction. Austin thought the thesis “simple and glaring.” While it is probably the dominant view among analytically inclined philosophers of law, it is also the subject of competing interpretations together with persistent criticisms and misunderstandings.
R. Joshua ignored all the signs and wonders because they were simply not relevant. Halacha is a system. It must be so in order for it to survive within the context of a social structure – something Judaism has always been; not simply a variety of suggestions for individuals. A system cannot float with the wind by the whims of any individual who thinks he knows what is right and wrong, because such a system will not survive.
This being the case, the correctness of a particular Halacha in question ultimately cannot be practically determined by what is truly right and wrong, but only by the rules of the system. This is legal positivism. Consequently, if the system says to follow the majority, then if the majority says x the law is x, even if in truth what is right is not x.
Getting back to the story; according to the rules of the system they were working with (in this case specifically the rule that when an established court convenes on a matter, majority rules), R. Joshua’s ruling is Halacha and R. Eliezer’s is not. Thus, even if God himself comes down and tells us R. Eliezer’s opinion is the truth, it does not change the Halacha, because the Halacha must follow the rules of its own system. This is R. Joshua’s rebuke, and this is the meaning of God’s laughing with joy, saying “My sons have defeated Me.”
ONE APPROACH TO understanding the Halachic authority of the Talmud is to see it as an outline of a Halachic system which was ‘signed into law’ so to speak by its redactors, who did so by invoking the power of the court – the beis din, which rules by majority – the same power utilized by R. Joshua. According to this approach, when we submit full Halachic authority to the Talmud it is not at all necessary to assume that we do so due to a presumption of any inherent inferiority on our part. This is not necessary, because we can say that we submit to the Talmud because it is the law. We might have thought differently than some Amora’im did had we been involved in their discussions, but it doesn’t matter. Since the Talmud has effectively been signed into law, we have moved from discussing the law itself to discussing its application.
It is for this reason that there is not much Halachic value in questioning the validity of a Talmudic ruling. Although as students of history we may speculate this way or that, we cannot effectively change the ruling, which has already been signed into law. What we can do is find loopholes which affect the application of the Halacha. We can even, based on extra-Halachic considerations, sometimes manipulate the Halacha this way or that, by zeroing in on ambiguities in the text. The ‘spirit of the law’ is certainly important, as we are dealing with a divine law – but then again let’s not forget that the Torah anticipated we would do this when it made it possible for R. Joshua to rule correctly against the truth! So who’s to determine the true spirit of the law?
It isn’t difficult to theorize why the Torah would lend itself to such treatment. While it is most fundamentally a teacher of a moral and spiritually-connected way of life, the Torah’s goal is not to aid in the perfection of one individual alone, but an entire society’s. For it to be of any benefit to an entire society it is understandable why it would limit its application with rules of legislation that are necessary in any legal system, in order to insure that people submit to the system and don’t end up running around swallowing each other whole because “I hold like this.”
The commenter who asked, “If it can be knocked out through logic, why didn't Chazal think of it before choosing that interpretation?” presupposed that our assumption of Chazal is that their knowledge was perfect. The basis for this assumption, as evident in a later comment, was the fact that we never find a Rishon questioning the validity of a position taken by the Talmud on a Halachic issue. Clearly then these Rishonim believe that Chazal’s knowledge was infallible – so how can they, in all honesty, take issue with them on Aggadic matters? But I believe this assumption is faulty. The only reason we never find a Rishon questioning the validity of a Halacha is because there is no point! It’s irrelevant, as I’ve stated, because with regard to the Halachic positions of the Talmud we are legal positivists. On the other hand, when dealing with matters of Aggadah or for that matter anything which is a question of ‘truth,’ these Rishonim will often deviate from the opinions of Chazal when there is no claim from Chazal that their interpretation is a Tradition. Because no one ever claimed Chazal were infallible, and in matters of truth, logic trumps all.
 Deuteronomy 30:12
 Exodus 23:2
 Stanford Encyclopedia of Philosophy; entry “Legal Positivism”: