The name, almost literally, conjures up certain images for us. Certain assumptions, certain expectations, and certain dreams. Gandalf and Merlin. Ged and Elric. The enemies of Conan and allies of Fafhrd and the Grey Mouser. From Harry Potter to Harry Dresden. Even Old Ben Kenobi plays the part well.
And, yet, RPGs seem to consistently fail to capture the mystique and majesty of magic. It is a list of powers, a bit of extra book-keeping, and a pain in the ass to balance. Why is this? Well, it's very easy to lay the blame at the feet of D&D. (And, admittedly, I tend to more often than I care to admit.) After all, they took one specific archetype that is quite deliberately unusual ("Vancian" magic), and then very deliberately devolved it into a system of rules suitable for miniatures combat. It was then re-evolved, if you will, into a system suitable for a true RPG. Mostly.
However, much of the problem is actually endemic to the role-playing game. The nature of what makes for good fiction, what makes for good religion/science, and what makes for a good game are all quite different. As a related note, there is precious little fiction that describes actual magical traditions in a true and realistic fashion. Real magical traditions, much like real science, real economics, or real politics, are either too tedious, too complex, or too subtle to make for exciting and accessible conflicts. However, that doesn't mean that many authors aren't capable of capturing the feel of a given magical tradition, even if the details are off. So, similarly, we'd like an RPG that captures that same feel.
Magical Mystery Tour
One of the first hurdles to overcome is mystery. In both fiction and reality, magic is very deliberately mysterious and poorly defined. It is not always consistent, and is not always reliable. Even traditions and depictions that do have intricate rules are usually exceptionally complex. An RPG, however, needs a clear, concise system. It doesn't need to be reliable. However, in my experience, unreliable magical systems are often extremely frustrating to the player, which runs counter to most design goals. So, we end up needing a list of "spells" (or some equivalent), specific conditions under which they do and do not work (or at least a clear way to calculate the probabilities of them working), and a relatively simple system for telling us how many "spells" can be used at a time. This generally ends up being well-known to most of the players, for fairly obvious reasons. Having clear, concise, well-known rules for magic really undermines any mystery it might be shooting for.
Along with the mystery of how magic works, RPGs tend to erode any mystery of why magic works. Setting information will almost always detail where magic comes from (in more or less detail, depending on the designer). It is very rarely left completely unspecified. This is for good reason. For the GM to make reasonable calls about the effects of and on magic in unusual circumstances, it helps considerably to know just what magic is. However, this knowledge very rarely stays with the GM. Some players will read the details because they are playing mages, and have some "need to know." Some players will read the details because they also are, were, or are planning to be a GM for the setting. Some players will read the details because they are just so fascinated by the setting that they can't help devouring every detail they get their hands on. Some players will read the details because they are dicks, and are looking to gain some advantage. Regardless of the reason, players end up with knowledge that their characters don't have. And, no matter how good a player may be at separating OOC and IC knowledge, simply having the knowledge at all erodes or ruins the mystery.
A Balancing Act
Magic, in fiction, is rarely "balanced." Gandalf is, quite simply, more powerful than any other member of the Fellowship. There are always counter-balances in the story. The wizard is restricted by his knowledge of consequences, or the magic is tiring, or is too slow. But, inevitably, the wizard is a peer with kings, champions of the realm, or, in some stories, servants of the very gods. It is hard to keep these paragons of wisdom and holders of dark secrets on a level with regular soldiers, let alone a band of outcast misfits destined to save the world.
To be honest, a fictional wizard's magic is also a plot device, to be used at the whim of the author. In an RPG, you cannot exert the kind of fiat control that an author has. Sometimes, it may be cool for a wizard to light his pipe, or even the candles in his room, with merely a thought. When that same thought is turned to burning through a rope to drop a chandelier on the enemy, it may be inappropriate. An author simply chooses to have the wizard not burn through the rope, and resolves the scene in some other way. A GM does not have that luxury. He needs to have a reason to give the player, so that the player will opt not to burn through the rope. Again, limits must be imposed, limits that generally are not present in fiction. And, further, those limits may end up stifling other cool actions, such as the wizard escaping from being tied up by burning the ropes. The flip side to this, as well, is that the wizard is often, even in character, simply a tool of destiny, there to lead and guide the hero. For reasons that are generally glossed over, a farmboy with a sword makes a better hero than a master of mystic might.
Years of Work for Moments of Wonder
A fictional wizard does a lot of his action "off screen." He spends years honing his craft, and making enchanted items to assist him. He sets up wards on his house, and spends hours engaged in divination to see the threads of fate around him. Then, when the moment calls for it, he is prepared to act in a decisive and powerful manner. (And, in truth, this is how most real traditions function as well.) In most "traditional" RPGs, this kind of action is extremely hard to model. "Downtime actions" are hard to regulate, hard to track, and easy to abuse (and equally easy to under-utilize or mis-apply). More modern systems often resolve this by simply handing the wizard a batch of "script control" points. This allows the wizard to achieve the effects of precognition and contingency plans by retconning said plans. Simply adding the proper descriptive text ("Ah, yes, the bones told me that would happen.") adds the right atmosphere and mystery. But, such a system also tends to discourage more proactive applications of the mystical arts.
Wizards suffer from other problems as they make the translation into RPG terms. In fiction, they are often rare, and often chosen by fate, bloodline, or the touch of some greater power. This works well, as it explains why there is a wizard with a group of individuals looking to change the world. It creates problems, however, when a second wizard wants to join the group. The party either suddenly becomes a significant power bloc in the arcane community, or verisimilitude is broken as the supposed rarity is clearly no longer in effect.
The training of a wizard is a difficult thing. In both fictional and historical traditions, it takes a lot of dedication, hard work, and focus to bend the laws of reality. It also takes sacrifice. But, in RPGs, it is assumed that if you write "wizard" at the top of your sheet, all the rest of that is assumed. It is up to the player (and, in some cases, certain restrictions enforced by the setting) to bring the effects of a lifetime of toil to the table. It is a very rare system that assists in this process, beyond "wizards are frail and suck in combat, because they spent all their time growing up reading."
You Rush a Miracle Man, You Get Rotten Miracles
There are aspects to the actual casting of magic that become difficult to translate effectively. Ritual magic, particularly in real-world traditions, is a very lengthy process. A proper blessing (or curse) rarely takes less than five minutes, and often as much as an hour. Contacting the spirit world is often a matter of hours. The conceit of an incantation that can be tossed off in seconds, in the midst of combat, does not have many analogues that pre-date D&D. And, yet, requiring those kinds of rituals often means that the wizard is simply left out of combat. After all, he is frail.
The "combat wizard" as mobile artillery is also a conceit that does not have much historical or fictional precedent. A wizard's contributions to the battle were generally in the form of precognition, scrying, laying blessings upon the fighters, and healing them afterwards. Very traditional, but also pretty boring for the player of the wizard. And, even more particularly, extremely hard to model well in the miniature combat systems that shaped D&D, and hence most RPGs that came after. And, so, masters of the elements who were able to toss around fireballs and lightning bolts became the order of the day. As RPGs have developed, this concept of a wizard's role has stuck fast, and for good reason. It is, generally speaking, what the players want.
Taking all of these points into consideration, it becomes obvious that trying to duplicate the role and actions of a fictional wizard or historical practitioner is going to be an uphill battle at best, and most likely futile. The RPG wizard is simply a different beast, with different assumptions and needs shaping him. And, yet, it is not impossible to shape the rules and the play environment so that Merlin can still feel like Merlin.