The idea of a low-magic fantasy setting seems a bit odd to me, in that the idea that the world we live in is low-magic strikes me as a very modern one. As far as I can tell at most times and places in our world, which has no magic at all, people nonetheless believed that the world was chock full of magic. It might have been hard to make use of reliably, though most superstitions seem to me to be every bit as formulaic as D&D wizard spells, but it lurked everywhere and you needed lots of protection against it.
I can kind of see wanting a setting where objective proof of the existence of magic is hard or impossible to come by if you want something that feels like our world. And I certainly get not wanting the solution to every problem to be just magic it away. But many (most?) low-magic settings I’ve seen in games take it much farther than that, to where hardly anybody even claims to do magic or have never encountered anything they regarded as supernatural, and that doesn’t quite feel right to me. To the modern mind the difference between natural and supernatural is obvious and complete: your cattle catching a disease vs. somebody levitating in front of your eyes are completely distinct kinds of phenomena. In a setting based on the pre-modern world I’m pretty sure that shouldn’t be true.
Tales of the Rambling Bumblers 
In the Canterbury Tales, one of Chaucer’s characters mentions something to the effect that these days the only elf or fairy one might come across on the highways of England was the occasional wandering lecher priest. The concept of a world of diminished magic is old as dirt.
Contrary, I’d argue that uber-magic fantasy settings is something fairly recent, brought out by Dungeons & Dragons. Most iconic fantasy stories pertain to the diminishment of magic in the world. The stories of Middle Earth chronicle the transition of the world from a high magic to a low magic setting, where magic continually diminished. Chronicles of Prydain take place at the tail end of the age of myth, ending with departure of magic from the land.
I think that to effectively implement a ‘low fantasy’ setting, one ought to look to those works, in which people:
a) know and strongly suspect magic/monsters exists
b) most have never seen it.
c) get kind of blown away when they come across it, all “Holy, crap, I knew it!”
Sort of like when Sam finally sees an Oliphaunt and it blows his mind.
Prydain and Middle Earth certainly aren’t the first things I would think of as “low magic” settings, or even settings where magic and the supernatural are common enough but the protagonists have little access to or interaction with it, and I think it’s telling that the stories aren’t set in the period after magic leaves the world. I mean, the staid inhabitants of the Shire regarded Gandalf’s blatant displays of magic as an amusing light show, not as Holy Crap! Magic is real! And even the Canterbury Tales seem to take place in a world where there is a strong belief in the supernatural; even supposing that the pilgrimage was for the most part not really about visiting the remains of the saint, the Parsoner’s very position and the fact that he actually has ready buyers for his fake relics seem to point to a world rife with superstition. There don’t have to be castles floating in air or people shooting fireballs from their fingertips for there to be a pervasive belief in magic and miracles.
Yes, but compare Middle Earth from the Lord of the Rings & the Hobbit to the ‘old’ times discussed by characters who remembered the First Age.
Middle Earth had gone from a place where gods and demons strode the earth alongside mortals to one where the last remnants of these were hidden in the most remote dark places, and the average person could live and die never having encountered anything particularly mystical or magic.
Then again, whole essays could be written on the theme of diminishment in Middle Earth. The end of the age of light, Morgoth’s fall from near omnipotence, the departure of the Valar, the gradual departure of all elves from middle earth…
Prior to the 1st age, everything was magic, because all things were intimately tied to the music of creation.
1st Age could be considered “high magic”; though there wasn’t wanton slinging of spells that you find in RPGs, gods, dragons, demons, wizards, fairies, etc. still roamed the earth.
2nd Age, you have no gods, significantly less dragons, demons, wizards & fairies.
By the 3rd Age, there are no gods, no religion, one dragon, the few demons left hide in the deep, the fairies who are left are also largely hidden away and almost forgotten, the dwarves are nearly extinct, and the elves are making their exodus.
As for Gandalf and his fireworks, he probably had been doing fireworks in the Shire long enough that it had become seen as commonplace. Impressive, yes, but the familiarity with it, if only through stories passed down, places it outside the realm of supernatural.
Back to the original topic, I think that ‘low-fantasy’ settings are the ones that make the most sense: magic in the world is diminished and hiding, but adventure awaits, because if you seek magic you can find it. It allows for magic to be, well, magical. I’ve always found high magic settings kind of unsettling because magic is so commonplace that the most bizarre things don’t cause people to bat an eyelash. Necromancers in towers on the edge of town are a commonplace nuisance, you worry more that they will lower your property value than pose a threat to the world… Though it does make it interesting to think that one can find steady, honest blue-collar work ridding towns of unwanted wizards and monsters.
And no, Prydain is not low magic (it’s actually very high magic), but each story ends with some portion of the magic in the world irretrievably lost or destroyed (Achren’s bastions, the Cauldron, the spellbook of Llyr, etc.) until there is none left.