John Ronald Reul Tolkien in 1916, shortly before beginning what would become The Silmarillion.
John Ronald Reul Tolkien had been born in Bloemfontein's, South Africa, in 1892 to British parents. His father died when he was just three years old whilst the rest of the family were visiting the UK on holiday. Remaining in the UK, Tolkien and his brother were raised by their mother in and around Birmingham, in conditions of some financial distress. Tolkien's mother died when he was twelve from complications arising from diabetes and he was raised by a priest who had been a friend of the family. Tolkien attended Oxford University and married his childhood sweetheart, Edith Bratt, after a courtship complicated by interference from both their families on religious grounds (Tolkien was a Catholic and Edith a Protestant; she later converted, with some regrets).
In 1916 Tolkien was deployed to the Western Front to fight as part of the British Army in the First World War. Tolkien spent just four months in the trenches, although the horrors he saw during that time never left him: those four months included the bulk of the Battle of the Somme. In October of that year Tolkien came down with trench fever and was invalided back to England. Repeated bouts of sickness kept him confined to the home country. With Edith only able to visit occasionally, Tolkien was bored out of his mind. He was already a keen writer, poet and artist, and had already used fantastical imagery in his amateur works: he had already painted a great blazing tree of light in 1916 and had written a poem called The Lonely Isle. He had also started developing his own imaginary languages, driven by a love of philology inherited from his mother (who had taught him Latin as a young child). He had no greater plan in mind for these works but during his convalescence he decided to start writing an actual story, about the arrival of a great warrior at a glorious city called Gondolin, one of the few surviving strongholds in the midst of a great war. The Fall of Gondolin was the first of what he came to call The Book of Lost Tales, consisting of stories drawn from a common backdrop but varying immensely in tone and content.
Ted Nasmith's rendition of the meeting of Beren and Luthien, the most personal and favourite of Tolkien's own stories.
Others soon followed. A few months later Tolkien and his wife went on a picnic and Edith danced for him among the flowering hemlock. This inspired Tolkien to write his epic romance, The Lay of Leithian, better-known as The Tale of Beren and Luthien, which became a core part of Tolkien's developing legendarium.
With the war ending, Tolkien took a job working on the Oxford English Dictionary before becoming a university lecturer and professor, first at Leeds and then back to Oxford. He became Rawlinson and Bosworth Professor of Anglo-Saxon at Pembroke College in 1924. During this time he continued working on The Book of Lost Tales, adding many more episodes to the book and developing a vast, mythological backstory. In this story supernatural beings known as Valar (servants of the One God Eru, also called Illuvatar) create the world and rule it as a paradise, until one of their number, Melkor (later Morgoth) turns against them and brings disquiet and corruption into paradise. Melkor is responsible for numerous conflicts, culminating in him seizing control of three powerful, magical jewels and securing them in a stronghold in the central continent of the world (hence, the "Middle of the Earth", although the name also drew on Midgard and other mythological roots). During his villainous antics, Melkor had betrayed the immortal "gnomes" and suffered their retribution in the form of a bloody crusade launched against his forces in Middle-earth over the course of centuries. Deciding that "gnomes" wasn't quite right, Tolkien recast them as the heroic and otherworldly elves, to the relief of the many imitators who came after. Eventually, with the assistance of the supposedly "lesser" races of dwarves and men (not to mention the intervention of the Valar), the elves defeat Melkor and he is driven from the world in defeat, although only at the cost of the annihilation of parts of Middle-earth in a titanic flood.
Tolkien's grand mythological cycle was complete in concept, but Tolkien found structuring it and making it more comprehensible to general readers to be difficult. He tried creating a framing structure in which a shipwrecked mariner from the modern age washes ashore on the Lonely Isle of Tol Eressea (having somehow found the "Straight Road" from our world to that of Valinor, the land of the Uttermost West) and hears about the Lost Tales from his hosts, but found this unsatisfying. He continued working on the legends and polished them into a more familiar form, changing names and races and events and even the book's title - by 1930 The Book of Lost Tales had become The Silmarillion - but still could not find a satisfying way of presenting his world of Middle-earth as he wanted. But, as is usually the way, inspiration struck from an unexpected quarter.
The Hobbit, or There and Back Again, published in 1937 and a vital moment in the development of the epic fantasy genre.
In or around 1930, Tolkien was marking papers when he found a blank sheet of paper left in the middle of an essay. On a whim, he wrote down "In a hole in the ground there lived a hobbit," and put the paper aside. Later, he picked it up and became intrigued by what exactly a "hobbit" could be. He resolved to find out.
Over the course of the next six years, whilst working hard as an academic and tinkering with The Silmarillion, Tolkien fleshed out his new story by inventing episodes for it as bedtime tales for his four children (who successively grew up hearing it). He set it down on paper, and later a typewriter, before leaving it unfinished. It was only because a former student-turned-friend, Susan Dagnall, had become employed by George Allen and Unwin, a London publishing house and had heard about the book, that it was ever finished.
The Hobbit, or There and Back Again, was published in September 1937 to considerable critical acclaim, winning Tolkien several awards and selling well in both the UK and the United States. Tolkien's publishers quickly asked for a sequel but Tolkien had not planned one, and had ended the book by saying that Bilbo Baggins lived long and happily to the end of his days. He could not think of a way of writing a sequel that did not contradict this. Nevertheless, he tried and by Christmas had started The Hobbit II, another light-hearted adventure about hobbits. Among the ideas he had placed in the first novel but had not developed fully was an off-page, secondary villain called "The Necromancer" and the backstory for a curious magic ring, discovered by Bilbo and granting the bearer the ability to become invisible. Keen for the second book to develop naturally from the first, Tolkien followed up on these plot points and made them more central to the narrative, eventually hitting on the idea that the Necromancer was the creator of the ring and that he was looking for it. The story took a darker turn when Tolkien had his band of hobbits chased across the Shire by a sinister Ring-wraith, with the language and atmosphere both becoming more adult and oppressive. Tolkien chose to remain in this mode. A final element clicked into place when Tolkien suddenly realised that the Necromancer could be Sauron, a minor villain in The Silmarillion (although a primary antagonist in The Tale of Beren and Luthien) whose fate had not been revealed at the end of that story. Tolkien's new story was not going to be just a sequel to The Hobbit, but also The Silmarillion itself, elevating it to a new level in Tolkien's eyes and also resulting in its new and decidedly more epic title: The Lord of the Rings.
In the event, it would be seventeen years before the new book would be published and the rest of the SFF field was not standing idle.