science-fiction – Rushi Luhar

Explaining Reinforcement Learning with Human Feedback with Star Trek

February 14, 2023February 10, 2023 by rushi

Microsoft announced today that it will include results from a Large Language Model based on GPT-3 in Bing results. They will also release a new version of the Edge browser that will include a ChatGPT-like bot.

GPT-3 has been around for almost two years. What has caused this sudden leap forward in the capabilities of Large Language Models 🤔?

The answer is – *Reinforcement Learning From Human Feedback* or RLHF.

By combining the capabilities of a large language model with those of another model trained on the end-users preferences, we end up with the uncannily accurate results that ChatGPT seems to produce.

Ok – but how does RLHF work? Let me try and explain with a (ridiculous) analogy.

In the Star Trek series, the Replicator is a device that can produce pretty much anything on demand.

When Captain Picard says, “Tea, Earl Grey, Hot!” it produces the perfect cup of tea. But how might you train a Replicator? With RLHF, of course!

Let’s see how:

1. Feed the Replicator with all the beverage recipes in the known universe.

2. Train it to try and predict what a recipe would be when given a prompt. I.e. when a user says “Tea, Earl Gray, Hot!” – it should be able to predict what goes into the beverage.

3. Train *another* model – let’s call it the “Tea Master 2000” with Captain Picard’s preferences.

4. When the Replicator generates a beverage, the Tea Master responds with a score. +10 for a perfect cup of tea, -10 for mediocre swill.

5. We now use Reinforcement Learning (RL) to optimize the Replicator to get a perfect ten score.

6. After much optimization, the Replicator can generate the perfect cup of tea – tuned to Captain Picard’s preferences.

If you substitute the Replicator with an LLM like GPT-3, and substitute the Tea Master with another ML model called the *Preference* model, then you have seen RLHF in action!

It is a lot more complicated, but I will take any opportunity to generate Star Trek TNG-themed content 🖖.

Review: Ninefox Gambit by Yoon Ha Lee

September 10, 2016 by rushi

Ninefox Gambit by Yoon Ha Lee
My rating: 4 of 5 stars

I read a lot of science fiction. I find reading science fiction diverting, stimulating and usually a lot of fun. I finish most books in three to five days depending on my schedule and what else is going on in my life. I struggled with Ninefox Gambit. It took me 3 weeks to finally finish it. I abandon books if I am not having fun after two or three sittings. I am glad I didn’t give up on Ninefox Gambit.

We are thrown right in the deep end at the beginning of Ninefox Gambit. Cheris is a soldier and a gifted mathematician. She serves the Hexarchate; six factions or guilds that govern her part of the universe. Each faction is responsible for a facet of life within the Hexarchate. Cheris is a part of the Kel, the military faction. Ninefox Gambit starts with Cheris involved in a bloody skirmish. We get some idea of how combat works in the Hexarchate. Kel soldiers are able to use exotic weapons by deploying in a formation that uses the effects of the Hexarchate “calendar”.

I know. I was baffled too. We get given no indication of what the calendar is or how a formation works. I knew there was something important going on, but I felt too lost to be able to follow what was happening.

I put away my Kindle and picked up another book. Yet, I kept thinking of Cheris and the world of Ninefox Gambit. So I picked up where I had left off and powered through. I am glad I did, because we are swiftly introduced to General Shuos Jedao – disembodied, disgraced and quite possibly insane. Jedao is immortal and imprisoned by the Kel hierarchy after causing a brutal massacre 400 years ago. He may or may not be crazy but is a brilliant military tactician and is used by the Kel when his expertise is required.

An important world (the wonderfully named Fortress of Scattered Needles), is taken by heretics who install their own calendar in rebellion against the doctrines of the Hexarchate. The Fortress is protected by unassailable defences and lies in a strategic sector. Cheris is chosen as Jedao’s anchor – together they command the Hexarchate’s forces as they attempt to subdue the rebellion and retake the fortress.

We get to learn a lot more about the world through conversations between Cheris and Jedao as well as short vignettes from other characters caught up in the action. There are plots within plots and a lot of political intrigue. There are games with exotic rules and flashbacks to Jedao’s life and the events that led up to his immortality and imprisoning. There is also violence, and lots of blood and gore.

At times, Ninefox Gambit reads like conventional military science fiction. Exotic weapons (deadly fungus anyone?), spies and shouty sergeants. Yet, all of this action makes sense in the context of the structure of the Hexarchate. The world is governed through a combination of indoctrination and brute strength. Cheris and Jedao are the tip of the spear that is intended to destroy the rebels.

If you are still with me, you probably know that Ninefox Gambit relies on the reader being somewhat familiar with the tropes of science fiction and fantasy. I (and other readers on Goodreads) were reminded of Anne Leckie’s Imperial Radch trilogy. There are similarities – we have an unreliable narrator in Cheris and a brutal regime attempting to suppress a rebellion. Just like the Radch trilogy, Ninefox Gambit is deeper and a lot more interesting than your run of the mill military science fiction.

Yoon Ha Lee has built a compelling, and challenging, universe – one that I hope will be explored in further volumes in the “Machineries of Empire” series.

View all my reviews

Review: Revenger by Alastair Reynolds

July 24, 2016July 24, 2016 by rushi

Revenger by Alastair Reynolds
My rating: 4 of 5 stars

Review based on a copy sent to me by NetGalley

Revenger appears to be the first instalment in an intriguing new science fiction series by Alastair Reynolds. This fast paced, character driven novel is an enjoyable departure from the dense, exposition-heavy space opera that Alastair Reynolds has been writing for the last 20 years.

We follow the story of Arafura Ness, a teenager with a sheltered life and a love of books as she ventures out from her father’s home in search of adventure with her more outgoing (and reckless) older sister Adrana.

Arafura comes from Mazarile, one of thousands of planetoids and habitats in a ruined Solar System that has seen multiple civilisations come and go. The economy is based on finding and exploiting artefacts from previous, more advanced civilisations. We have space ships with salty, and cynical crews sailing solar winds, exploring abandoned habitats looking for treasure. Arafura and Adrana start their adventure in one of these ships. Things go awry pretty quickly and the sisters are separated. The main plot of the book follows Arafura as she attempts to find her sister.

We get many hints to the shape and structure of the universe of Revenger. However, the book is written from the point of view of a teenager coming to grips with a chaotic and violent world and there are no dull expository passages.

There are a few other characters: mainly crew mates of Arafura and Adrana, but the story is very much Arafura’s. The dialogue can be a bit awkward at times, but I enjoyed following Arafura’s journey. The story builds to a violent and bloody climax. I didn’t find the violence gratuitous and it made sense in context of the plot and the wider world of Revenger.

Fans of Alastair Reynold’s work will find much to enjoy here: a strong female character, strong action scenes and a fantastic world to explore. The book has also been clearly written to attract readers of the burgeoning “Young Adult Fiction” genre, and I think it will be a great read for those readers who are looking for more science in their YA fiction.

View all my reviews

Buy Revenger from Amazon (affiliate link):

Review: Oryx and Crake by Margaret Atwood

July 24, 2016February 28, 2016 by rushi

Oryx and Crake by Margaret Atwood
My rating: 3 of 5 stars

“Oryx and Crake” is a beautifully written book set in a dystopian future where genetic engineering has gone wrong.

The book follows two plot lines. “Snowman” finds himself alone; slowly starving to death and doubting his sanity in a world that has been devastated by plague. His days are spent scavenging and hiding from mutant pigs (“pigoons”) and nasty wolf / dog hybrids (“wolvogs”). There are also the “Crakers”, gentle, genetically engineered humans that seem to be designed for this post-apocalyptic world. The Crakers see Snowman as a sort of mentor. We find more about Snowman’s relationship with the Crakers as the book progresses.

The second plot strand is set in the past. This is before the plague when Snowman was known as Jimmy. Corporations run fabulously appointed enclaves (called Compounds). Jimmy grows up in one of these compounds, alienated from his scientist father and coming to terms with being abandoned by his mother. The world outside the compounds, the “pleeblands”, is rife with poverty, crime and those people who are not lucky enough to work for one of the compounds.

Jimmy meets Crake, a strange and brilliant teenager while in high school. We follow their lives through to adulthood. The world, as described by Ms. Atwood, is teetering on the brink. Almost everything is available for sale, and the Compounds follow some ethically and morally questionable business practices. We come to understand how Snowman’s world came about. We also meet Oryx, a woman who both Jimmy and Crake fall for and who has a compelling and tragic story herself.

“Oryx and Crake” is the first installment of the “MaddAdam” trilogy. While I enjoyed reading the book and marvelled at Ms. Atwood’s writing; it was clear that Ms. Atwood does not approve of genetic engineering and does not hold the capitalistic motive in high regard. This results in a slightly laboured and cynical book. I might change my mind after reading the other two books in the MaddAdam trilogy. But for now, “Oryx and Crake” gets an average rating.

View all my reviews

Buy Oryx and Crake from Amazon (affiliate link):

Review: The Long Way to a Small, Angry Planet by Becky Chambers

July 24, 2016December 30, 2015 by rushi

The Long Way to a Small, Angry Planet by Becky Chambers
My rating: 4 of 5 stars

I read “The Long Way to a Small, Angry Planet” (TLW) in one sitting on a long flight. At the time, it felt like a particularly enjoyable story arc from Star Trek TNG or perhaps a binge session of Firefly. A week or so later, I keep thinking about the book, about the characters and about the patched up freighter and far flung moons with their plucky and weird colonists that inhabit TLW.

After the first chapter, I thought the book would centre around Rosemary and her journey from desperate runaway to some sort of ass kicking space warrior-scribe. I was wrong. The book is not just about Rosemary but about the entire crew of the Wayfarer and there is little in the way of ass-kicking. The pacifist captain, the rambunctious techs, mysterious navigator(s) and all the other colourful (literally in a few cases) characters that inhabit TLW have depth and agency.

The plot revolves around a long journey undertaken by the Wayfarer, a sort of space highway construction ship, to the Small Angry Planet of the title. Along the way, we visit markets, colonies, and planets while getting to know the crew and how the universe of the book works. The structure of the book may be conventional, yet it has a lot of say about gender, identity, violence and coming to term with one’s past. TLW is open about its politics: the captain of the Wayfarer is a pacifist, the doctor comes from a species that chose voluntary extinction after a decades of brutal warfare, and my favourite character has to consciously tone down her affection for her human crew mates because we are so weird about public displays of affection. Even the most curmudgeonly character has redeeming features.

TLW may not seem appealing if you like your science fiction to be of the military variety, or if you are a fan of hard science fiction from the likes of Alistair Reynolds. It certainly is different to the usual science fiction books I read, but I found it rewarding. Ms. Chambers clearly cares deeply about the Universe and the characters she has created. There is none of the nihilism and little of the violence that can be off putting about a lot of modern science fiction. TLW is character driven and while there are a few expository data dumps, things never get tedious.

I look forward to more books by Ms. Chambers and am glad that she is currently working on a companion piece that is set in the same Universe as TLW. A strong recommend from me.

View all my reviews

Buy The Long Way to a Small, Angry Planet from Amazon (affiliate link):

Review: Slow Bullets by Alastair Reynolds

June 13, 2015 by rushi

Slow Bullets by Alastair Reynolds
My rating: 3 of 5 stars

Review based on a copy received from NetGalley and Tachyon Publications

Alastair Reynold’s Slow Bullets is a novella exploring issues of identity, memory and revenge. There are echoes of Iain Bank’s The Use of Weapons as well as some of the ideas explored in the Poseidon’s Children trilogy. The tone and setting of Slow Bullets is quite different to that of the gothic space opera (Revelation Space) or generation spanning science fiction (Poseidon’s Children).

The majority of the action is set on the prison transport Caprice as it recovers from a calamitous malfunction. The novella’s protagonist Scur and her fellow passengers awaken from hibernation to find the ship in orbit around a frozen planet and suffering from an acute case of bit rot.

Caprice’s passengers include war criminals from the two opposing religious factions. The war was over and a cease fire declared as the ship set off on it’s ill fated mission. As the ship’s systems fall apart, Scur and her fellow passengers have to deal with religious tensions, long simmering vendettas, as well as figuring out how to preserve millennia’s worth of cultural and scientific knowledge.

I quite enjoyed Slow Bullets. However, it feels more like a short story that was extended to a novella than a novel’s worth of ideas condensed to the shorter format.

View all my reviews

Review: Station Eleven by Emily St. John Mandel

June 8, 2015 by rushi

Station Eleven by Emily St. John Mandel
My rating: 4 of 5 stars

Station Eleven is a book that cannot be categorised easily. Is it science fiction? Is is literary fiction? Is it post-apocalyptic fiction? It is all of the above, and yet it does not conform to the tropes of genre fiction.

The apocalyptic event – a pandemic caused by a highly infectious and deadly strain of flu straddles the two main plot arcs in Station Eleven. Before the end of the world as we know it, we follow the life (in reverse) of Arthur Leander, a famous actor, now in his middle ages and playing Lear on stage. The second, post collapse arc, follows Kirsten Raymonde an actor with the Travelling Symphony as it moves between small settlements on the shores of Lake Michigan performing Shakespeare, playing classical music and avoiding trouble as best as they can.

Reviews on Goodreads, and on other similar sites are full of quotes from the book, and for good reason. Emily St. John Mandel’s prose has a simple, descriptive style that manages to convey both the beauty and the desolation of her post apocalyptic world. We find beauty and grace in burnt out houses, dark forbidding forests and abandoned rust streaked airplanes parked nose to tail at the airport, going nowhere. There is danger in the form of “The Prophet” and his followers as they stalk the Travelling Symphony. Yet, this book is not like The Stand or perhaps Justin Cronin’s The Passage. The minutiae of survival and self defence are ignored as the book focuses on the emotional impact of societal collapse on those that lived through it and those that were too young to remember the world as it was but are surrounded by the decaying scaffolding of civilisation.

The book is a meditation on art and, in a sense, of mortality. Arthur Leander, in some ways the central character of this book is not remembered for his films or his wealth. He lives on through small acts of generosity, giving the eponymous “Station Eleven” comic book to Kirsten when she was a child, or through photographs and articles in decaying gossip magazines.

So why 4 out of 5 stars? Despite the beautiful writing, I found myself skimming passages. Arthur Leander was a fabulously wealthy, successful actor and serial divorcee, but not a particularly interesting character. There are almost too many characters and side plots that don’t seem to add much to the story. I found myself impatient to go back to the Travelling Symphony and to Kirsten as they made their way around post-apocalyptic Michigan.

A strong recommend from me for fans of literary fiction who want to dip their toes into the burgeoning post-apocalyptic literature genre, and for fans of science fiction curious about the tropes of literary fiction. I thoroughly enjoyed this beautiful and inspiring book.

View all my reviews

Review: Poseidon’s Wake by Alastair Reynolds

May 18, 2015 by rushi

Poseidon’s Wake by Alastair Reynolds
My rating: 4 of 5 stars

Poseidon’s Wake is the third volume in the Poseidon’s Children series by Alastair Reynolds. The events of this book are set a few centuries in the future from the second book – On The Steel Breeze. The main protagonists are still part of the Akinya clan. We find Mpozi, Goma and Ndege on Crucible and Kanu in the Solar System.

The book explores the results of the arrival of the Watchkeepers and the aftermath of the Mandala event at the conclusion of “Steel Breeze”.

Let me be honest – I found the book hard going, yet worthy of the four stars I have given it. There are long passages meditating on the meaning of life and the role of belief. Stay well clear if you are looking for action scenes or military science fiction. This is very much in the vein of Existence by David Brin. We have a McGuffin – vast alien artefacts on the planet Poseidon. The plot revolves around separate expeditions from the Solar System and from Crucible to the hitherto unvisited system following the receipt of a mysterious transmission.

Along the way, we find the machine civilisation explored in the first two books, we find super intelligent elephants as well as inscrutable aliens. Reading this reminded me of Rendezvous with Rama – it has the similar mix of hard science fiction as well the plot point of humans trying to figure out the motivations of an unknowable alien. It is a fitting conclusion to the series and a book that has stayed with me more than I expected it to.

SIDENOTE – There is one thing I never figured out about these books. Where are the White people? We have a future where all the conversation happens in Swahili, or Mandarin or Portugese – but no English. We have characters that are of different ethnicities, but no WASPS. Whats up with that?

View all my reviews

Review: Daemon by Daniel Suarez

March 20, 2014 by rushi

Daemon by Daniel Suarez
My rating: 3 of 5 stars

TLDR – Daemon starts off as a promising techno-thriller with a somewhat plausible premise but ends up being a run of the mill action roller coaster with killer robots. It’s a fun, if unsatisfying genre piece.

Daniel Suarez knows his domain – contemporary / near future technology and it’s implications. What I am also quite sure about that he has a rather pessimistic view of where things are going. Daemon starts as a crime procedural with a small town cop trying to solve two seemingly unconnected deaths that appear accidental. We quickly find out that there is a rather sinister force behind these deaths. We encounter disenchanted, anarchist-libertarian hackers, mysterious computer programmers who are not quite who they seem, and lots of sinister government types who simply know whats best for everyone.

The key character is the eponymous Daemon – a networked, non sentient computer system that is a dead computer genius’s gift to humanity. The Daemon has very specific plans (though they are never revealed – we will have to wait for the sequel) for humanity and it goes about recruiting brilliant, motivated followers through a variety of somewhat plausible means. We have entertaining descriptions of computer games and call centre software amongst other things. I really enjoyed this, the first half of the book. A particular standout was the police / FBI raid on the dead computer genius’s computerised mansion which I found most satisfyingly and gratuitously violent and explosively entertaining.

The book jumps forward a few months around the half way mark. This is where things get problematic. There are large passages that involve discussions between nameless “important” people in the FBI, CIA, NSA and other alphabet agencies as they wring their hands and try and figure out just exactly what is going on. Yes, we know government bureaucrats are clueless, thank you. The episodic, multi-character structure of the book also becomes a problem here. There are a number of characters who fade in and out. There is a particular character, a FBI special operations type fellow, who must be based on someone the author dislikes. He appears in two long passages, and appears to take a huge amount of punishment: being blown up, burnt, shot at, attacked by killer robots, being thrown off a car, etc. But it is difficult to really care too much because we don’t know anything at all about this particular, long suffering sap. The book builds up to an explosive climax involving a long car chase and,yes, more killer robots.

I enjoyed reading Daemon – just like I enjoy big budget sci-fi / action movies or playing first person shooters. There are some neat touches, cool technology, lots of explosions, and killer robots. But, in keeping with genre tropes, we also get gratuitous violence, paper thin characters, and an inconsistent plot. A strong recommendation for those who like computer games and are anarcho-techno-libertarians. An entertaining and somewhat lightweight read for the rest of us.

View all my reviews

Review: The Shadow of the Torturer by Gene Wolfe

November 19, 2013 by rushi

The Shadow of the Torturer by Gene Wolfe
My rating: 3 of 5 stars

My rating : 3.5/5

“The Shadow of the Torturer” is a difficult book to read and to review. The New Sun series of books by Gene Wolfe are often described as the best fantasy ever written. I approached “The Shadow..” expecting to be blown away, but ended up a little baffled and quite intrigued.

The book is narrated by Severian, the titular Torturer. The book is structured as Severian’s memoir written some unspecified time in the future. Severian informs us at the start of his memoirs that he has an eidetic memory. His tale is structured as a series of vividly remembered vignettes that loosely follow Severian’s journey from his time as an apprentice Torturer in the Citadel to his banishment the beginnings of his travels.

You would expect somebody with total recall to be the most objective narrator possible. Yet, following Severian’s story often leaves us perplexed. To put it briefly, things happen to Severian. He behaves like a toy that is wound up and left to rattle across the city of Nessus. He meets characters, gets into and out of difficult situations in strange places, yet emerges unscathed without much comment or reflection. This is not because of poor writing, but at this early stage in the series seems like a clear plot device. I suppose this is the beauty of the book. You know something is up, but it is not clear exactly what and why.

The descriptions of a (post apocalyptic?) Earth are very well done. The city of Nessus, where all the action takes place, is intriguing. It is somewhere in the Southern Hemisphere, and its people live in a monarchial, class based, almost medieval society with some hints of far future technology. Think medieval swordsmen riding genetically modified uber-horses. The story is set in the far future, and the author often throws in passages that convey a sense of decay and melancholy. There are no “information dumps”, and the history of this society is not something that we know about. The “Urth” is as it is, and forms an intriguing background to Severian’s journey.

Since this is the first in a series of books, I will not comment much on the characters. We meet many, but we don’t get to know any of them, since they are all described to us by Severian. He may have total recall, but as the story progresses, he is quite possibly a lousy judge of character. You may find this endearing or irritating. At this stage, I am willing to give the author some leeway.

View all my reviews

Further Reading