Their goal is to “Democratize AI.” They have done so by open-sourcing the Stable Diffusion text-to-image model and are working on releasing other models, including large language models.
This approach is in stark contrast to the one taken by OpenAI, Facebook, Google, etc. These companies have gated access to ML models like GPT-3 via APIs or invite-only programs. The reasoning is that these models could be used to generate hateful text and images and are generally too dangerous to be released to the ignorant masses.
In a recent interview, Emad Mostaque, the CEO of Stability.Ai and a fascinating thinker, talks about the inevitability of generative and large language models leaking out to the wild. He wants to focus on giving people a framework for the ethical use of AI while giving them the tools to build and train models for their specific uses.
Stability.Ai has struck a deal with Eros Interactive to get access to their massive library of Indian content. Can you imagine what could be trained using that data?
Congratulations to Stability.Ai. I am curious about what this more open (or perhaps reckless?) approach to ML will bring us.
Image generated by Stable Diffusion – Prompt: “A happy robot drinking champagne at a cocktail party at night, oil painting, muted, candid, high resolution, trending on artstation”
For some, matrix multiplication may trigger memories of tedious high school algebra exercises. Last week, this humble mathematical operation was also the topic of a significant breakthrough in machine learning.
Art generated by Stable Diffusion
Background – Matrix Multiplication
Matrix multiplication is the foundation on which many core computational operations are built. Graphic processing, machine learning, computer gaming, etc. – all rely on matrix multiplication. At any given point in time, there are millions of computers doing (probably) billions of matrix multiplication operations. Making this humble operation faster would result in significant computational and efficiency gains.
Why do we want faster matrix multiplication?
Multiplying two matrices involves doing a large number of multiplication and addition operations. For example, multiplying a 4X5 and a 5X5 matrix involves 100 multiplication operations using the traditional matrix multiplication method that has been around since the early nineteenth century. In 1969, a mathematician, Volker Strassen, came up with an ingenious method that reduced the number of operations required by about 10%. This was hailed as a groundbreaking discovery in the world of mathematics.
DeepMind Enters the Arena
This brings us to DeepMind’s paper last week, where they used the AlphaTensor deep learning model to discover a new algorithm for matrix multiplication that is faster by about 10 – 20% than the Strassen method. This is a *colossal deal*! We are seeing a machine learning model find new algorithms to solve material, real-world problems. We have already seen DeepMind make groundbreaking discoveries in computational biology with AlphaFold. We now see applications of its Deep Learning models (based on playing games) to foundational aspects of the modern world. Exciting times are ahead!
I have played video games for thirty years. With two kids and a busy job, I don’t get as much time to play as I would like, but I pick up my Xbox controller whenever I get a chance. Over the last few months, this has meant playing Elden Ring, a role-playing game where you explore and adventure in a fantastical realm called the Lands Between.
Elden Ring – Bandai Namco
I am terrible at Elden Ring, yet I have spent hours playing it over the last six months. I am in awe of the game’s scale, beauty, and challenge.
When not playing video games, I support software development teams. Over the last ten years, I have worked as an Engineering Manager and, more recently, as a CTO at early-stage startups.
Managing and supporting teams is hard. You have to balance competing priorities and make decisions under conditions of ambiguity. Periods of stability can be interrupted by crises. It’s not that different from playing a game like Elden Ring!
As I reflected on why I enjoyed playing some video games more than others, I realized there are parallels between excellent video game design and supporting high-performance teams.
In this post, I explore what makes a video game great and what lessons we can apply from video game design to become better managers.
What Makes a Good Video Game?
Video game design is a vast and evolving topic. However, there are three critical elements to a good video game.
Good vs. Bad Video Design
The Story: What is this video game about, and why should I bother playing it?
A good video game story makes players want to invest their time in learning more about the world and the story. Games like Elden Ring, Horizon Zero Dawn, and God of War have stories that push players to do all sorts of side quests and missions. Exploring the world helps fill out the story, and each task moves the character and the story forward.
Hades – Supergiant Games
The Setting: Where am I going to be spending my time?
Seasoned gamers are familiar with the “one more turn” phenomenon. You want to keep playing because the game world is so darn fun. Dungeons filled with loot (and traps), exciting side missions, and beautiful scenery make the game’s exploration and progression fun. Games like the Mass Effect series make you care not just about the characters but also the broader game world and lore.
Gameplay Loop:How do I play the game?
Elden Ring is brutally difficult, yet I keep returning to the game. The reason is that while challenging, the gameplay is fair and predictable. And I get a real sense of accomplishment after clearing a particularly tricky dungeon or boss encounter. Hades is another game that has wonderfully compelling gameplay. Great video games have a simple yet addictive core gameplay loop. These are the actions that a player is expected to perform most often to make progress in the game. These must be balanced to avoid tedium while presenting fun and challenging experience.
From Video Games to High-Performance Teams
What do video games have to teach us about supporting high-performance teams?
We will approach this by looking at the same attributes that we explored for successful video games:
Story → Vision
Setting → Workplace
Gameplay → The Day-to-Day Work
Vision: Why am I being asked to do this?
A compelling narrative is about selling a vision – what will the player gain at the end of this quest line, boss battle, or challenging project? An honest, well-articulated vision helps give direction to a team. In his viral talk, “Start with Why,” Simon Sinek talks in detail about this “inside out” approach. Having a vision contradictory or inconsistent with the day-to-day work could lead to frustration and a lack of trust. The narrative must be straightforward and backed up with action aligned with the company’s stated values.
Workplace: Where do I spend my time?
A leader must create a workplace that maximizes productivity while allowing creativity, serendipity, and exploration. This is true both for in-person and remote work. Encouraging (reasonable) risk-taking and exploration enable more engaged and motivated teams. A video game with a predictable and tired setting (post-apocalyptic zombie infestations, for example..) is boring. Similarly, an environment that is dull or unpleasant is a drag on motivation and productivity. Psychological safety is also essential. As any player of online games knows, dealing with abuse and cheating makes for a miserable experience. A workplace perceived as hostile and a leader unwilling to support and protect their team will lead to people walking out of the door.
The Day to Day: How I do my work…
A manager must focus on the “gameplay” loop for their team. What are the challenges that stop them from doing their work? For software engineering teams, this could be the ease of making changes, getting pull requests approved, and getting changes into production. I have rage-quit lots of games because “life is too short.” Online games where I keep getting taken out by snarky teenagers with fast twitch reflexes are a particular bug-bear. Elden Ring can also veer into frustrating territory until I realized I could avoid most difficult encounters until I was leveled up and ready. When supporting a team, you need to consider what can be done to remove obstacles for your team. It may mean picking the right battles and knowing when to compromise. Making the workday loop engaging for your team is a critical leadership skill.
Conclusion – Gaming and Learning
Video games are the dominant entertainment and artistic form of our time. Oscar Wilde opined, “Life Imitates Art far more often than Art Imitates Life.” I agree.
Video games have been around far longer than modern software engineering tools such as Agile, DevOps, and other current paradigms. The art of video game design has been refined through decades of experimentation and many, many failures.
Indeed, as managers, most of us will be supporting teams that grew up playing video games. As a medium, video games create interactive, compelling worlds where people enjoy spending their time.
Taking cues from how video games are designed could help us become more effective supporters and advocates for our teams.
Stuart Kirk, former journalist for the FT and now former banker at HSBC got into trouble last week for suggesting that climate change risks are overblown.
Before his suspension, he was the head of Responsible Investing for HSBC asset management.
The “markets”, in all probability, already priced in climate change risks
Climate change adaptation is more pragmatic and likely cheaper than mitigation
By the time climate change hits, we will all be dead anyway. So why bother?
I found the presentation interesting and a little horrifying – in the drunk uncle holding forth – sense. He makes some good points – about the short term nature of markets and investing, about the necessity of climate change mitigation, for example. But the general attitude can be summarised as 🥱🤷♂️.
I am still surprised that after 2008, after COVID, Ukraine and all the other shocks, people like Mr Kirk still think in terms of normal distributions. I.e. the probability of events can be modelled as a bell curve – with very bad or very good events having low probabilities, and predictable “average” events being the most common.
Or to channel mathematician, philosopher and truculent Twitter warrior N. Taleb, the likes of Mr Kirk believe that the impact of climate change to be an ergodic process while it is most definitely not.
Doing a Google search for “Ergodicity” will lead you to baffling mathematical and statistical explanations. But it is, at its core, an intuitive concept. In a non-ergodic system, things that are true for the aggregate may not be true for the individual.
In Mr Kirk’s presentation he plots economic growth from the 1930s to the present day and states, pretty much, that the “line goes up” despite world wars, economic upheaval, recessions etc. He uses this trend to assert that we will be fine despite the risks of climate change. The benefits of a growing economy will overcome the downsides of climate change.
However, the story of aggregate growth over the last 100 years hides tales of individual ruin.
For example, someone who invested all their savings in tech stocks in 2002 probably didn’t have anything left to make money when the market finally moved up. For those unlucky investors, it was game over. Therefore, we are modelling a process that is non-ergodic (individual outcomes can be radically different than aggregate outcomes) as an ergodic process.
So, what does this have to do with climate change?
I believe that the effects of climate change make our economic system even more non-ergodic. It makes it much more likely to have extreme events – heat waves, wild fires, hurricanes, droughts, etc. This makes modelling based on aggregate probabilities a little suspect. Sure, you could increase insurance premiums for coastal communities to account for higher flooding risk. This is what Mr Kirk means by the risk being “priced in”. But what happens when entire communities are wiped out due to an unprecedented storm surge, or heat wave, or forest fire?
Climate change adds more chaos to a complex system. It heightens the likelihood of extreme events that have catastrophic outcomes. Adaptation measures are necessary but they will do little to mitigate the impact of “black swan” events. So it doesn’t matter how complex your modelling is, and how sophisticated your investment strategy is. If you die due to a freak hurricane, you are done.
The likes of Mr Kirk are making a category error. The only way to “win” in an non-ergodic system is to survive. We should be thinking of what can be done to ensure that we don’t face catastrophic loss, so that we can continue to reap the benefits of growth in the future.
You live in the up-and-coming suburb of Cryptoville and you want to buy a house. It costs $1m.
There might be some transaction fees involved, but you won’t actually know how much the fees will be until you complete the transaction. Oh, you are not competing with anyone to buy the house, it’s just a transaction fee. Can’t be too bad right?
On the day of closing, the transaction goes through. The transaction fees are $250,000! And there was no way to tell until you tried to buy the house. It’s just the way things work in Cryptoville..
This is pretty much what happened on Saturday when Yuga Labs, the company behind the Bored Ape Yacht Club, held a much anticipated virtual land / NFT sale on the Ethereum network. Gas fees (i.e. transaction fees on Ethereum) spiked as the network coped with thousands of ApeCoin holders looking to buy some virtual land for their virtual Apes.
The shocking thing was that it caused the entire Ethereum network to clog up – raising transaction costs for everyone – not just those looking to buy virtual land. Folks looking to buy NFTs valued at under a dollar were seeing transaction fees of $3,500!
This points to a serious, and well-known, issue with throughput on Ethereum. It does not scale under load. Perhaps the long-delayed migration to Proof of Stake may change this – when it happens.
But – do you know what happened to the “high-performance” blockchain Solana on Saturday? You see where this going..
Footnote Ethereum can only process about 15 transactions per second. It is just the way it is designed. However, miners can be incentivized to process transactions by increasing gas (transaction) fees. This is what happened on Saturday – as the demand to mint NFTs skyrocketed, so did the transaction fees. Gas fees have since come down, but it shows the big issues that Ethereum continues to face as it remains the de-facto standard for blockchain development.
I have been trying to avoid the whole Elon Musk / Twitter drama, but it has been challenging. I am ambivalent about whether Mr. Musk’s takeover of Twitter is a good or bad thing. My vibe is 🤷🏾♂️.
But, I do have an issue with one of Mr. Musk’s ideas: open-sourcing the Twitter algorithm to ensure there is no “bias.”
I think this is disingenuous, and Mr. Musk is playing to his (adoring) audience a little bit.
It is improbable that there is the “one true algorithm” at Twitter. They probably use a combination of machine learning-based recommendation models with other systems such as entity and intent detection. Take a look at Twitter’s engineering blog to see how much ML drives recommendations on the social network.
So, if the intention is to look at the code and delete any (left-wing | right-wing) bias, things will be.. difficult.
Now, a discussion should be had about how the ML models are trained and if there are any biases in the labeled datasets that are used to drive recommendations, detect abusive content, etc. This is a complex problem, however!
An important effect of the pervasive deployment of ML technologies is that it makes computing *probabilistic* instead of *deterministic*. i.e., we know what is likely to happen, but it is difficult to predict what *will* happen.
This paradigm shift makes it very difficult to point the finger at one or more woke/radical/reactionary programmer who decides to censor or advocate for free speech.
Mr. Musk knows all this, of course. The entire Tesla “full self-driving” stack is built on ML. So, perhaps, a little bit of intellectual honesty might lead to a more interesting discourse about bias.
