Tech

Google, Microsoft and the Search Wars

by

A demo cost Google’s shareholders $100bn dollars last week. Why?

Google’s Share Price after the Bard event

Google has dominated search and online advertising for the last twenty years. And yet, it seems badly shaken by Microsoft’s moves to include a ChatGPT-like model in Bing search results. 

Why is this a threat to Google?

1️⃣ Advertising: Google’s revenues are driven by the advertisements it displays next to search results. The integration of language models allows users to get answers – removing the need to navigate to websites or view ads for a significant subset of queries.

2️⃣ Capital Expenditure: Search queries on Google cost around $0.01 (see link in the comments for some analysis). Integrating an LLM like ChatGPT *could* cost an additional 4/10th of a cent per query since the costs of training and inference are high. Even with optimization, integrating LLMs into Google search will increase costs in running search queries. According to some estimates, this directly impacts the bottom line to almost $40bn. 

3️⃣ Microsoft’s Position: Bing (and, more broadly, search) represents a small portion of Microsoft’s total revenues. Microsoft can afford to make search expensive and disrupt Google’s near-monopoly. Indeed Satya Nadella, in his interviews last week, said as much (see comments). 

4️⃣ Google’s Cautious AI Strategy: Google remains a pioneer in AI research. After all, the “T” in GPT stands for Transformer – a type of ML model created at Google! Google’s strategy has to sprinkle AI in products such as Assistant, Gmail, Google Docs, etc. While they probably have sophisticated LLMs (see LaMDA, for example) on hand, Google seems to have held off releasing an AI-first product to avoid disrupting their search monopoly. 

5️⃣ Curse of the demo: Google’s AI presentation seemed rushed and a clear reaction to Microsoft’s moves. LLMs are known to generate inaccurate results, but they didn’t catch a seemingly obvious error made by their BARD LLM in a recorded video. This further reinforced the market sentiment that Google seems to have lost its way.

References and Further Reading

Ben Thomson’s “4 Horsemen of the Tech Recession”

by

In the last month, we have had huge layoffs across technology, yet the “real economy” seems robust. What is going on?

Meta is making 2023 ‘a year of efficiency’. Microsoft, Alphabet, and many other companies have stated economic headwinds as the reason for letting thousands of people go. 

However, last week, the US posted the lowest unemployment numbers in 50 years(!) while adding half a million jobs. 

Ben Thomson discusses this in this week’s excellent Stratechery article. 

He points to 4 factors that are causing this disconnect:

1️⃣ 😷 The COVID Hangover -> Companies assumed COVID meant a permanent acceleration of eCommerce spending. Customer behavior has reverted (to a certain extent) to pre-pandemic patterns

2️⃣ 💻 The Hardware Cycle -> Hardware spending is cyclical. After bringing forward spending due to the pandemic, customers are unlikely to buy new hardware for a while.

3️⃣ 📈 Rising interest rates -> The era of free money is over. Investing in loss-making technology companies in anticipation of a future payout is no longer attractive.

4️⃣ 🛑 Apple’s Application Tracking Transparency (ATT) -> ATT has made it difficult to track the effectiveness of advertising spending. This caused enormous problems for companies like Meta, Snap, etc. that rely on advertising.

Big Tech’s Layoffs, AI, and the Closing of the Productivity Gap

by

Big Tech has let go of thousands of workers in the last couple of months. In addition to the end of the era of cheap money and a broader economic slowdown, this story may have another angle.

This is the impact of AI and the possible closing of the “Productivity Gap.” 

The Productivity Gap is a phenomenon where workers’ output, especially in developing countries, has been growing slower than expected. The shift to cloud computing and SaaS business models in the mid-2010s led to an explosion in both the valuations of technology companies and increases in the productivity of individual engineers and teams. A small startup could spin up and scale a business faster than ever. 

Fast forward to the mid-2020s, and suddenly cloud computing is a commodity. Innovative Frameworks from the last decade, like React, Spring, and others, are bloated and complex. 

For the last few years, companies like Meta, Alphabet, and Microsoft could hedge their bets and grow their teams because they were less likely to become disrupted by a small startup. Hoarding talent and doing “acqui-hires” was a feasible strategy.

Explaining the Tech Layoffs

Now there is once more a disruptive technology on the horizon. Generative AI Models are making giant leaps – a small team of ML-native programmers could build something that could blow incumbent services out of the water. 

Alphabet’s panic over OpenAI’s ChatGPT is a case in point. Suddenly it doesn’t make sense to hoard talent to work on a platform that is about to be irrelevant. 

AI-enabled software and infrastructure could close the productivity gap and fuel the rise of disruptive startups. 

The incumbents are then cutting costs and preparing themselves for the next round of disruption by making significant investments in AI. 

It no longer makes sense to hoard programmers when the entire industry could undergo a paradigm shift similar to that brought about by Cloud Computing 15 years ago.

The brutal layoffs we have seen in the last three months could be the result.

The Limits of Generative AI

by

AI is having a moment. The emergence of Generative AI models showcased by ChatGPT, DALL-E, and others has caused much excitement and angst. 

Will the children on ChatGPT take our jobs? 

Will code generation tools like Github Copilot built on top of Large Language Models make software engineers as redundant as Telegraph Operators? 

As we navigate this brave new world of AI, prompt engineering, and breathless hype, it is worth looking at these AI models’ capabilities and how they function. 

Models like the ones ChatGPT uses are trained on massive amounts of data to act as prediction machines. 

I.e., they can predict that “Apple” is more likely than “Astronaut” to occur in a sentence starting with: “I ate an.. “.

The only thing these models know is what is in their training data. 

For example, GitHub Copilot will generate better Python or Java code than Haskell. 

Why? Because there is way less open-source code available in Haskell than in Python. 

If you ask ChatGPT to create the plot of a science fiction film involving AI, it defaults to the most predictable template. 

“Rogue AI is bent on world domination until a group of plucky misfit scientists and tough soldiers stops it.” 

Not quite HAL9000 or Marvin the Paranoid Android. 

Why? Because this is the most common science fiction film plot.

Cats and Hats

Generative AI may generate infinite variations of a cat wearing a hat, but it has yet to be Dr. Suess. 

AI is not going to make knowledge work obsolete. But, the focus will shift from Knowledge to Creativity and Problem-Solving. 

Machine Learning and its consequences

by

Machine Learning has brought huge benefits in many domains and generated hundreds of billions of dollars in revenue. However, the second-order consequences of machine learning-based approaches can lead to potentially devastating outcomes. 

This article by Kashmir Hill in the New York Times is exceptional reporting on a very sensitive topic – the identification of abusive material or CSAM. 

As the parent of two young children in the COVID age, I rely on telehealth services and friends who are medical professionals to help with anxiety-provoking (yet often trivial) medical situations. I often send photos of weird rashes or bug bites to determine if it is something to worry about.  

In the article, a parent took a photo of their child to send to a medical professional. This photo was uploaded to Google Photos, where it was flagged as being potentially abusive material by a machine learning algorithm. 

Google ended up suspending and permanently deleting his Gmail account and his Google Fi phone and flagging his account to law enforcement. 

Just imagine how you might deal with losing both your primary email account, your phone number, and your authenticator app. 

Finding and reporting abuse is critical. But, as the article illustrates, ML-based approaches often lack context. A photo shared with a medical professional may share similar features to those showing abuse. 

Before we start devolving more and more of our day-to-day lives and decisions to machine learning-based algorithms, we may want to consider the consequences of removing humans from the loop.

Crypto and Transaction Costs

by

You live in the up-and-coming suburb of Cryptoville and you want to buy a house. It costs $1m. 

There might be some transaction fees involved, but you won’t actually know how much the fees will be until you complete the transaction. Oh, you are not competing with anyone to buy the house, it’s just a transaction fee. Can’t be too bad right? 

On the day of closing, the transaction goes through. The transaction fees are $250,000! And there was no way to tell until you tried to buy the house. It’s just the way things work in Cryptoville.. 

This is pretty much what happened on Saturday when Yuga Labs, the company behind the Bored Ape Yacht Club, held a much anticipated virtual land / NFT sale on the Ethereum network. Gas fees (i.e. transaction fees on Ethereum) spiked as the network coped with thousands of ApeCoin holders looking to buy some virtual land for their virtual Apes. 

The shocking thing was that it caused the entire Ethereum network to clog up – raising transaction costs for everyone – not just those looking to buy virtual land. Folks looking to buy NFTs valued at under a dollar were seeing transaction fees of $3,500! 

This points to a serious, and well-known, issue with throughput on Ethereum. It does not scale under load. Perhaps the long-delayed migration to Proof of Stake may change this – when it happens.

But – do you know what happened to the “high-performance” blockchain Solana on Saturday? You see where this going..

Links:
Ethereum Gas Prices Spike
Solana Performance Issues
Introduction to Ethereum Scaling

Footnote
Ethereum can only process about 15 transactions per second. It is just the way it is designed. However, miners can be incentivized to process transactions by increasing gas (transaction) fees. This is what happened on Saturday – as the demand to mint NFTs skyrocketed, so did the transaction fees. Gas fees have since come down, but it shows the big issues that Ethereum continues to face as it remains the de-facto standard for blockchain development.

Elon Musk & The Twitter Algorithm

by

I have been trying to avoid the whole Elon Musk / Twitter drama, but it has been challenging. I am ambivalent about whether Mr. Musk’s takeover of Twitter is a good or bad thing. My vibe is 🤷🏾‍♂️.

But, I do have an issue with one of Mr. Musk’s ideas: open-sourcing the Twitter algorithm to ensure there is no “bias.”

I think this is disingenuous, and Mr. Musk is playing to his (adoring) audience a little bit. 

It is improbable that there is the “one true algorithm” at Twitter. They probably use a combination of machine learning-based recommendation models with other systems such as entity and intent detection. Take a look at Twitter’s engineering blog to see how much ML drives recommendations on the social network.

So, if the intention is to look at the code and delete any (left-wing | right-wing) bias, things will be.. difficult. 

Now, a discussion should be had about how the ML models are trained and if there are any biases in the labeled datasets that are used to drive recommendations, detect abusive content, etc. This is a complex problem, however! 

An important effect of the pervasive deployment of ML technologies is that it makes computing *probabilistic* instead of *deterministic*. i.e., we know what is likely to happen, but it is difficult to predict what *will* happen.

This paradigm shift makes it very difficult to point the finger at one or more woke/radical/reactionary programmer who decides to censor or advocate for free speech. 

Mr. Musk knows all this, of course. The entire Tesla “full self-driving” stack is built on ML. So, perhaps, a little bit of intellectual honesty might lead to a more interesting discourse about bias.

Links:
Why Elon Musk Wants to Open Source Twitter

Elon Musk’s Poll on whether the Twitter “algorithm” should be open-sourced: https://twitter.com/elonmusk/status/1507041396242407424

Twitter Engineering Blog: https://blog.twitter.com/engineering/en_us

MIT Technology Review has a good writeup about this: https://www.technologyreview.com/2022/04/27/1051472/the-problems-with-elon-musks-plan-to-open-source-the-twitter-algorithm/

Between Rock and a.. podcast?

by

Just because you can do it doesn’t make it a great business model. Take music streaming, for example.

Image by Chloe Ridgeway on Unsplash

Spotify, the world’s most popular streaming service, has been the target of some Internet ire in the last week or so. Neil Young, the creator of the legendary Pono digital media player (apparently he made some music too?), decided he didn’t want anything to do with Spotify. 

Why all the righteous indignation?

Spotify pays Joe Rogan, a media personality / MMA commentator / master of “doing his own research,” over $100m to have exclusive rights to his wildly popular podcast. 

Apparently, Mr. Rogan has some interesting ideas around COVID, vaccinations, and horse de-worming medication. Not particularly controversial topics 😬. 

Why is this a big deal for Spotify?

Music streaming is a terrible business. Spotify has been bleeding cash for years and only recently turned a meager profit. The company had an operating margin of 1.4% in the first nine months of last year. No hockey sticks in sight.

The reason? It has to pay royalties to music labels for each music stream. The value from streaming accrues to the music companies, not to the streamers or artists.

Spotify makes its money not from streaming but from selling subscriptions and advertising. 

This is where podcasts come in. Spotify pays millions to Joe Rogan because he brings in a massive audience in the highly desirable 18-34 demographic. Spotify offers targeted advertising on podcasts to its most important customers, advertisers. This makes much more economic sense than making tiny margins on each stream of, let’s say, “Rockin’ in the Free World.” 

The risk to Spotify in this, slightly ridiculous, situation is not losing access to rock & roll; its not being able to monetize their investments in podcasting. 

Spotify would rather you come for the music and stay for Elon Musk smoking some fine herb  with his buddy Joe Rogan. 

They have set up expectations for their users that they can stream any song at any time. So they have to double down on more economically viable content like the Joe Rogan Experience. 

I am sure there is a Neil Young song about rocks and hard places..

On crypto outages

by

What to do when your decentralized, scalable, performant blockchain turns out to be not so scalable, sort-of-centralized and not so performant? 

Crypto’s selling point is robustness that is built on decentralization. No single points of failures should mean no downtime right? Right?

Turns out, crypto’s weaknesses are same as those of other, more mundane technologies. Bad code, bad actors and the fact that building scalable, distributed (and decentralized) systems is hard!

Solana, a Layer 1 blockchain, suffered a long outage over the weekend. This happened when the crypto markets are melting down.. 

Solana is supposed to be the answer to Ethereum’s performance and scalability issues. And yet, Solana has been plagued by performance issues and outages over the last few months. 

This weekend’s issue was caused by “program cache exhaustion” due to “excessive duplicate transactions”. Solana developers released an emergency patch to resolve this issue and begged every validator to upgrade.

Where there is code, there are bugs.. 

Welcome to the brave new world, where the problems are the same as the ones in the old world. They just cost you a lot of funny money.

On Roblox’s Outage

by

Roblox is one of the world’s biggest game platforms. With over fifty million daily users, it is a wildly popular platform to build and play games. 

In October last year, they had an outage where the entire platform was down for over 72 hours. This was all over the news at the time..

Today, Roblox published a post mortem about the incident. It is fascinating reading for anyone interested in distributed systems, DevOps, and Engineering (link below). I will write up a more detailed note in a couple of days.

Summary
– The outage was due to an issue in their service discovery infrastructure which is implemented in Consul
– Roblox is deployed on-premise(!!) on 18,000 servers which run 170,000 service instances
– These services rely on Consul (from HashiCorp) for service discovery and configuration
– An upgrade to Consul and the resulting switch to the way services interact with Consul lead to a cascading set of failures resulting in the outage

Some Initial Thoughts
– Distributed systems are hard, and the use of service-oriented architectures come with costs of coordination and service discovery
– Microservice architectures do not reduce complexity, just move it up a layer of abstraction
– The complexity of the modern software stack comes not just from your code, but also from your dependencies. 
– Leader election is one of the hardest problems in Computer Science 🙂