#304: DeepMind’s New RETRO AI Models Are Nimble And Could Have Unique Healthcare Applications, & More

Large language models (LLMs) like GPT-3 have supercharged the branch of artificial intelligence (AI) called natural language processing (NLP). Because NLP algorithms operate on text data, they have myriad applications but are complex and expensive, often limiting their utility. GPT-3, for example, contains 175 billion parameters, costs ~$12 million to train, and requires roughly 20 GPUs for inference alone.

Recently, Google (GOOGL) subsidiary DeepMind published a novel type of LLM architecture called RETRO(Retrieval-Enhanced Transformer). RETRO models are unique because they reference a separate database for information instead of relying on learned knowledge contained in their model parameters. Therefore, RETRO models can shrink to a fraction of the size of other LLMs. Though RETRO is roughly 5% the size of GPT-3 with 7.5 billion parameters, RETRO and GPT-3 perform on par with one another.

As some have suggested, RETRO-style models will be useful in healthcare applications. Unlike many machine learning (ML) tasks with immutable truth labels, healthcare and biology knowledge changes frequently. Traditional LLMs would require constant retraining to remain relevant in biology. Because RETRO references a separate database, the models can change without recalibration and could be useful in analyzing patient medical records and informing genetic counselors.

Despite a mere 6% share of global vehicle sales in 2021, electric vehicles (EVs) punched above their weight in their demand for semiconductors, copper, and lithium. Compared to vehicles powered by internal combustion engines, EVs require two to three times as many semiconductors and roughly four times as much copper. They also account for roughly 74% of global lithium consumption.

Because the proliferation of EVs is taxing the demand for raw materials, auto companies are evolving creative supply chain strategies, with varying degrees of success. On their fourth quarter earnings calls, for example, while both Tesla and Ford described efforts to reduce the number of parts in their EVs, ARK noted a divergence in their supply chain expectations, as shown below. Producing 936,000 EVs in 2021 with plans to grow more than 50% this year, Tesla does not expect battery constraints. In contrast, producing only 6% the number of Tesla EVs last year with plans to produce 600,000 units by 2023, Ford sees battery supply as its main constraint.

For informational purposes only and should not be considered investment advice, or a recommendation to buy, sell or hold any particular security. Source: ARK Investment Management LLC, data sourced from: www.capitaliq.com.

Shares of Meta Platform (META) plunged during its earnings call on Thursday as the company issued disappointing guidance, noting increased costs, Apple’s new Advertising Identifier Declaration (IDFA) privacy standards, the weak retail backdrop and, perhaps most important, competition from TikTok. According to Mark Zuckerberg, “People have a lot of choices for how they want to spend their time.”

As ARK has noted, while more a digital entertainment app like YouTube or Netflix than a social media app like Instagram or Facebook, TikTok is taking significant share of consumer mindshare and is posing a threat to Facebook. As the saying goes, time is money, and TikTok is the only non-Meta-owned app to reach 3 billion downloads globally.

Meta hopes to solve its TikTok problem with Reels, a short-form video platform on Instagram. Given its focus on The Metaverse, we are wondering if Meta has the DNA and time to compete on the short-form video front. We believe TikTok’s differentiation includes a superior suite of high-end video creation and editing tools, to which Reels has no match.

Last week, in the second largest DeFi hack to date, hackers stole $320 million of ether from Wormhole Protocol, a bridge that “connects” the Ethereum blockchain to Solana. By default, public blockchains like Ethereum, Solana, and Avalanche were not designed to communicate with one another such that tokens and smart contracts “native” to one chain or network cannot interact with those native to other chains. DeFi participants have embraced “bridging technologies” that have enabled interoperability across networks.

When implemented correctly, protocol bridges allow tokens from one chain––ether on Ethereum, for example––to be “wrapped” for use on another chain––like Solana––enabling ether to be used as collateral for DeFi lending and trading on other chains. Building bridges is difficult and complex, requiring teams with expertise in multiple smart contracting languages and security models.

Last week’s ether thief exploited a bug in one of Wormhole Protocol’s smart contracts––on the Solana side. The hacker “spoofed” a feature of the protocol, cryptographic signatures. Those signatures validate that a user has deposited value on one side of the bridge––say, the Ethereum side––before minting it as “wrapped ether” on the other side––the Solana side. Using the spoofed signatures, the attacker minted 120,000 wrapped ether out of thin air and then cashed them for 120,000 native ether, on the Ethereum side, draining most of the Wormhole Protocol’s balance.

Wormhole isn’t the first cross-chain protocol to suffer an attack. A week earlier on January 28, hackers stole $80 million on the Ethereum-to-Binance Smart Chain bridge, Qubit Finance, and last year a hacker stole $600 million from Poly Network.

To maintain the financial integrity of wrapped ether on Solana after the Wormhole Bridge hack, crypto firm and Wormhole sponsor, Jump Crypto, refunded the $320 million of ether within a day.

In our view, precedents creating bailout expectations are dangerous, particularly now that DeFi is scaling so rapidly. In response to such hacks, we are hoping that developers and users focus on risk management measures like guarded launches to hedge against bugs and attacks. That said, the prevalence of cross-chain attacks also could spur interest in blockchain networks designed for interoperability like Polkadot and Cosmos, and, in our view, highlights the resilience of more narrow-use blockchains like Bitcoin.