The Vagueness and Vagaries of ‘Personal Superintelligence’

Mark Zuckerberg is not much of a visionary. He is ambitious, sure, and he has big ideas. He occasionally pops into the public conscience to share some new direction in which he is taking his company — a new area of focus that promises to assert his company’s leadership in technology and society. But very little of it seems to bear fruit or be based on a coherent set of principles.

For example, due to Meta’s scale, it is running into limitations on its total addressable market based on global internet connectivity. It has therefore participated in several related projects, like measuring the availability of internet connectivity worldwide with the Economist, which has not been updated since 2022. In 2014, it acquired a company building a solar-powered drone to beam service to people in more remote locations; the project was cancelled in 2018. It made a robot to wrap fibre optic cable around existing power lines, which it licensed to Hibot in 2023; Hibot has nothing on its website about the robot.

It is not just Meta’s globe-spanning ambitions that have faltered. In 2019, Zuckerberg outlined a “privacy-focused vision for social networking” for what was then Facebook, the core tenets of which in no way conflict with the company’s targeted advertising business. Aside from the things I hope Facebook was already doing — data should be stored securely, private interactions should remain private, and so on — there were some lofty goals. Zuckerberg said the company should roll out end-to-end encrypted messaging across its product line; that it should add controls to automatically delete or hide posts after some amount of time; that its products should be extremely interoperable with those from third-parties. As of writing, Meta added end-to-end encryption to Facebook Messenger and Instagram, but it is only on by default for Facebook. (WhatsApp was end-to-end encrypted by default already.) It has not added an automatic post deletion feature to Facebook or Instagram. Its apps remain stubbornly walled-off. You cannot even sign into a third-party Mastodon app with a Threads account, even though it is amongst the newest and most interoperable offerings from Meta.

Zuckerberg published that when it was advantageous for the company to be seen as doing its part for user privacy. Similarly, when it was smart to advocate for platform safety, Zuckerberg was contrite:

But it’s clear now that we didn’t do enough. We didn’t focus enough on preventing abuse and thinking through how people could use these tools to do harm as well. That goes for fake news, foreign interference in elections, hate speech, in addition to developers and data privacy. We didn’t take a broad enough view of what our responsibility is, and that was a huge mistake. It was my mistake.

Then, when it became a good move to be brash and arrogant, Zuckerberg put on a gold chain and a million-dollar watch to explain how platform moderation had gone too far.

To be clear, Meta has not entirely failed with these initiatives. As mentioned, Threads is relatively interoperable, and the company defaulted to end-to-end encryption in Facebook Messenger in 2023. It said earlier this year it is spending $10 billion on a massive sub-sea cable, which is a proven technology to expand connectivity more than a solar-powered drone could.

But I have so far not mentioned the metaverse. According to Zuckerberg, this is “an embodied internet where you’re in the experience, not just looking at it”, and it was worth pivoting the entire company to be “metaverse-first”. The company renamed itself “Meta”. Zuckerberg forecasted an “Altria moment” a few years prior and the press noticed. In announcing this new direction in 2021, Zuckerberg acknowledged it would be a long-term goal, though predicted it would be “mainstream in the next five to ten years”:

Our hope is that within the next decade, the metaverse will reach a billion people, host hundreds of billions of dollars of digital commerce, and support jobs for millions of creators and developers.

Granted, it has not been even four years since Zuckerberg made these announcements, but are we any closer to his company’s vision becoming mainstream? If you broaden the definition of “metaverse” to include all augmented and virtual reality products then, yes, it appears to be a growing industry. But the vision shown at Connect 2021 is scarcely anywhere to be found. We are not attending virtual concerts or buying virtual merch at virtual after-parties. I am aching to know how the metaverse real estate market is doing as I am unaware of anyone I know living in a virtual house.

As part of this effort, Meta announced in May 2022 it would support NFTs on Instagram. These would be important building blocks for the metaverse, the company said, “critical for how people will buy, use and share virtual objects and experiences” in the virtual environment it was building. Meta quickly expanded availability to Facebook and rolled it out worldwide. Then, in March 2023, it ended support for NFTs altogether, saying “[a]ny collectibles you’ve already shared will remain as posts, but no blockchain info will be displayed”.

Zuckerberg has repeatedly changed direction on what his company is supposed to stand for. He has plenty of ideas, sure, and they are often the kinds of things requiring resources in an amount only possible for a giant corporation like the one he runs. And he has done it again by dedicating Meta’s efforts to what he is calling — in a new manifesto, open letter, mission statement, or whatever this is — “personal superintelligence”.

I do have to take a moment to acknowledge the bizarre quality of this page. It is ostensibly a minimalist and unstyled document of near-black Times New Roman on a white background — very hacker, very serious. It contains about 3,800 characters, which should mean a document barely above four or five kilobytes, accounting for HTML tags and a touch of CSS. Yet it is over 400 kilobytes. Also, I love that keywords are defined:

<meta name="keywords" content="Personal 
Superintelligence, AI systems improvement, 
Superintelligence vision, Mark Zuckerberg 
Meta, Human empowerment AI, Future of 
technology, AI safety and risks, Personal
AI devices, Creativity and culture with 
AI, Meta AI initiatives">

Very retro.

Anyway, what is “superintelligence”? is a reasonable question you may ask, and a term which Zuckerberg does not define. I guess it is supposed to be something more than or different from artificial intelligence, which is yesterday’s news:

As profound as the abundance produced by AI may one day be, an even more meaningful impact on our lives will likely come from everyone having a personal superintelligence that helps you achieve your goals, create what you want to see in the world, experience any adventure, be a better friend to those you care about, and grow to become the person you aspire to be.

He decries competitors’ ambitions:

This is distinct from others in the industry who believe superintelligence should be directed centrally towards automating all valuable work, and then humanity will live on a dole of its output. At Meta, we believe that people pursuing their individual aspirations is how we have always made progress expanding prosperity, science, health, and culture. This will be increasingly important in the future as well.

I am unsure what to make of this. It is sorely tempting to dismiss the whole endeavour as little more than words on a page for a company deriving 98% of its revenue (PDF) from advertising.¹ If we consider it more seriously, however, we are left with an ugly impression for what “valuable work” may consist of. Meta is very proud of its technology to “generate photorealistic images”, thereby taking the work of artists and photographers. Examples of its technology also include generating blog posts and building study plans, so it seems writing and tutoring are not entirely “valuable work” either.

I am being a bit cheeky but, with Zuckerberg’s statement entirely devoid of specifics, I am also giving it the gravitas it has earned.

While I was taking way too long to write this, Om Malik examined it from the perspective of someone who has followed Zuckerberg’s career trajectory since it began. It is a really good piece. Though Malik starts by saying “Zuck is one of the best ‘chief executives’ to come out of Silicon Valley”, he concludes by acknowledging he is “skeptical of his ability to invent a new future for his company”:

Zuck has competitive anxiety. By repeatedly talking about being “distinct from others in the industry” he is tipping his hand. He is worried that Meta is being seen as a follower rather than leader. Young people are flocking to ChatGPT. Programmers are flocking to Claude Code.

What does Meta AI do? Bupkiss. And Zuck knows that very well. You don’t do a company makeover if things are working well.

If you are solely looking at Meta’s earnings, things seem to be working just fine for the company. Meta beat revenue expectations in its most recent quarter while saying the current quarter will also be better than analysts thought. Meta might not be meeting already-low analyst expectations for revenue in its Reality Labs metaverse segment, but the stock jumped by 10% anyhow. Even Wall Street is not taking Zuckerberg seriously as an innovator. Meta is great at selling ads. It is not very exciting, but it works.

Back to the superintelligence memo, emphasis mine:

We believe the benefits of superintelligence should be shared with the world as broadly as possible. That said, superintelligence will raise novel safety concerns. We’ll need to be rigorous about mitigating these risks and careful about what we choose to open source. Still, we believe that building a free society requires that we aim to empower people as much as possible.

And here is what Zuckerberg wrote just one year ago:

Meta is committed to open source AI. I’ll outline why I believe open source is the best development stack for you, why open sourcing Llama is good for Meta, and why open source AI is good for the world and therefore a platform that will be around for the long term.

[…]

There is an ongoing debate about the safety of open source AI models, and my view is that open source AI will be safer than the alternatives. I think governments will conclude it’s in their interest to support open source because it will make the world more prosperous and safer.

No mention of being careful, no mention of choosing what to open source. Zuckerberg took an ostensibly strong, principled view supportive of open source A.I. when it benefitted the company, and is now taking an ostensibly strong, principled view that it requires more nuance.

Zuckerberg concludes:

Meta believes strongly in building personal superintelligence that empowers everyone. We have the resources and the expertise to build the massive infrastructure required, and the capability and will to deliver new technology to billions of people across our products. I’m excited to focus Meta’s efforts towards building this future.

On this, I kind of believe him. I believe the company has the resources and reach to make “personal superintelligence” — whatever it is — a central part of Meta’s raison d’être, just as Malik says in his article he has “learned not to underestimate Zuckerberg”. The language in Zuckerberg’s post is flexible, vague, and optimistic enough to provide cover for whatever the company does next. It could be a unique virtual assistant, or it could be animated stickers in chats. Whatever it is, this technology will also assuredly be directed toward the company’s advertising machine, as its current A.I. efforts are providing “greater efficiency and gains across our ad system”. Zuckerberg is telling investors imagine what we could do with superintelligence.

In December 2023, Simon Willison wrote about the trust crisis in artificial intelligence, comparing it to the conspiracy theory that advertisers use audio from real-world conversations for targeting:

The key issue here is the same as the OpenAI training issue: people don’t believe these companies when they say that they aren’t doing something.

One interesting difference here is that in the Facebook example people have personal evidence that makes them believe they understand what’s going on.

With AI we have almost the complete opposite: AI models are weird black boxes, built in secret and with no way of understanding what the training data was or how it influences the model.

Meta has pulled off a remarkable feat. It has ground down users’ view of their own privacy into irrelevance, yet its services remain ubiquitous to the point of being essential. Maybe Meta does not need trust for its A.I. or “superintelligence” ambitions, either. It is unfathomably rich, has a huge volume of proprietary user data, and a CEO who keeps pushing forward despite failing at basically every quasi-visionary project. Maybe that is enough.