That Secret TikTok Document Obtained by the New York Times Has Been Circulating Publicly for a Year Under a Different Name ⇥ protocol.com

In his Sunday “Media Equation” column in the New York Times, Ben Smith said he obtained an internal document created for new TikTok employees:

The document, headed “TikTok Algo 101,” was produced by TikTok’s engineering team in Beijing. A company spokeswoman, Hilary McQuaide, confirmed its authenticity, and said it was written to explain to nontechnical employees how the algorithm works. The document offers a new level of detail about the dominant video app, providing a revealing glimpse both of the app’s mathematical core and insight into the company’s understanding of human nature — our tendencies toward boredom, our sensitivity to cultural cues — that help explain why it’s so hard to put down. The document also lifts the curtain on the company’s seamless connection to its Chinese parent company, ByteDance, at a time when the U.S. Department of Commerce is preparing a report on whether TikTok poses a security risk to the United States.

What is interesting to me is the lengths the Times went to so that it could obscure this relatively mild piece of internal documentation. Unlike many other artifacts obtained by the Times, a copy was not linked within the article, and even embedded diagrams were reproduced instead of the originals being shown.

Whether those were precautions borne of a secrecy promise, or perhaps because the original documents had legibility problems, I feel like Smith buried the lede. After wading through an overwrought exploration of the file’s contents, Smith reports on the many lingering connections the ostensibly independent TikTok has with its predecessor app Douyin:

TikTok’s development process, the document says, is closely intertwined with the process of Douyin’s. The document at one point refers TikTok employees to the “Launch Process for Douyin Recommendation Strategy,” and links to an internal company document that it says is the “same document for TikTok and Douyin.”

It turns out the Douyin version of that shared internal document has been circulating publicly for months.

Protocol’s Zeyi Yang, writing in the Source Code newsletter:

In fact, another closely related app uses the same secret sauce. In January, a document titled “Guide to a Certain Video APP’s Recommendation (Algorithm)” was released on several Chinese platforms. While it intentionally obscured which app it’s referencing, there are plenty of hints that it’s about Douyin, TikTok’s Chinese version.

For one, the Chinese document describes how it makes recommendations in the exact same formula and language (yes, word for word) as the document leaked to the Times. They also used the same challenge to the algorithm as a case study.

And in a Q&A entry about competitors, the document mentioned three other major Chinese apps — Toutiao, Kuaishou and Weibo — that rely on recommendation algorithms, but not Douyin, the app that does it the best.

The link above is now dead, but you can find plenty of copies on Chinese social networks — one that was uploaded to CSDN, for instance. It is in Chinese, but it appears to be exactly the same file.