BEIJING — Jason Todd first discovered his school’s secret on the internet.
It was late September 2018, less than a month after high school had started. Jason was idly scrolling through his news feed on the Chinese microblogging site Weibo when he saw a trending hashtag — #ThankGodIGraduatedAlready — and clicked it.
Under the hashtag, someone had posted a photo depicting a bird’s-eye view of a classroom. Around 30 students sat at their desks, facing the blackboard. Their backpacks lay discarded at their feet. It looked like a typical Chinese classroom.
Except for the colored rectangles superimposed on each student’s face. “ID: 000010, State 1: Focused,” read a line of text in a green rectangle around the face of a student looking directly at the blackboard. “ID: 000015, State 5: Distracted,” read the text in a red rectangle — this student had buried his head in his desk drawer. A blue rectangle hovered around a girl standing behind her desk. The text read: “ID: 00001, State 3: Answering Questions.”
Jason thought the photo was a scene from a sci-fi movie — until he noticed the blue school badges embroidered on the chest of the familiar white polos worn by the students. It was exactly the same as the one he was wearing.
“F**k, no,” he thought.
Jason is a 16-year-old student at Niulanshan First Secondary School in Beijing. He wears a pair of black-framed glasses and likes to read DC comics so much that he chose to use the name of one of his favorite characters for the sake of anonymity. If he hadn’t seen that image online, he wouldn’t have questioned the presence of the tiny white surveillance camera installed above his classroom’s blackboard. After all, Niulanshan never informed him — or any of its 3,300 other students — that facial recognition cameras were capturing their every move in class. In fact, it’s unlikely that the combined 28,000 students in the six other schools testing the same system know they are part of China’s grand artificial intelligence (AI) experiment.
杰森是名正在牛栏山一中就读的 16 岁学生。他戴着一副黑框眼镜，特别喜欢看 DC 漫画。他（在接受采访时）甚至用了里面最喜欢的角色的名字隐藏真实身份。
要不是他在网上看见了那张图片，他不会去质疑那台安装在黑板前的白色摄像机。毕竟牛栏山一中从未通知过他，或者整个学校当中 3300 多名学生，面部捕获摄像机正记录着他们课上的一举一动。事实上，测试这一系统的另外六所学校中的共计 28000 名学生可能都被蒙在鼓里——他们是中国庞大人工智能实验的一部分。
In July 2017, China’s highest governmental body, the State Council, released an ambitious policy initiative called the Next Generation Artificial Intelligence Development Plan (NGAIDP).
在 2017 年 7 月，中国国务院发布了一项重大政策，叫做“新一代人工智能发展规划”。
The 20,000-word blueprint outlines China’s strategy to become the leading AI power in both research and deployment by 2030 by building a domestic AI industry worth nearly $150 billion. It advocates incorporating AI in virtually all aspects of life, including medicine, law, transportation, environmental protection, and what it calls “intelligent education.”
这篇两万多字的蓝图表明，中国要在 2030 年前建成全国性的人工智能产业，行业总值达 1500 亿美元，成为全球领先的人工智能国家。它号召将 AI 普及到几乎生活的方方面面，包括机械、法律、交通、环保，以及所谓的“智能教育”当中。
Following the NGAIDP’s release, Chinese tech companies have rushed to secure government support and investor funding for various AI projects, several of which are being tested in Chinese schools. Over the past two months, I interviewed a dozen students in schools that have installed different “intelligent education” systems and spoke to tech entrepreneurs whose companies are developing the systems used in those schools.
该政策发布后，中国的科技企业倾巢而出，寻求政府支持和投资者投资，开发大量的 AI 项目。其中有一部分正在中国的学校中进行测试。在过去的两个月里，我采访了十几个在校学生，他们的学校里都安装了各种“智能教育”系统，并且还和开发这些系统的企业领导聊了聊。
While advocates claim that using facial recognition to monitor students’ in-class behavior can accurately assess attention levels and help them learn more efficiently, most students I spoke with had a different opinion. They showed concern that the school never asked for their consent before harvesting their facial data. One student expressed anxiety at the idea that the times he was caught slacking off in class would eventually be used to determine his chances of attending his dream university. Other students disconnected the cameras at their school before final exams in protest.
In addition, teachers and experts question the extent to which facial recognition systems can improve student performance. Experts say there are many technological, legal, and moral barriers to overcome before facial recognition can be widely deployed in Chinese education. But the government’s AI push is already introducing this technology before settling such debates, leading experts to agree that regulations on the technology urgently need to catch up.
"Class Care System"
In the upper left corner of that photo Jason found on Weibo, you can just make out the words “CCS Class Care System.” CCS is one of the flagship products of Hanwang Education, a subsidiary of Hanwang Technology. Hanwang is a household name for China’s younger generation. Growing up, they watched catchy commercials for the company’s text-to-speech reading pen on television. Today, Hanwang builds hardware and software products that provide facial and biometric recognition services and optical character recognition, as well as air quality monitors and purifiers. Hanwang Education was founded in 2014 as part of an initiative to expand the company’s market into China’s education sector.
杰森找到的那张图片，左上角能依稀分辨出“课堂呵护系统”的字样。“课堂呵护系统”是汉王教育的旗舰产品之一，后者是汉王科技的子公司。汉王在中国年轻一代当中可谓家喻户晓。这家公司的撩人广告伴随着这些年轻人一路长大——诸如电视上的点读笔广告等。而现在，汉王造的软硬件产品能提供面部识别、生物识别和光学字符识别服务，甚至还有空气质量监测器和净化器。汉王教育成立于 2014 年，它是汉王在中国教育领域扩张计划的一部分。
At the research center, which occupies a nondescript, three-story house in a residential estate in northwestern Beijing, Hanwang Education General Manager Zhang Haopeng rushes downstairs to greet me. He’s a busy man, currently helming the deployment of CCS in partner schools across the country. Fortunately, he’s managed to find time for an interview.
If transitioning from text-to-speech pens to facial recognition technology seems like something of a quantum leap, it’s really not, Zhang explains. Recognizing a student’s facial expression and reading out handwritten notes are variants of the same thing: pattern recognition. To the machines, faces and handwriting are just data, and identifying them is all about spotting patterns. Since 2015, Hanwang Technology has also been using deep learning, a type of AI that mimics the brain’s analytical models, to buttress its existing technology. “Hanwang is at the forefront of pattern recognition,” Zhang says proudly.
Zhang Haopeng, Hanwang Education’s general manager, checks the company’s “Class Care System”servers in Beijing, Jan. 2, 2019. Xue Yujie/Sixth Tone
张浩鹏在北京。他正检查他公司的“课堂呵护系统”。2019 年 1 月 2 日，薛玉洁/第六声音 摄
At first, Hanwang struggled to monetize its cutting-edge facial recognition products, which it only used for commercial purposes, such as face scanners that log employee work hours or smart billboards that tailor their advertisements to audience gender. Zhang tried unsuccessfully to push Hanwang into the education sector, marketing interactive whiteboards and an “e-schoolbag” tablet that acted as a multipurpose textbook. But once China announced the NGAIDP in 2017, Hanwang finally found its niche: education analysis.
一开始，汉王努力地商业化它家的面部识别产品。这些产品只用于商业目的，例如面部识别打卡机。张浩鹏当时试图用“交互式白板”和“电子书包”（就是多用途的教科书）进入教育领域，然而并不成功。但中国在 2017 年发布了“新一代人工智能发展规划”之后，汉王终于找到了自身的定位——教育分析。
The plan’s “intelligent education” section describes in detail how China’s government hopes to use AI to boost the country’s education system. Zhang reads me one paragraph from the guidelines without stammering. “So detailed. It’s like they wrote it with a [facial recognition] product right in front of them,” he says.
Zhang, who keeps a framed photo of his smiling children on his desk, says that parents in China are yearning for more information about their children’s school performance. It’s a country where test scores can make or break an individual’s future. Visit any Chinese primary school at the end of the day, he says, and you’ll see parents bombarding teachers with questions. “Did my son fall asleep during English class again?” he says, mimicking the questions parents might ask. “Did my daughter and her deskmate talk too much during class? Should we separate them?”
Zhang says that for most Chinese parents, school is the only time they let their kids out of their sight. “Parents worry when their children aren’t around. They want to take care of every aspect of their lives,” he tells me. “But the teacher only has one pair of eyes.”
CCS technology was born from the desire to care for every kid in the classroom.
- Zhang Haopeng, Hanwang Education General Manager
“Do you know the two types of students teachers pay the most attention to?” Zhang asks. “The smartest and the naughtiest.” Hanwang’s CCS technology was born from the desire to care for every kid in the classroom, even the “often-ignored, average students,” he adds.
Zhang shows me one of Hanwang’s CCS cameras. It’s the size of a mandarin orange, but it can recognize all 50 students in a classroom. “Just five years ago, this kind of technology was unimaginable,” he says, explaining how the camera homes in on students’ unique facial features to identify each individual — and in a much less obtrusive way than a fingerprint or iris scanner. “We can identify a person’s face from just one picture, even when the frame size is as low as 640 x 480 pixels,” Zhang says.
张浩鹏给我展示了汉王“课堂呵护系统”的一台摄像机。它只有橘子大小，但是它能识别教室里的所有 50 名学生。“仅仅五年之前，这种技术还是不可想象的。”他说。他还解释摄像机是如何通过捕获学生独特的面部表情识别每个人的，这比指纹或者虹膜识别器造成的打扰低很多。“我们只需要一张图片就能能识别一个人的面部，即便画幅只有 640 x 480 分辨率。”张浩鹏说。
Even though CCS hasn’t received official approval from China’s Ministry of Education yet, it’s already been implemented as a pilot project in seven schools around the country since its launch in December 2017. In these schools, a white, dome-shaped camera is installed above the blackboard at the front of each classroom. Once per second, it takes a photo of the entire class and sends the footage to a server where Hanwang’s deep-learning algorithms identify each student’s face and classify their behavior into five categories: listening, answering questions, writing, interacting with other students, or sleeping. The algorithms then analyze each student’s behavioral data and give them a weekly score, which is accessible through a mobile app.
虽然“课堂呵护系统”还未受到中国教育部的官方批准，但从 2017 年 12 月，它已经在全国各地共计七所学校中先行得到采用。在这几所学校中，每间教室的黑板上方都安了一个白色圆顶状的摄像机，它会每隔一秒给全班拍一张图片，并发送到汉王的服务器中。服务器的深度学习算法会识别学生的面部特征，将他们的行为分成五类——听讲、回答问题、书写、和其他学生交互，以及睡觉。接下来，这一算法会分析每个学生的行为数据，每周都给他们打一次分，通过移动应用就可以查看到。
Zhang takes out his phone and logs into a user account on CCS’s mobile app. The account belongs to a teacher at Chifeng No. 4 Middle School in the city of Chifeng in northern China’s Inner Mongolia Autonomous Region. The interface allows teachers to view scores for every student in class. A green down arrow appears next to the student’s score when it decreases, and a red up arrow when it increases. A bar graph shows how many minutes the student spent concentrating, sleeping, or talking in class.
“The parents can see it, too,” Zhang says, tapping on a student’s name. “For example, this student’s report shows that he rarely volunteers to answer the teacher’s questions in class. So his participation in English class is marked as low. Number of questions answered: one,” Zhang reads from the AI-generated report. “This week, the student spent 94.08 percent of class time focusing. His grade average is 84.64 percent. He spent 4.65 percent of the time writing, which was 10.57 percent lower than the grade average.”
“这些数据他们家长也能看到。”张浩鹏点了点学生的名字说，“比如，这个学生的报告表明，他很少自己起来回答老师的问题，那么他在英语课上的参与度就被评级为‘低’。总共才回答了一个问题。”张浩鹏读着人工智能生成的报告，“这一周，该学生专注于课堂听讲的时间有 98.08%，年级平均专注率是 84.64%，他书写的时间占 4.65%，比全年级平均水平低 10.57 个百分点。”
“What’s your takeaway from the report?” Zhang asks, before once again answering his own question. “This student doesn’t like to answer questions in English class, so maybe his parents and teacher should do something.”
On the app, teachers and parents are also able to see up-to-date photos of the classroom and check students’ behavior, just like in that photo with the colorful rectangles. Zhang plans to expand CCS to 100 schools around the country by the end of 2019 and, eventually, he wants to create a nationwide platform for all schools.
张浩鹏计划，在 2019 年底之前，将“课堂呵护系统”推广到全国 100 个学校，最终建立一个覆盖全国所有学校的平台。
“Now do you understand the ‘care’ part of our product name? Not a single student is missed,” Zhang says, smiling.
He leads me to a tiny room where two dusty CCS servers sit under a desk. One, which costs 60,000 yuan ($8,900), can analyze facial recognition data from five to six classrooms simultaneously. The other can monitor 20 classrooms and costs 150,000 yuan. Wiring a single classroom costs around 30,000 yuan. Zhang won’t tell me how much it costs per school, but if the high school I went to installed CCS in its 36 classrooms, it would cost at least 1.38 million yuan.
他把我带到了一个小房间中，里面有两个落满灰的“课堂呵护系统”服务器，放在一个桌子底下。其中一个的成本是 60000 元（合 8900 美元），它能同时分析五到六个教室的面部识别数据。另外一个服务器可以同时监控 20 个教室，成本 150000 元。部署一个教室的花费大概是 30000 元。张浩鹏并没告诉我每个学校要花费多少钱，但是如果我采访的那个学校给全部 36 个教室都安了这个系统，那会花费至少 138 万元。
But so far, none of the schools using CCS have paid the staggering price. The government has offered financial incentives to local education bureaus to encourage them to use big data and AI — incentives that cover the installation costs. “[Local education bureaus] couldn’t be happier to implement the [AI development] policy,” Zhang tells me.
But if the social media reaction to CCS is anything to go by, the technology isn’t convincing everyone. I ask Zhang about the fearful messages posted under last year's leaked photo of the children at Jason Todd’s school. “People are overreacting,” Zhang says with a slight smile. “CCS doesn’t violate the students’ privacy. We don’t share the reports with third parties, and you see that on the in-class pictures we send to the parents, all the faces other than their child’s are blurred out.”
Before I leave, I ask Zhang one final question: “Do the students know?”
“Of course. You can’t use it without their consent,” he replies.
“What do they think of it?”
“They hate it.”
Zhang told me that at Chifeng No. 4, students unplugged the cameras before the day of their final exams. The cameras captured everything until the last moment.
Two hours away from Hanwang’s research center, I peep through the backdoor window of Jason Todd’s biology class. Most of the 30-odd 15- and 16-year-old students are listening carefully, taking notes from time to time. Some gaze out the window. Three nap with their heads buried behind stacks of books.
For the most part, it’s a run-of-the-mill classroom. The huge blackboard at the back of the room is decorated with drawings of red lanterns, snowmen, and fireworks to celebrate the coming Lunar New Year. Motivational quotes from Isaac Newton and Albert Einstein hang on the walls. However, there are three cameras installed in the classroom, including a small white one right above the blackboard.
Once the students notice me, a murmur of noise starts to build in the classroom. “Pay attention!” the teacher says, knocking on the blackboard. But nothing stops a classroom of curious teenagers, and it remains noisy until the bell rings for recess.
Before the students have a chance to leave, another teacher enters the classroom and announces: “Those of you who didn’t do the face sampling last time, go to the music room right now. Hurry.” The students empty out.
A group of six boys who completed the process stay in the classroom, handing out homework as their classmates leave. None of them, other than Jason, know what the face sampling was for — the teacher didn’t tell them. I ask what the cameras in the classroom are for. “To see if we are behaving well in class?” one tells me, although he says he isn’t sure. The students say the cameras have been there since they started school in September.
Since discovering that photo on Weibo, Jason’s been observing the cameras installed in the classroom, trying to guess their functions. The smaller one at the front is used for facial recognition, he deduces from the picture on Weibo. The bigger one at the back of the classroom is used for livestreaming, judging from the angle of the video he previously spotted on his teacher’s laptop. He’s not sure about the third camera on the window side of the classroom, though he thinks it’s a backup.
Jason’s teacher, Guo Yuzhuo, a short-haired woman with a calm voice and over 20 years of teaching experience, confirms his speculations. The two other cameras, besides the one used for Hanwang’s CCS, were installed by the school itself. The teachers use the camera in the back to check the class in real time, without having to peep through the backdoor window and interrupt the students. Guo says they tend to notice right away that teachers are watching, as I’ve experienced.
杰森的老师郭玉卓（音），是个声音平和的短发女老师，有着超过 20 年的教学经验。她证实了杰森的猜测。前面的摄像机确实是汉王课堂呵护系统用的，而另外两个摄像机则是学校自己装的。老师会用后置的摄像机实时对教室情况进行检查，这样就不必偷偷从后门窗户看，也不会打扰学生了。郭老师说，如果老师在后门看，学生们通常马上就会发现，正如我在后门看时一样。
According to Guo, the teachers use Hanwang’s CCS occasionally to check for any abnormal statistics in the AI-generated weekly reports, but so far, the school hasn’t told the students about the system. She says the school will eventually inform the parents “when it’s time to do so,” explaining that they want to avoid possible obstructions from reluctant students and parents. Guo’s answer contrasts with Hanwang Education manager Zhang’s reassurances that the system wouldn’t be used without students’ consent. Dong Wencheng, a Hanwang Education technician responsible for installing CCS in Beijing, says it’s the school’s job to inform the students, not Hanwang’s. “We suggest the schools ask for the students’ consent before using CCS,” he says, “but it’s just our suggestion. If they don’t, there’s nothing we can do.”
根据郭老师所言，老师们偶尔会使用汉王的“课堂呵护系统”检查 AI 生成的每周统计报告中有无异常的数据。但截至目前，学校还尚未告诉学生这一系统的存在。她说，“当时机成熟时”，学校自会告诉家长。老师解释说，这样做是为了避免来自学生或者学生家长可能的阻碍。郭老师的说法和汉王教育张浩鹏“不经学生允许不得使用”的保证正好相反。汉王教育负责北京系统安装的技术人员董文城（音）称，询问学生的意见是学校该做的事，而不是汉王的。“我们的建议是，学校在使用‘课堂呵护系统’之前先征得学生同意，但是如果学校不这么做的话，我们也没办法。”他说。
We suggest the schools ask for the students’ consent before using CCS, but it’s just our suggestion. If they don’t, there’s nothing we can do.
- Dong Wencheng, Hanwang Education technician
Guo Yuzhuo, a teacher at Niulanshan First Secondary School in Beijing, Jan. 3, 2019. Xue Yujie/Sixth Tone
北京牛栏山一中教师郭玉卓（音）。2019 年 1 月 3 日，薛玉洁/第六声音 摄
Back in the classroom, my questions about the cameras evoke curiosity among the boys. Jason tells them everything he knows. There is a gasp, followed by silence. “I want to smash it,” one boy says. “Shhh!” Another boy shakes a warning glance at Hanwang’s camera behind us. “What if the camera just captured everything?”
The rest of Jason’s classmates are still unaware they are being watched by Hanwang’s CCS camera. But some 1,400 kilometers away at Hangzhou No. 11 Middle School in eastern China’s Zhejiang province, the students know exactly what the cameras in their classrooms are capable of.
杰森的其他同学仍然并不知道他们正被汉王的“课堂呵护系统”监视着。但是在 1400 公里外，中国东部的杭州十一中，学生们确切地知道，教室里的摄像机到底能干什么。
Hangzhou No. 11 uses the “smart classroom behavioral management system” developed by Hangzhou-based Hikvision, the world’s largest manufacturer of video surveillance equipment. Like CCS, Hikvision’s facial recognition technology also monitors students using cameras installed above each classroom’s blackboard. In addition to in-class behaviors, which are divided into six categories — reading, writing, listening, standing up, raising hands, and lying on the desk — Hikvision also identifies seven different facial expressions: neutral, happy, sad, disappointed, angry, scared, and surprised. The data is used to generate a student’s score, which is displayed on a screen installed on the wall of each classroom. Each class’s overall attention level also displays on a huge screen in the hallway for the whole school to compare and rank.
Hangzhou No. 11’s facial recognition system made domestic and foreign headlines last May. Unsurprisingly, the program was met with criticism, but the school’s principal, Ni Ziyuan, said he believed the system would boost educational standards. “It's the same as teachers having an assistant,” he said in an interview. Unlike a teaching assistant, facial recognition cameras don’t interrupt the class, and capture the most natural status of the students, he claimed. The school’s vice president, Zhang Guanchao, also said in later interviews that the system has had positive effects since its implementation.
However, the students feel differently about the system. One anonymous Hangzhou No. 11 student I found on the internet tells me she felt shocked and scared when the teacher demonstrated the system in front of the whole class. “The camera can magnify 25 times of what it captures,” she says, adding: “It can see not only your face, but the characters on your notebook. After all, it’s from Hikvision.” Another student tells me his classmates were totally “crushed” after the installation of the system. Because the system gives students a public score, he and his classmates don’t dare nap or even yawn in class for fear of being penalized, an incentive that doesn’t necessarily increase focus on learning. In fact, the students spend their time focusing on staying awake until class ends. “Nobody leaves the classroom during the class break,” he says. “We all collapse on the desks, sleeping.”
但是，同学们对这个系统却持有不同意见。在网上我找到的一名不愿透露姓名的杭州十一中的学生向我表示，看见老师当着全班的面演示这个系统的时候，她吓坏了，很害怕。“这摄像头可以把捕获的图像放大 25 倍，”她说，“它不仅能看见你的面部特征，还能看见笔记本上你写的字，毕竟是海康威视嘛。”另一名学生告诉我，安了这系统以后，他们全班都炸了，因为这系统会给每个人都打分，而且人人可见，他们都不敢再上课打盹，连打呵欠都不敢了，害怕被处罚。这种激励（机制）对集中注意力学习并无帮助。事实上，学生们会把上课的时间都用在集中注意保持清醒上，直到下课。“下课了没人出教室，”他说，“我们全趴桌上睡觉呢。”
A programmer at Hanwang Education’s research center demonstrates the company’s “Class Care System” in Beijing, Jan. 2, 2019. Xue Yujie/Sixth Tone
Online, China’s “intelligent education” systems have faced critical scrutiny. Under the #ThankGodIGraduatedAlready hashtag, which has over 23 million views, Weibo users have compared Hanwang’s CCS to George Orwell’s dystopian sci-fi novel “1984.” In a forum discussing Hangzhou No. 11’s system on Zhihu, China’s Quora-like online platform, the comments are uniformly critical. “The smiling face you see on a monkey in a circus is not a smile of joy, it’s a grimace of fear,” one user wrote. “Can you manage to focus in class when you know there’s someone standing behind the classroom? Let alone knowing there’s a camera.” And when students from other schools asked why their teachers started collecting facial data, someone responded: “The whole country is advocating ‘intelligent education.’ It’s probably your principal who wants to add glory to his career accomplishments.”
在网上，国内各路“智能教育系统”正面对着批判性的审视。“幸亏毕业早”的话题已经被浏览了 2300 万次。微博用户们把汉王的“课堂呵护系统”和乔治·奥威尔的科幻小说《1984》进行比较。知乎上的意见高度统一，全是批评：“马戏团的猴子表演的“笑容”，其实是它们感到恐惧的表情。”一个用户写道。“知道有人在背后看着你，你还能专心听讲？更别说摄像机了。”而其它学校的学生在被问到他们学校的老师为什么开始收集面部数据的时候，有人回答：“全国都在推动‘智能教育’，可能是因为校长想给政绩镀金吧。”
When I ask to visit Hangzhou No. 11, Vice President Zhang Guanchao declines my request but insists the facial recognition system is a good thing, assuring me over the phone that the system doesn’t violate students’ privacy, since it only records facial data rather than displaying the students’ faces. In addition, he sends me a promotional video set to lively background music, featuring a student in the school’s uniform break dancing around the campus, scanning his face to pay for food at the canteen, borrow books from the library, and buy water from a vending machine. It turns out that the “smart classroom behavioral management system” is just one part of Hangzhou No. 11’s “five-star smart campus initiative,” which implements facial recognition technology throughout the school’s campus.
Back at Niulanshan, the students who went in for face sampling are still not finished. I head to the music classroom, where I find the seats rearranged so that only 16 are in the center of the classroom. Four A4 pieces of paper with the numbers 1 to 4 on them are stuck to different points on the blackboard. Hanwang’s Dong Wencheng tells the students to look at each number for 15 seconds, switch seats, and do it all again.
在牛栏山一中，学生们还没完成面部特征采集。我去了音乐教室，发现里面重排了座位，只有 16 个座位排在中间。黑板四角上各贴了一张 A4 纸，分别写着 1，2，3，4。汉王的董文城告诉学生们，要分别盯住四张白纸 15 秒钟，然后换座位再重复同样的动作。
我的理解是，学生在每个座位上都要坐一遍，每次都要盯住纸片各 15 秒钟。按照描述，每个学生的采集过程应该需要 16 分钟。
The students were updating their facial data, Dong explains. When they move their heads to look at the numbers, CCS’s camera is able to capture their face from different angles. Schools normally capture the students’ facial data at the beginning of the semester and update it every couple of months. “These teenagers look different every two months. Puberty, you know!” Dong laughs.
I ask him about the facial recognition system used at Hangzhou No. 11. “I don’t know if you’ve seen Hikvision’s product,” Dong says about his company’s competitor, “but I am not impressed by it. They only scan the classroom every 30 seconds, 80 times in a 40-minute class. We do it once a second, 2,400 times in total!”
我问董知不知道杭州十一中用的面部识别系统。“我不知道你看没看海康威视的产品，”董谈论着他们的竞争对手，说道，“我反正对它没啥好印象。他们每隔 30 秒才扫一遍教室，一堂 40 分钟的课只扫 80 次。我们每秒就扫一次，总共 2400 次呢！”
Jason’s teacher, Guo, tells me she’s quite impressed by CCS so far, although the teachers don’t use it that often. “It’s good, but it’s machines, not humans,” she says, adding that there are some features that need to be more “intelligent” — the reports are mostly numbers, but the teachers want to know more about what these numbers mean and reflect.
Niulanshan’s principal, Wang Peidong, who has over 40 years of teaching experience, is also dismissive of CCS. “It’s not very useful,” he says. “You think a teacher standing on a podium needs AI to tell her if a student is sleeping in class?”
“Then why is it still used in your classrooms?” I ask.
“Zhang Haopeng is an alumnus of our school. He wants to do experiments here, so we let him. We don’t need to pay for it anyway,” he says, already walking away to get to his next meeting.
A Booming Business
In May 27, 2017, AlphaGo, AI developed by Google’s DeepMind, defeated Chinese Go player Ke Jie — the best human Go player in the world. Chinese-American entrepreneur Kai-Fu Lee refers to this as China’s “Sputnik moment.” Indeed, China’s best player losing an ancient Chinese chess game to AI developed by a Silicon Valley tech company later became the catalyst for the country’s drive to close the gap on AI technology.
2017 年 5 月 27 日，由辜狗的子公司 DeepMind 开发的人工智能 AlphaGo 打败了中国的围棋棋手柯洁。柯洁是全世界最好的人类围棋棋手。华裔美籍的企业家李开复认为这是中国应该奋起直追的时刻。事实上，中国最好的中国传统围棋棋手败给了美国硅谷一个科技企业开发的 AI，这件事后来成为了中国弭平 AI 技术鸿沟动机的催化剂。
Less than two months after Ke Jie lost to AlphaGo, the Chinese central government issued its ambitious development plan to improve the country’s AI capabilities. By the end of 2017, China’s venture capital investors had responded to that call, pouring record sums into AI startups in all sectors, including education, health care, and finance. In total, Chinese capital made up 48 percent of all global AI venture funding for 2017, surpassing the United States for the first time.
柯洁输给 AlphaGo 之后不到两个月，中央政府发布了宏大的人工智能实力发展规划。截至 2017 年底，中国的风投公司响应号召，给包括教育、医疗、金融等等的各路 AI 初创公司疯狂投钱，2017年，中国资本在 AI 智能领域的投入，已经占了全球该领域投资的 48%，首次超过了美国。
According to Wang Shengjin, a professor at Tsinghua University’s Department of Electronic Engineering, facial recognition is currently the technology’s most feasible and mature application, based on a greater potential for widespread use. This is due to the technology’s ability to utilize deep learning, which has turbocharged the pattern recognition capabilities of machines.
据清华大学电子工程系教授 王生进 所言，面部识别目前正是（人工智能）技术最易行、最成熟的应用，并且具备更大的广泛使用的潜力。这是因为面部识别技术有利用深度学习的能力，而后者对机器类型识别能力有巨大的提升作用。
As mentioned above, CCS uses deep-learning algorithms to analyze the identifying patterns of a person’s face. Wang tells me that, traditionally, this is done with the facial feature detection method called Eigenface, which can be unreliable and significantly limited to facial angles, distances, and resolution. “If you laugh or cry, the distances and shapes of your facial features change completely,” Wang says. But by utilizing deep learning, systems like CCS avoid these limitations. Professor Wang explains that this method essentially studies a multitude of different, labeled images and learns to identify a specific subject under all possible conditions.
之前提到过，“课堂呵护系统”分析人脸模式使用的是深度学习算法。王生进告诉我说，之前这个功能是用的一个叫 Eigenface 的面部特征探测方法。它并不稳定，对面部角度、（面部）和摄像机间距离、分辨率等等都有很大限制。“如果你笑或者哭的话，面部特征（点之间）的距离和形状就会发生完全的变化。”他说。然而，通过深度学习，类似“课堂呵护系统”之类的就可以避开这些限制。王教授解释说，深度学习的这种方法基本上会学习被标记的大量不同的图片，在各个不同的场景下识别出特定的主题来。
It’s a complex process, but one thing is clear: Training machines to identify a person’s face with deep-learning methods requires massive amounts of data. Wang tells me that Silicon Valley-based tech companies and research institutes are still in the lead when it comes to advanced AI academic research. But China has one advantage: massive amounts of data. According to The New York Times, China had around 200 million public surveillance cameras as of July 2018; the country is expected to install 626 million by 2020. Not all of these surveillance cameras have facial recognition capabilities, but the images they gather could provide huge amounts of data to train deep learning-powered facial recognition tools like CCS.
这个过程很复杂。但是一件事情很明确：用深度学习的方法训练机器识别人脸，需要海量的数据。王生进告诉我说，在高级人工智能学术研究方面，硅谷科技企业和研究院仍然占据领先地位，但中国有个优势，就是大量的数据。据纽约时报报道，截至 2018 年 7 月，中国有大约 2 亿公共摄像头，并且计划到 2020 年要安装共计 6.26 亿只摄像头。并非全部这些摄像头都有面部识别能力，但它们收集的图像可以提供海量的数据，用于训练基于深度学习的面部识别工具，比如“课堂呵护系统”。
The utilization of deep learning algorithms allows for three broad types of facial recognition: 1:1, 1:N, and M:N. The 1:1 type is often used at transportation hubs to verify that a person matches their ID photo, for example. 1:N is used to identify one person from a group of people, like when clocking in at the office. And M:N, the most complex of the three methods, is used to identify multiple people within a larger group, such as spotting criminals on the streets.
These methods create various market opportunities for facial recognition, the most obvious being policing and public safety. For example, in one instance, police used AI-based technology to identify and arrest 25 criminals at a beer festival in Qingdao, a city in eastern China’s Shandong province. However, many companies are also adapting this technology for commercial use, such as KFC and search engine giant Baidu, who have collaborated to develop facial recognition technology that can customize food orders by identifying a customer’s age, gender, and mood.
这些（需求）给面部识别创造了广泛的市场机遇。最明显的就是警察和公安。例如，警察在青岛的一场啤酒节上使用基于 AI 的技术识别并逮捕了 25 名犯罪分子。此外，很多公司也会将这种技术改用作商业用途。例如肯德基和百毒合作开发了面部识别技术，通过识别消费者的年龄、性别和情绪帮他们自定义食品菜单。
Last December, I visited a facial recognition tech company called Ovopark based in Suzhou, a canal city in eastern China’s Jiangsu province. Ovopark recently partnered with Meituan — China’s answer to Yelp — to help brick-and-mortar businesses install facial recognition cameras in their stores. When a customer enters the store, the camera uses facial recognition to identify if the person is a VIP customer or a frequent visitor. The camera captures the shopper’s shopping history and stores it, allowing the sales assistants to provide a more customized shopping experience. Ovopark CEO Zhou Youwen tells me the facial recognition cameras at Ovopark can also provide customized shopping suggestions.
去年 12 月，我采访了苏州一家叫作 万店掌 的面部识别技术公司，它和美团合作帮助实体经济公司在门店中安装面部识别摄像机。摄像机能使用面部识别技术识别贵客和常客，还会收集购买者的购买记录并且存储，销售助理就可以为顾客提供更加个性化的购物体验。万店掌的 CEO 朱友闻 告诉我说，万店掌的面部识别摄像机还能（直接）提供个性化的购物建议。
Wu Yingxia, Ovopark’s sales manager, stands in front of a product that rates people’s perceived attractiveness in Suzhou, Jiangsu province, Dec. 18, 2018. Xue Yujie/Sixth Tone
吴映霞（音）是万店掌的销售经理。她站在一个能给人的外貌打分的设备前。摄于江苏苏州，2018 年 12 月 18 日 薛玉洁/第六声音 摄
In Ovopark’s showroom, I stand in front of the camera, which identifies me as an angry 40-year-old female — 15 years older than my actual age. “Try a bigger smile,” Ovopark Sales Manager Wu Yingxia suggests. This time it works. The angry face emoji next to my face on the screen changes to a happy one, and the machine correctly identifies me as 25 years old. I have to maintain my awkward smile, otherwise the emoji will immediately revert to an angry face.
在万店掌的展览室中，我站在摄像机前，被识别成了一个愤怒的 40 岁女性——比我的实际年龄大了 15 岁。吴映霞建议我说：“你笑得明显点嘛。”这回机器没什么问题了。屏幕上我脸旁边表示生气的 emoji 变成了表示开心的，机器正确地识别出我 25 岁大。我不得不保持我局促地笑容，不然那个 emoji 就会变回生气的样子。
Another huge screen fitted with a mounted camera tells me which celebrity I look like and scores me based on my perceived attractiveness — the higher the score, the bigger the discount. Ovopark sells this product to shopping malls for entertainment. I score 95 out of 100. Wu tells me Ovopark adjusted the scoring criterion so that it doesn’t give customers too low of a score — she scores a 93. “It’s just for fun, after all,” Wu says as her score flickers in bright blue lights on the LED screen with the message: “You look like an angel.”
另外一个大屏幕连着的摄像机告诉我我看起来像哪个明星，还会按照我的外貌给我打分。分数越高，折扣越大。万店掌把这种设备卖给超市供娱乐用。我得了 95 分（满分 100）。吴映霞说公司调整了打分的标准，不给顾客打分太低。她得 93 分。“就是图个乐子的嘛。”屏幕上，这个分数闪着蓝色的光，上面还有句话说“你看起来像天使”。
It makes me think of the students at Hangzhou No. 11 Middle School.
A Modern-Day Panopticon
Accuracy isn’t much of a concern with the more frivolous applications of facial recognition, like the one I tested at Ovopark. But when the same technology is used in identification-sensitive fields like policing or finance, the resulting inaccuracies could lead to wrongful accusations and convictions, fraud, or theft. For instance, in November 2018, Chinese authorities wrongly accused entrepreneur Dong Mingzhu of jaywalking after a streetside camera identified her face in an ad on the side of a bus.
对于像我在万店掌看到的那些粗浅的面部识别应用，精确度并没有什么所谓。但是这种技术用于识别的时候（比如公安、金融这种敏感领域），识别结果如果不精准就可能导致错误的逮捕和定罪，或者欺诈和盗窃。例如，在去年 11 月，中国街边的摄像机在公交车的车身广告上拍到了董明珠的头像，以为她在横穿马路。
Everyone I talked to at Hangzhou No. 11 Middle School and Niulanshan First Secondary School expressed skepticism about the accuracy and reliability of facial recognition technology. As part of their smart campus initiative, Hangzhou No. 11 uses Hikvision’s facial recognition cameras to record the students’ attendance rate and for on-campus payments, but it doesn’t seem to work very well. A female student told me that Hikvision’s system is particularly inaccurate for girls. “Once we change our hairstyles or wear glasses, the camera won’t recognize [us] anymore,” she says through text. The different lighting and angles of their faces also slow down the recognition process, making the lines during lunch extremely long.
“The technology is not perfect yet,” admits Professor Wang Shengjin, “but you can’t always wait for technology to become perfect before using it.” Wang believes that practice makes perfect: The more we use facial recognition technology, the more problems we discover and solve, ultimately leading to perfected facial recognition systems.
The laws aren’t perfect, either. In fact, there are none. “No, there is no law regulating the use of facial recognition technology or other biometrics data in China,” says Hu Lin, an assistant law professor at Shanghai University of Economics and Finance. He tells me that, since there is currently no law prohibiting facial recognition in China, there’s nothing illegal about what these “intelligent education” systems are doing.
法律法规也不健全。事实上，根本就没有相应的法律法规。“没有。中国还没有能规范面部识别技术或者其它生物数据（使用）的法律。”上海财经大学法学院副教授 胡凌 说。他告诉我说，由于现在还并没有法律禁止在国内使用面部识别，“智能教育系统”的所作所为自然也就没有什么非法。
But there’s a bigger question: Even if it’s technologically possible and legally acceptable for schools and tech companies to use in-class evaluation systems powered by facial recognition, should we use them?
I ask He Shanyun, an associate professor of education at Zhejiang University. She tells me that the schools and “intelligent education” developers need to prove that the data they’re collecting is reliable for measuring educational performance. “If a student was burying his head in his desk but was actually looking for a pen, or if two students were talking but to discuss the teacher’s question, it’s not fair to classify them as being distracted,” He says.
Facial recognition technology would also need to consider the “cultural context” of people’s facial expressions and behaviors, He points out. She gives the example of a silent student who’s not answering questions or showing any expression on their face. “People from some cultures are more likely to express themselves through facial expressions and actions, but Chinese are normally more reserved,” He says. She also notes that classrooms are like mini ecosystems, where countless human interactions happen simultaneously in a small space. Each student brings their own culture, family values, and experiences. None of these factors are easily analyzed by capturing facial expressions. Machines are still less adept than humans at understanding cultural context and behaviors, according to Professor He. “We should encourage the use of new technology in daily life, but when it is used as a tool to evaluate individuals, more caution is needed,” she tells me.
Hangzhou No. 11 also seems to realize the technology’s inadequacies. Over the phone, Vice Principal Zhang Guanchao tells me the school has updated its facial recognition system and will no longer evaluate facial expressions. The students will now only be given a negative score if they’re captured lying on the desk.
But even if facial expressions accurately reflect students’ educational performance, does that justify the schools and developers using facial recognition on students? The answer, Professor He tells me, comes down to China’s philosophy on education.
“If our society thinks education is something that can be evaluated by statistics such as exam scores or in-class performance,” says He, “then don’t blame the schools for using algorithms to determine if you are a good student or bad student.”
She agrees that the intention of “intelligent education” is positive. As the NGAIDP guidelines suggest, the purpose of the initiative is to assist teachers in developing customized teaching methods and study plans for every student. But the current use of facial recognition technology in the classrooms worries her. “These statistics aren’t completely useless, but relying too much on them is not good for teaching,” He says.
“If in the end, the technology is only used to rank students by how many times each one yawns and punish them for doing so, it’s indeed a waste of the technology,” she says, suggesting that teachers should be trained on how to appropriately analyze and use the data as a reference rather than as decisive criteria. “We can’t push back the tide, but we should at least start trying to manage it.”
A focus on inaccurate statistics also worries most of the students I talked with about CCS. In particular, Jason Todd is afraid that his score will eventually be used to decide whether he is able to attend his dream school. Professor He believes schools and teachers should get the students’ consent and inform them of their intentions before using the data to evaluate performances.
Students at Niulanshan First Secondary School in biology class in Beijing, Jan. 3, 2019. Xue Yujie/Sixth Tone
北京牛栏山一中的同学们正在上生物课。2019 年 3 月 9 日， 薛玉洁/第六声音 摄
Most students at Niulanshan First Secondary School still don’t realize how teachers know everything that happens in the classroom. And although the students at Hangzhou No. 11 know their every wink and yawn is captured and expressed as a score, the school never asked for consent. Neither school has published any information on their websites indicating that they obtained the consent of students or parents prior to installing the system. Jason Todd even checked with his mom, who confirmed the school never asked for her permission, either.
According to law professor Hu Lin, the lack of consent in the use of the surveillance systems creates an imbalance of power. “The schools hold the power to evaluate, punish, and expel,” he says. “The parents won’t sacrifice the students’ futures by standing up against the schools, which leaves the students in the most vulnerable position.”
Hu refers to the panopticon, a circular prison discussed by French philosopher Michel Foucault in his book “Discipline and Punish,” in which inmates are observed by a single watchman but cannot tell if and when they are being watched, forcing them to act as if they are always being watched. To Hu, using systems like CCS will have the same impact, encouraging students to simply act like they’re behaving.
胡教授提到了一种圆形的监狱。法国哲学家 米歇尔·福柯 在他的著作《规训与惩罚》中提到过这种设计：圆形监狱里的囚犯会受到一名监控者的监视，但是囚犯并不知道他们是否在被监视着。这会使得他们一直像受到监视一样活动。胡教授认为，使用类似“课堂呵护系统”的设备也会有类似的效果——这会鼓励学生们约束自己的行为。
But Hanwang’s Zhang Haopeng isn’t worried. “We are all role-players in certain circumstances,” Zhang says. “You can pretend for one hour, two hours. But if you can make it work by pretending to listen to class for eight hours a day, I respect you.” Even if students are simply acting like they’re listening in class to pass the CCS, maybe one day it will become a real habit rather than role-play, Zhang tells me.
The increasing use of facial recognition technology is already raising concerns in Western countries. In July 2018, Microsoft’s president and chief legal officer, Brad Smith, wrote an open letter calling for U.S. federal regulation of facial recognition technology. In February, Amazon followed suit. San Francisco is considering banning its city agencies from using facial recognition, and the European Union’s General Data Protection Regulation (GDPR) has categorized facial data as “sensitive personal data,” which affects how it can be handled.
对面部识别技术的使用日益增长，已经引起了西方国家的担忧。在 2018 年 7 月，微软的主管和首席法律（事务）官 布拉德·史密斯 写了封公开信，呼吁美国联邦政府立法规范面部识别技术。在二月，亚马逊也仿效了微软。旧金山正考虑禁止其市政机构使用面部识别技术。欧洲的通用数据保护条款（GDPR）将面部数据分类为“敏感个人数据”，这会影响其被处理的方式。
But the current situation in China is worrisome. The government’s “intelligent education” initiative has attracted other companies eager to profit from bringing surveillance technology into China’s classrooms. According to a report published last December by state-run newspaper Global Times, 10 schools in southwestern China’s Guizhou province now use chip-equipped “smart uniforms” developed by a local company to track the exact locations of their students to encourage better attendance rates. A high school in southern Guangdong province has recently received criticism for forcing students to wear “smart bracelets” that also track their location. And a Massachusetts-based startup called BrainCo — founded by Chinese entrepreneurs — has signed a deal with a Chinese distributor to provide schools with 20,000 headsets that monitor student concentration levels by reading and translating their brain signals in class.
但是现在中国的现状很是让人担心。政府的“智能教育”计划正吸引其它公司蜂拥而来，把监控技术带到课堂当中牟取利润。去年 12 月，《环球时报》发表了一份报告，贵州省的十所学校采用了一家当地公司开发的带芯片的“智能校服”，跟踪学生们的确切位置来提高出勤率。广东的一所高中强制学生佩戴智能手环跟踪学生位置，受到了社会批评。马萨诸塞州的一家公司和中国的一家分销商签署了订单，给学校提供 20000 套佩戴式设备，监测学生们的脑电波，判定他们课上的专心级别——前者也是由中国的企业家创办的。
More schools and education bureaus are jumping on board, too. The Jiangsu Provincial Department of Education issued guidelines for developing “intelligent education” last May, calling for more schools to build “smart classrooms” that are capable of “collecting in-class behavioral data.” It also plans to subsidize more schools and companies to bring these AI-driven initiatives into classrooms.
Despite the shortcomings and ethical criticism, Hanwang Education’s Zhang Haopeng remains confident in the CCS initiative. “There have [always] been researchers in the academic field doing classroom behavioral observations. Do you remember in primary school, there were tutors occasionally sitting in the back of the classroom, taking notes and evaluating the teacher’s and the classroom’s performance?” Zhang asks me. “We just replaced them with a camera.”
On Lunar New Year, I text Jason Todd, wishing him good luck and following up to see whether his mom thinks his school should use CCS. His mom is right next to him, and he replies immediately. “She said yes, because she hopes the school will keep a better eye on us,” he writes, adding a string of emojis with bitter smiles.
I ask him to show his mom the classroom photo he discovered online — the one where each kid’s face is surrounded by a colored rectangle. A few minutes later, Jason texts back: “She says, ‘No way. It looks like a prison.’”
Illustrations: Wang Zhenhao; visual editor: Ding Yining; editors: Julia Hollingsworth, Chris Bolin, Clayton D’Arnault, and Matthew Walsh.
This article was published in collaboration with “The Disconnect.”