Facebook’s ‘Red Team X’ Hunts Bugs Beyond the Social Network’s Walls

In 2019, hackers stuffed portable network equipment into a backpack and roamed a Facebook corporate campus to trick people into joining a fake guest Wi-Fi network. That same year, they installed more than 30,000 cryptominers on real Facebook production servers in an attempt to hide even more sinister hacking in all the noise. All of this would have been incredibly alarming had the perpetrators not been Facebook employees themselves, members of the so-called red team charged with spotting vulnerabilities before the bad guys do.  

Most big tech companies have a red team, an internal group that plots and plans like real hackers would to help head off potential attacks. But when the world began working remotely, increasingly reliant on platforms like Facebook for all of their interactions, the nature of the threats began to change. Facebook red team manager Nat Hirsch and colleague Vlad Ionescu saw an opportunity, and a need, for their mission to evolve and expand in kind. So they launched a new red team, one that focuses on evaluating hardware and software that Facebook relies on but doesn’t develop itself. They called it Red Team X.

A typical red team focuses on probing their own organization’s systems and products for vulnerabilities, while elite bug-hunting groups like Google’s Project Zero can focus on evaluating anything they think is important no matter who makes it. Red Team X, founded in the spring of 2020 and led by Ionescu, represents a sort of hybrid approach, working independently of Facebook’s original red team to prod third-party products whose weaknesses could impact the social giant’s own security.

“Covid for us was really an opportunity to take a step back and evaluate how we’re all working, how things are going, and what might be next for the red team,” Ionescu says. As the pandemic wore on, the group increasingly got requests to look into products that were outside of its traditional scope. With Red Team X, Facebook has put dedicated resources toward running down those inquiries. “Now engineers come to us and request that we look at things they’re using,” Ionescu says. “And it can be any kind of tech—hardware, software, low-level firmware, cloud services, consumer devices, network tools, even industrial control.”

The group now has six hardware and software hackers with broad expertise dedicated to that vetting. It would be easy for them to go down hacking rabbit holes for months at a time prodding every aspect of a given product. So Red Team X designed an intake process that prompts Facebook employees to articulate specific questions they have: “Is data stored on this device strongly encrypted?” say, or “Is this cloud container managing access controls strictly?” Anything to give direction about what vulnerabilities would cause Facebook the biggest headaches.

“I’m a huge nerd about this stuff and people I work with have the same tendencies,” Ionescu says, “so if we don’t have specific questions we’re going to spend six months poking around and that’s not actually that useful.”

On January 13, Red Team X publicly disclosed a vulnerability for the first time, an issue with Cisco’s AnyConnect VPN that has since been patched. It’s releasing two more today. The first is an Amazon Web Services cloud bug that involved the PowerShell module of an AWS service. PowerShell is a Windows management tool that can run commands; the team found that the module would accept PowerShell scripts from users who shouldn’t have been able to make such inputs. The vulnerability would have been difficult to exploit, because an unauthorized script would only actually run after the system rebooted—something users likely wouldn’t have the power to trigger. But the researchers pointed out that it might be possible for any user to request a reboot by filing a support ticket. AWS fixed the flaw. 

The other new disclosure consists of two vulnerabilities in a power system controller from industrial control manufacturer Eltek called Smartpack R Controller. The device monitors different power flows and essentially acts as the brains behind an operation. If it’s connected to, say, line voltage from the grid, a generator, and battery backups, it might detect a brownout or blackout and switch system power over to the batteries. Or on a day when the grid is functioning normally, it might notice that the batteries are low and initiate charging them.