If you work outside of a tech company, chances are you’ve spent this week primarily concerned with getting ready for Christmas. If you work inside a tech company, there’s a significant chance your company spent much of this week patching a critical vulnerability in an open source Java logging library called Log4J.
Here’s a non-technical explanation of the problem:
It’s a vulnerability that was discovered in a piece of free, open source software called log4j. This software is used by thousands of websites and applications, to perform mundane functions most people don’t think about, such as logging information for use by that website’s developers, for debugging and other purposes.
Every web application needs functionality like this, and as a result, the use of log4j is ubiquitous worldwide. Unfortunately, it turns out log4j has a previously undiscovered security vulnerability where data sent to it through that website — if it contains a special sequence of characters — results in log4j automatically fetching additional software from an external website and running it. If a cyberattacker exploits this, they can make the server that is running log4j run any software they want — including software that can completely take over that server. This is known as a Remote Code Execution (RCE) attack.
To use a technical phrase, this is Really Bad.
The net result is that, left unaddressed, cyberattackers right now can completely take over thousands of websites and online applications, allowing them to steal money, data, and access. The security community has been completely focused on this vulnerability for the past two days, and updating servers running log4j as quickly as possible to protect against this vulnerability.
The good news is that mitigations are relatively easy to implement. The bad news is that left unmitigated, the vulnerability is extremely easy to exploit. iCloud, Minecraft, Baidu, and many other sites have been confirmed to be vulnerable so far, and you’ll likely hear more about many other sites being vulnerable in the coming days.
And those companies are just the tip of the iceberg. LAMP stacks (Linux, Apache, MySQL, and PHP) are used as the technological underpinnings for a wide variety of web applications of all sizes. (It’s not universal, as NGINX has taken over as a market leader from Apache, and there are still a few all-Microsoft houses that use IIS, and neither of them have the vulnerability.)
Open Source has been a revolutionary invention because it provides rapid development by armies of distributed developers, and Linus’s Law states that “with enough eyes, all bugs are shallow.” But there are tens of thousands of Open Source components out there running critical infrastructure that haven’t had nearly as many eyes on the code as the Linux kernel. It’s simply the nature of the beast. XKCD had a cartoon for this occasion:
Internet applications gain usefulness from widespread adoption and the number of other components they tie into and support. You know what creates new vulnerabilities? A larger user base and the number of other components they tie into and support, which creates more attack surfaces for malicious actors to exploit.
The flaw isn’t the fault of Random Guy in Nebraska, the fault is the company adopting software that they can’t possibly test for all the use-cases they’re going to use it for. Surprise! Just about every high tech company in the world is in the same boat. Pretty much everyone uses a wide panoply of open source tools for their Internet applications, and no one can test all the permutations of how each component might be put to use.
You can’t eliminate the risk, you can only minimize and mitigate it. You can use containerization strategies (Docker, Kubernetes, Container D, etc.) to minimize attack surfaces and limit contagion. You can run all your code through security scanning tools on your CI/CD platform of choice. You can do constant testing and keep rolling backups of everything to limit risk and speed recovery. (You can also train your employees not to click on random email links without verifying the sender is who they say they are, and not to give any any account information or passwords over the phone, and train them enough so that the lessons stick, even though phising and human engineering weren’t factors in the Log4J vulnerability.)
But there still a good chance that the platform you’re using today is different than the platform you’ll be using ten years from now, and you’ll have to go through the same learning lessons discovering new vulnerabilities for the new platform all over again.
Castles made of sand all fall into the sea eventually…
Tags: Apache, data security, Internet, Java, Linux, Log4J, Open Source, technology
Dan Geer lost his job in 2004 (paper published in 2003) warning of software monocultures, of this this is a minor example:
https://en.wikipedia.org/wiki/Dan_Geer
This specific flaw was exposed 2016 – it’s a bit of a mystery why it only became widely know just now:
https://www.blackhat.com/docs/us-16/materials/us-16-Munoz-A-Journey-From-JNDI-LDAP-Manipulation-To-RCE.pdf
The specific flaw in this case seems to be a flagrant case of treating data as code – which is a mortal sin.
Kurt
[…] rather than “really, really bad,” which means its not nearly as bad as, say, Log4J was. Your Amazons and eBays are probably safe from the […]