In case you thought LLMs will solve all our problems and we can all enjoy the back seat now, because that’s not how it works. LLMs will simply expand a prompt with what’s likely. Or to be more precise: what’s likely a human has once written. Simple as that. AI is great for finding correlations in setups consisting of multiple factors (where the magnitude is overwhelming for a human). LLMs are great in digitalizing certain workload or synthetising content. But unless somethign has been well documented, LLMs won’t tell you X is wrong and you should do Y instead. We still need engineers.
Luckily, there’s entrepreneurship – the ability to find a challenge and, driven by good greed, offer a solution for it, in the form of a product. These days, the solution often is delivered through technology (and such is the definition of this word: asking the right questions and finding the most elegant answers). Sometimes the product itself is not technically relevant, but once the adoption of the product reaches millions, the enterprise can afford some great engineering at a scale.
Such is the case of ByteDance. A bit unusual for me to write about this, as I have nothing positive to say about their product and personally prefer to read text. I sigh whenever interesting content is only available as video (toss of a coin chance that it is viewable in 2x speed mode). Yet, TikTok is used by millions of customers and such scale allows the company to hire some great engineers, looking for some great solutions.
What have ByteDance engineers proposed? An improvement to IPC! IPC stands for inter-process communications and it is a POSIX standard adopted by Linux, Windows and many other operating systems. What’s interesting is that the standardisation process started in 1988, so quite a while ago and there’s usually little chance something so mature can be further improved, right?
Wrong. ByteDance engineers discovered that instead of using the POSIX available interfaces such as signals, sockets, shared memory or threads, processes can also communicate as function calls, similarly how a compiled program binary calls linked libraries. The solution is called RPAL, for “Run Process as Library” and allows avoiding duplicate memory allocation and involving data and control planes each time a communication shall occur. They have proposed this in a Linux kernel patch.
The results? For an unit testing type of scenario, the engineers are reporting 91.3% improvement over CPU cycles required. In a more realistic data center scenario, on a micro-service architecture, the improvement is an outstanding 15.5% improvement, which is truly remarkable.
What does it mean for the rest of the world? This improvement is likely to be merged into the Linux mainline, and as you might know, almost everything runs on Linux these days. This means a problem solved for TikTok will soon let any data center run more efficiently, including any app in the cloud, anywhere in the world, any business sector. It still won’t change my view on TikTok though 😉