Great video here. Pretty happy Vercel did very well without any special hacks or support. Some notes: - Vercel's default pricing is now Active CPU, where we only bill for CPU when it is used (i.e. not when waiting for the LLM) - We no longer use Lambda for all requests, but the video is absolutely correct that we use Firecracker VMs for all requests! - Theo is right that on a node.js CPU bound workload on our Performance-tier may only be 30% or so faster. Vercel's standard tier gives you 1 vCPU which is all you need for a single-threaded JS workload. The speed up comes from node.js having some internal multi-threading and Vercel's rust core running on a different thread. - As speculated in the video, the floating point math difference is likely explained by the v8 version. For such "loop benchmarks" it's extremely important to avoid hoistability since the VM is gonna try to cheat and not do any work. On Vercel, the v8 version depends on the node.js version you select. Cloudflare has to constantly update v8 because every v8 exploit is a cloudflare platform exploit (due to lack of VM boundary) and so they must live on the absolute edge (sometimes newer than Chrome) - @theo is 100% right that what really matters is being close to your database and other backends. My recommendation: Default to a compute provider in the same datacenter as your database (0-2ms baseline latency)
A few days ago, a new YouTube channel appeared and accused me of a lot of things. In particular, they claimed I had never actually benchmarked Cloudflare and Vercel, and Cloudflare is actually 3x faster at compute. So I built a benchmark to figure out the truth. Vid out now.
