20. The GPU, Canvas 2D, WebGL and WebGL2
“The CPU asks: what should I do next? The GPU asks: how many of you are there?”
The story: painting the mural
Section titled “The story: painting the mural”The city commissions a mural the size of a building wall. You could hire three brilliant painters — masters who can paint anything, reason about composition, change plans mid-stroke. They’d take months.
Or you could hire ten thousand people who can each do one simple thing very well: “paint your square this colour, according to this formula.” You give them all the same instructions at once — “if your square is above the horizon line, blend from orange to blue based on your height; otherwise, paint the sea” — and the whole wall is done in an afternoon. None of them could design a mural. Together, following one simple program in parallel, they’re unbeatable.
The CPU is the three masters: a few powerful cores, excellent at complex, branching, sequential logic. The GPU is the ten thousand painters: thousands of simple cores, all running the same small program on different data at the same time. Graphics — compute the colour of each of two million pixels, sixty times a second — is the perfect job for the crowd.
But there’s a catch the muralist would recognise: the crowd is fast once they’re working, and slow to brief. Every time you call them together to hand out new instructions (a new paint, a new formula, a new section of wall), everyone stops. That briefing is a draw call, and counting briefings turns out to be one of the most important things in graphics performance.
The graphics pipeline
Section titled “The graphics pipeline”Every GPU API — OpenGL, WebGL, Vulkan, Metal, Direct3D, WebGPU — is built on the same pipeline:
- Vertices. You describe shapes as triangles: lists of points (vertices) with attributes — position, colour, texture coordinates, normals — stored in buffers in GPU memory.
- Vertex shader. A small program that runs once per vertex, in parallel. Its main job: transform the vertex’s position from model space into screen space (applying model, view and projection matrices), and pass along data for later stages.
- Primitive assembly & rasterization. The GPU groups vertices into triangles and figures out which pixels (fragments) each triangle covers. Vertex outputs are interpolated across the triangle’s surface — which is why a triangle with red, green and blue corners shows a smooth gradient.
- Fragment shader (pixel shader). A small program that runs once per fragment, in parallel — potentially millions of times per frame. It computes the colour: sampling textures, lighting, effects. This is where blurs, lens distortion and refraction live.
- Per-fragment tests and blending. Depth testing (is something nearer already drawn here?), stencil, and blending with what’s already in the framebuffer (transparency).
- Framebuffer. The result lands in an image in GPU memory — the screen, or a texture you’ll use in a later pass.
Programs give the shaders their inputs through uniforms (values that are the same for every vertex/fragment in a draw: a time, a matrix, a colour), attributes (per-vertex data from buffers), and textures (images the shaders can sample).
Draw calls and state: why “how many briefings” matters
Section titled “Draw calls and state: why “how many briefings” matters”To draw something, the CPU-side code (your JavaScript, through the browser and the driver) must:
- Set state: which shader program, which buffers, which textures, which blend mode, which framebuffer, which uniforms.
- Issue a draw call: “draw these N vertices as triangles.”
Each draw call has a CPU-side cost — validation by the browser (WebGL is checked carefully for security), translation by ANGLE or the browser’s backend into native API calls, driver work — and each state change can force the GPU to flush or reconfigure. The GPU might render a draw call of 10,000 triangles as fast as one of 10. So:
- 1 draw call of 100,000 triangles: fast.
- 10,000 draw calls of 10 triangles each: slow — the CPU spends the whole frame briefing the crowd, the GPU sits idle. You’re CPU-bound in a graphics workload.
That’s why draw call count is one of the first numbers graphics engineers look at, and why techniques for reducing it — batching, instancing, texture atlases — matter so much.
The flip side is being GPU-bound: too much work per pixel. Expensive fragment shaders (many texture samples, blur kernels, refraction), high resolution (a phone at device pixel ratio 3 has 9× the pixels of DPR 1), and overdraw (many transparent layers on the same pixels) make the GPU itself the bottleneck. The author’s lens filters are a GPU-bound workload (expensive per-pixel work on large areas) multiplied by a CPU-bound one (many separate contexts and passes).
Try it: a raw WebGL triangle (no library)
What you'd see: one triangle with red, green and blue corners, blended smoothly across its face by the GPU's interpolation. Raising the triangle count issues one draw call per triangle, which is the slow way on purpose: draw-call overhead is paid on the CPU, so thousands of tiny draws eventually drop frames even though the GPU is bored. Instancing (Chapter 20) draws them all in one call.
Canvas 2D: immediate mode
Section titled “Canvas 2D: immediate mode”The <canvas> element with getContext('2d') gives you an immediate-mode drawing API: fillRect, arc, drawImage, fillText, paths and gradients. “Immediate mode” means the canvas doesn’t remember objects, only pixels: once you’ve drawn a circle, it’s just coloured pixels. To move it, you clear and redraw everything. (SVG and the DOM are retained mode: the browser keeps a scene graph of objects.)
When Canvas 2D is enough:
- charts and data viz with up to thousands of elements,
- simple games, drawing apps, image manipulation,
- particle effects in the low thousands.
In modern browsers, Canvas 2D is often GPU-accelerated under the hood, but you don’t control the GPU directly, and per-object JavaScript calls limit scale. When you need tens of thousands of animated objects, custom shaders, or 3D, you need WebGL (or WebGPU).
OffscreenCanvas
Section titled “OffscreenCanvas”A canvas renders on the main thread by default — so its drawing code competes with your UI (Chapter 11). OffscreenCanvas lets you transfer a canvas’s control to a worker: canvas.transferControlToOffscreen(), post it to a worker, and draw there with 2D or WebGL. The frames still appear in the page, but the drawing work never blocks the main thread. For a continuously animating background — like the author’s wallpaper canvas — this is exactly the right move.
WebGL and WebGL2
Section titled “WebGL and WebGL2”WebGL (2011) brought GPU programming to the web, based on OpenGL ES 2.0 — the mobile graphics API of the late 2000s. You write shaders in GLSL ES, upload buffers and textures, and issue draw calls. It’s a state-machine API: you bind things, then draw with whatever is bound.
WebGL2 (supported across all major browsers since Safari added it in 2021) is based on OpenGL ES 3.0, and adds features that matter for performance and capability:
- Instanced drawing (core, not an extension): draw many copies of the same geometry with per-instance data in one call.
- Vertex Array Objects (VAOs) (core): capture all the attribute bindings for a mesh in one object, so switching meshes is one bind instead of a dozen calls.
- Multiple Render Targets (MRT): a fragment shader writes to several textures at once (deferred rendering, G-buffers).
- 3D textures and texture arrays.
- Transform feedback: capture vertex shader outputs into a buffer — GPU-side particle updates without a round trip to JS.
- GLSL ES 3.00: integer types, more flexible loops,
texelFetch, and more. - Non-power-of-two textures with full features, more texture formats (including floating point render targets with extensions), uniform buffer objects, sampler objects, occlusion queries.
Default to WebGL2 today; keep a WebGL1 fallback only if your audience includes very old devices.
Hello triangle, raw
Section titled “Hello triangle, raw”Here’s the minimum to draw one coloured triangle in raw WebGL — the same code the demo above runs, trimmed:
const gl = canvas.getContext('webgl2');
const vs = `#version 300 esin vec2 a_pos; in vec3 a_col; out vec3 v_col;void main() { v_col = a_col; gl_Position = vec4(a_pos, 0.0, 1.0); }`;const fs = `#version 300 esprecision mediump float; in vec3 v_col; out vec4 outColor;void main() { outColor = vec4(v_col, 1.0); }`;
function compile(type, src) { const s = gl.createShader(type); gl.shaderSource(s, src); gl.compileShader(s); if (!gl.getShaderParameter(s, gl.COMPILE_STATUS)) throw gl.getShaderInfoLog(s); return s;}const prog = gl.createProgram();gl.attachShader(prog, compile(gl.VERTEX_SHADER, vs));gl.attachShader(prog, compile(gl.FRAGMENT_SHADER, fs));gl.linkProgram(prog);
// x, y, r, g, b for three verticesconst data = new Float32Array([0, .8, 1, 0, 0, -.7, -.6, 0, 1, 0, .7, -.6, 0, 0, 1]);const vao = gl.createVertexArray(); gl.bindVertexArray(vao);gl.bindBuffer(gl.ARRAY_BUFFER, gl.createBuffer());gl.bufferData(gl.ARRAY_BUFFER, data, gl.STATIC_DRAW);const pos = gl.getAttribLocation(prog, 'a_pos'), col = gl.getAttribLocation(prog, 'a_col');gl.enableVertexAttribArray(pos); gl.vertexAttribPointer(pos, 2, gl.FLOAT, false, 20, 0);gl.enableVertexAttribArray(col); gl.vertexAttribPointer(col, 3, gl.FLOAT, false, 20, 8);
gl.clearColor(0, 0, 0, 1); gl.clear(gl.COLOR_BUFFER_BIT);gl.useProgram(prog);gl.drawArrays(gl.TRIANGLES, 0, 3); // one draw callAbout forty lines to draw a triangle. Compile two shader programs, link them, upload a buffer, describe its layout (each vertex is 5 floats = 20 bytes; position at offset 0, colour at offset 8), and draw. The colour gradient is free: the rasterizer interpolates v_col across the triangle.
What Three.js abstracts away
Section titled “What Three.js abstracts away”A scene in Three.js:
const mesh = new THREE.Mesh(new THREE.TorusKnotGeometry(1, .3, 128, 16), new THREE.MeshStandardMaterial({ color: 0xff8800 }));scene.add(mesh, new THREE.DirectionalLight(0xffffff, 2));renderer.setAnimationLoop(() => { mesh.rotation.y += 0.01; renderer.render(scene, camera); });Behind those lines, Three.js handles: generating geometry and uploading buffers; building and compiling shader programs for physically-based materials with lights, shadows and fog (and caching them); a scene graph with transform hierarchies; cameras and projection matrices; sorting objects to minimize state changes and handle transparency; frustum culling (skipping off-screen objects); texture loading, mipmaps and colour spaces; and handling resizes and device pixel ratio. That’s tens of thousands of lines you don’t write.
What it doesn’t do for you: decide how many draw calls your scene needs, how big your textures are, how many lights cast shadows, or how high your render resolution is. Those decisions are still yours, and they’re where performance lives. renderer.info reports draw calls, triangles, textures and geometries per frame — watch it.
For the author’s Three.js maths-learning site: a handful of meshes, lines and labels is trivially cheap. Expensive things to watch are thousands of separate objects (instance them), high-resolution render targets for post-processing, and text rendered as many separate meshes.
Instancing and batching
Section titled “Instancing and batching”The briefing problem’s two classic solutions:
- Batching: merge many static objects that share a material into one big buffer and draw them in one call. Great for static scenery; inflexible for things that move independently.
- Instancing: one geometry, one draw call, N copies, with per-instance data (position, rotation, colour) in a buffer the vertex shader reads with
gl_InstanceIDor instanced attributes. WebGL2 has this built in (drawArraysInstanced); Three.js exposesInstancedMesh. Ten thousand particles, trees, grid cells or data points: one draw call.
In the demo above, the slider draws each triangle with its own draw call — the slow way, on purpose. With instancing, 5,000 triangles is one call and barely registers.
Context loss
Section titled “Context loss”The GPU is a shared resource. The OS or browser can reset it: a driver crash, the GPU process restarting, the system reclaiming GPU memory, too many contexts. When that happens, your WebGL context is lost: every buffer, texture and program is gone, and drawing silently does nothing.
You get events: webglcontextlost (call event.preventDefault() to signal you’ll handle restoration) and webglcontextrestored (recreate all your resources and resume). Libraries like Three.js handle some of this. In production, you should at least: listen for loss, stop the render loop, show a fallback (a static image) and attempt restoration. You can test it with the WEBGL_lose_context extension.
GPU memory
Section titled “GPU memory”Textures dominate GPU memory. A texture’s size is roughly width × height × bytes per pixel — 4 for RGBA8 — plus about one-third more if it has mipmaps (smaller pre-filtered versions for distance). So:
- a 2048×2048 RGBA texture ≈ 16 MB (≈ 21 MB with mipmaps);
- a 4096×4096 one ≈ 64 MB (≈ 85 MB);
- a full-screen render target on a 1080×2400 phone ≈ 10 MB each — and post-processing often uses several (the author’s blur passes and offscreen buffers live here);
- floating-point textures are 2–4× bigger.
A JPEG on disk might be 300 KB, but decoded and uploaded as a texture it’s tens of megabytes. On phones, GPU memory is shared with system RAM; exceed what the system tolerates and the context is lost or the tab is killed (Chapter 17). Use compressed GPU texture formats (via KTX2/Basis Universal in Three.js), right-size textures, reuse render targets, and dispose of what you don’t need (texture.dispose() in Three.js — garbage collection does not free GPU memory promptly).
Software fallback
Section titled “Software fallback”If the GPU or driver is blocklisted or unavailable (Chapter 17), browsers may run WebGL on the CPU via a software implementation. Chrome historically used SwiftShader for this. Software WebGL is correct but slow: every vertex and fragment shader invocation runs on a few CPU cores instead of thousands of GPU cores. Fill-rate-heavy effects (full-screen fragment shaders, blurs) suffer most. Browsers have been tightening when they allow this automatic fallback, partly for security reasons — so on some blocklisted systems, WebGL may simply be unavailable.
What it means for you:
- Detect it: check
getContextsucceeds; read the renderer string; watch frame times. - Degrade: lower resolution, fewer effects, or a static image tier (Chapter 17).
- Never make critical content depend on WebGL.
The author’s numbers — gallery at 41 FPS with GPU and 21 without — are exactly this cliff.
Many canvases, many contexts
Section titled “Many canvases, many contexts”Now the audit’s line item: 17 canvases on one page.
Each WebGL canvas usually means its own WebGL context, and each context:
- has its own GPU resources (shaders compiled per context, buffers, textures, framebuffers) — the same lens shader and wallpaper texture duplicated 17 times;
- has its own command stream; switching contexts each frame adds overhead in the browser and driver;
- produces its own output surface that the compositor must composite as a separate layer;
- counts toward a per-page limit on active WebGL contexts. Browsers cap this (Chromium has long used a limit around 16); beyond it, the oldest context is forcibly lost with a console warning — “Too many active WebGL contexts. Oldest context will be lost.” Seventeen canvases is one too many.
The fix: one context, many views. Patterns:
- One full-page canvas behind (or above) the UI, positioned with
position: fixed; for each glass surface, render into its rectangle usinggl.viewportandgl.scissor(tracking element positions with IntersectionObserver andgetBoundingClientRect, carefully batched). One context, one shader compile, shared textures. This is how many “lots of small 3D views” sites work. - One offscreen context that renders each view and transfers the result to 2D canvases or
ImageBitmaps (transferToImageBitmap) — more copying, simpler layout integration. - Don’t use WebGL per component at all when the effect can be precomputed: a static pre-blurred backdrop image under a translucent surface costs almost nothing.
Plus the iframe multiplier: if each of those canvases lives in a separate iframe, each iframe is a separate document — possibly a separate process — making sharing a context impossible. Shared context requires the surfaces to live in one document.
Debugging tools
Section titled “Debugging tools”- Spector.js — a browser extension/library that captures one frame of WebGL commands: every draw call, every state change, every bound texture and shader, with a thumbnail of the framebuffer after each call. The fastest way to answer “why are there 900 draw calls?” or “why is this texture 4096×4096?”.
- DevTools Performance panel — the GPU track shows GPU busy time; the main thread shows your render loop’s JS (CPU-bound or not). Combine them: high main-thread time per frame with low GPU = too many draw calls/state changes; low main-thread with a pegged GPU = fragment-heavy or overdraw.
chrome://gpu— what’s accelerated, the renderer in use, blocklist status.- Three.js
renderer.infoand the stats panel (FPS, ms/frame, memory). - Shader errors:
getShaderInfoLog(always check compile status during development).
What you now know
Section titled “What you now know”You can explain the graphics pipeline and where shaders run; why draw calls and state changes cost CPU and how instancing and batching fix it; when Canvas 2D is enough and how OffscreenCanvas moves it off the main thread; what WebGL2 adds over WebGL; write a raw triangle and say what Three.js does for you; handle context loss; calculate GPU texture memory; understand software fallback; and explain — and fix — the cost of 17 canvases on one page.
Self-quiz
Section titled “Self-quiz”1. How many times does a fragment shader run for a full-screen pass at 1080×2400?
About 2.6 million times per pass (once per pixel), per frame.
2. Why can 10,000 tiny draw calls be slower than one huge one?
Per-draw CPU overhead (validation, translation, driver work, state changes) dominates; the GPU sits idle.
3. Name three WebGL2 features.
Instancing, VAOs, multiple render targets, 3D textures, transform feedback, GLSL ES 3.00.
4. How much GPU memory does a 2048² RGBA8 texture with mipmaps use?
About 16 MB × 1.33 ≈ 21 MB.
5. What happens when a page exceeds the browser’s active WebGL context limit?
The oldest context is forcibly lost (with a console warning).
Exercises
Section titled “Exercises”Warm-up 1. CPU- or GPU-bound? Trace A: main thread 14 ms per frame in your render loop, GPU 3 ms. Trace B: main thread 2 ms, GPU 22 ms. Diagnose each.
Solution
A: CPU-bound — too many draw calls/state changes or heavy JS; batch/instance. B: GPU-bound — expensive shaders, high resolution, overdraw or many passes; reduce render scale or effect cost.
Warm-up 2. Memory math. A scene uses four 4096² RGBA textures with mipmaps and two full-screen float16 render targets on a 1440×3200 phone. Estimate GPU memory.
Solution
Textures: 4 × 64 MB × 1.33 ≈ 340 MB. Render targets: 1440×3200×8 bytes ≈ 37 MB each → 74 MB. Over 400 MB — too much for many phones. Downsize textures and use compressed formats.
Warm-up 3. Retained or immediate? For: a 50-node org chart with clickable nodes; a 200,000-point scatter plot; a 3D globe.
Solution
SVG/DOM (accessible, hit-testing free); WebGL (or Canvas with care); WebGL/Three.js.
Lab 4. Lumen’s WebGL hero. Build a WebGL “lens” hero for Lumen: a full-screen quad whose fragment shader distorts the hero image around the pointer. Pause it when off-screen; render at reduced resolution on weak devices (Chapter 17’s tiers).
Hint
Upload the hero as a texture; pass pointer position and time as uniforms; sample the texture with offset UVs. Use IntersectionObserver for pausing and canvas.width = clientWidth * scale for render scale.
Lab 5. Instancing. Render 10,000 floating “dust” particles over the hero, first as 10,000 meshes, then as one InstancedMesh (or raw instanced draw). Compare draw calls and frame time with renderer.info and the Performance panel.
Expected outcome
10,000 draw calls vs. 1; the CPU per-frame time collapses.
Lab 6. Context loss drill. Use WEBGL_lose_context to force-lose the hero’s context; implement restoration and a static fallback.
Expected outcome
On loss, the static image shows; on restore, the effect resumes without a reload.
Lab 7. Spector capture. Capture a frame of the hero (or a Three.js example) with Spector.js. Count draw calls, list textures and sizes, identify one optimization.
Expected outcome
A frame report and a concrete fix (e.g., a texture larger than its display size, redundant state changes).
Staff 8. Re-architect the gallery. Write a one-page design for the author’s gallery: from 18 iframes × own canvas to a shared-context design with lazy mounting, budgets and fallbacks.
Model outline
Components render in one document (shadow DOM for style isolation instead of iframes); one fixed full-page WebGL canvas renders lens effects into each visible surface’s rectangle; surfaces register/unregister via IntersectionObserver; cap live surfaces; static fallback tier; measured targets (CPU ms/s, FPS, heap) before and after.