React Internals: High-Level Mechanics (pt 1)

This is the first post in a series of posts about React's internals. In this post I'll cover some very brief, very high-level, ground with respect to how React works and in the coming posts I'll dig a little deeper on reconciliation (Fiber), what it is, how it works, why it's fast, and how it integrates with Hooks. I'll also dig into rendering and how React translates its object representation of the DOM into actual DOM elements.

Why would I want to do this? To be real, it was all the hype around Hooks. What I originally started out wanting to know was how useRef works under the hood. How does it maintain its state through re-renders? I've read that it does it through closures (as do most--if not all--of the hooks), but that answer wasn't really sufficient. I wanted to see the difference in the code. But then I stumbled into the reconciliation package within the source. And that too was a mystery. I started to become overwhelmed. It took me almost two full days of exploration to get a semi-decent high-level of what was going on (two days!). So, I gave myself a goal, figure out what's going on when an application is initially loaded (ReactDOM.render(<App />, ...)). Seemed like a reasonable goal. These posts are my attempt to a) teach myself about React's internals (I learn by writing), and b) help others out by posting what I've figured out.

full disclosure: Max Koretskyi also wrote a series of wonderful posts on the topic here. Though, I've found that the code references are out-of-date, or no longer true, and the article was written before Hooks were released.

Onward...

Reconciliation

The Virtual DOM

It's natural to think of the DOM as a tree, effectively it is. We have a root node (<div id="root">, for instance), children, and siblings:

<div id="root">
  <div id="child1"></div> <!-- sibling to child2 -->
  <div id="child2"></div> <!-- sibling to child1 -->
</div>

When React renders your application it creates a representation of this relationship as a series of linked objects (a linked list). In fact, it creates two.

{
  type: 'div',
  state: { thing: 1 },
  child: { /* same object shape */ },
  props: { height: 100px, width: 100px, background-color: 'red' },
  parent: 234,
} // Where `child` can be many-levels nested (i.e. the whole app)

Here's a simplified look at what a representation of a simple div may look like (note: this is a simplification for illustration only)

It creates a linked list named current which describes the current state of the world and the other it calls workInProgress, which--you guessed it--describes the future state of the world. It's this second object which we colloquially refer to as the "Virtual DOM". From a forty-thousand-foot level, React looks at current, looks at workInProgress, then determines where to re-render. Pretty simple. But, hold on...that can get really expensive. For a really simple example, the time trade-offs are negligible, but for any sort of production-level application this is going to get ugly.

const App = () => {
  const [thing, setThing] = React.useState(0)
  const stylin = {
    backgroundColor: 'red',
    height: '100px',
    width: '100px',
  }
  
  return (
    <div>
      <div id="sibling1" onClick={() => { setThing(thing + 1) } style={stylin}>
        {thing}
      </div>
      <div id="sibling2" />
    </div>
  )
}

// ...and then, of course

ReactDOM.render(<App />, document.getElementById(“root”))

Let me introduce the canonical "App" for these posts: this is very simple and it may be slightly modified, but whenever App is mentioned this is what is being referenced.

This is where the React diffing algorithm, whose process is referred to as "reconciliation" comes into play. It employs a few things to take this complexity down to O(T) from a theoretical upper-bound (on the Edit Distance) of O(|T1|^2 |T2| log|T2|) or, roughly, O(T^3) (where T represents the number of items in the tree). That's a huge win, quadratic to linear! But how does it achieve this? Foremost, it leans heavily on its linked-list representation of the DOM.

This structure allows for avoidance of “walking” the tree (i.e. synchronously; which is how pre-16 React behaves). Within Fiber, the React Core team made a few decisions which allow for asynchronous updates (and, thus, better performance), but one decision in particular gives it its unique flexibility:

The JavaScript stack is avoided completely.

This means within the Fiber algorithm there’s no such thing as a recursive function (which necessitates the JS stack). What’s even more bizarre/cool is that Fiber’s implementation is its own stack! We’ll get more details on that design decision in future posts, but it opens the door to a much more flexible (albeit more complicated) system. But that’s not the only approach it takes; React will not rebuild your application tree on re-render unless one of these two things occurs:

  1. The element type of the root node was changed; e.g. <div> gets changed to <span>
  2. There was a reordering of siblings within the root; e.g. <li> elements were reordered

To expand on point 1: our root node can be any part of the app. In this case I use the term "root" to mean any parent node. If you change the element type, the root and all of its children will be rebuilt from scratch. Ex:

<!-- the DOM as seen by `current` -->
<div id="root">
  <div id="child1">Hey</div>
</div>

<!-- the DOM as seen by `workInProgress` -->
<span id="root">
  <div id="child1">Hey</div>
</span>

Oops! Rebuild from scratch time. The root changed element types.

This behavior makes intuitive sense. As far as the reconciliation process is concerned that's a totally different tree. Conceivably, it could determine that it was just an element change, but doing so would mean it would need to go into each child and check them against current's. Fine. But then we'd be back at O(T)^3 performance. Ouch. It's faster to rebuild the tree at this point. The grand lesson here is: avoid element switching unless you're not concerned with rebuild performance costs.

To point 2: If you simply reorganize the order in which the siblings are nested (using App as an example) say, putting sibling2 above sibling1, React will rebuild the tree. There is a way around this, however, and React will loudly educate you: the solution is to add a key attribute to the element with a unique identifier per sibling. In a sense, you're literally labeling the element so React can "look it up" in the other tree to see if anything has changed. Without that label (key), React will take the easy road and opt to rebuild.

Reconciliation, then, is what React does when the above two scenarios have not been met. Once a state change has been initiated (via setState) the workInProgress list will be generated (this is only true on initial load, after this the list is mutated and continues to live through the extent of the app's lifetime), which has the new state of the world reflected within it. React will then recurse on the children and "flush" any changes into the renderer. This, then, is where React translates its object into more familiar DOM elements or operations. In short, Reconciliation is the process of diffing the two trees.

{ // current
  type: 'div',
  state: { thing: 1 },
  child: { /* same object shape */ },
  props: { height: 100px, width: 100px, background-color: 'red' },
  parent: 234,
}

{ // workInProgress
  type: 'div',
  state: { thing: 2 },
  child: { /* same object shape */ },
  props: { height: 100px, width: 100px, background-color: 'red' },
  parent: 234,
}

As an example, say we have some update to our App: a button is clicked invoking setState which updates the workInProgress list. Now the reconciliation process is in full effect. React will now attempt to find a change in our App using the two objects above. The two “rebuild from scratch” rules have not been met so now it’s a matter of identifying the diff. In this case state.thing has increased by 1, so React will only update that property on that element, commit the work to the render process, and the client updates. So what’s the render process?

Render

get it into the DOM

There's a very fine dance going on between this process and the reconciliation process. The render process is the last step in painting your application to the browser window. It would actually be more accurate to state that the render process is what does the work of updating the App on the screen. The client interfaces with the correct renderer (ReactDOM in the case of a web client, for example) and the renderer interfaces with the reconciler. In the case of a web client, our first interaction with the render step is usually via:

ReactDOM.render(<App />, document.getElementById("root"))

The first argument is the element we wish to render and the second argument is the container we will render it into (note: the container is not manipulated by React, it's only used as a reference). On first call, anything within that container is recursively destroyed and replaced with the newly rendered element (if you're astute you'll notice this with a very brief white flash when you load a React application). We can see that in the source in ReactDOM.js#L493 (I've kept only the relevant code in the function):

function legacyCreateRootFromDOMContainer(container: DOMContainer, forceHydrate: boolean) {
    // on first load, `shouldHydrate` is `false` (comment mine, not in the source)
    if (!shouldHydrate) {
        let rootSibling
        while((rootSibling = container.lastChild)) {
           container.removeChild(rootSibling)
        }
    }
    return new ReactSyncRoot(container, LegacyRoot, shouldHydrate)
}

The function you see above (found in the ReactDOM file) is the last stop before venturing into the reconciliation process. As far as I can tell the first-load process order--roughly--goes: render --> reconcile --> render. Where the last render does the heavy lifting of translating React objects to DOM elements. Notably, the codebase doesn't specify the type of client to render to in the reconciliation process (web, native, etc), it leaves that to the client to determine. The responsibilities are very well separated.

Hopefully, this serves as a nice high-level for what React is doing when it receives a state change in your app. In the next post I'll speak more to the reconciliation processes and, specifically, Fiber. See ya then!