#privatemodel | Explore Tumblr posts and blogs

gampleman · 7 years ago

Text

Implementing VDOM in Elm

What if Elm didn't ship with a VDOM library? No HTML or SVG? You might be tempted that it wouldn't be very useful, but we can implement it ourselves using pure elm and ports. Ports? You heard right, good old trusty ports.

Simplest thing possible

So how can we render stuff? Well let's make a simple program to test:

module Main exposing (main) import MyHtml exposing (Html, program, div, text, href, a) import Time type Msg = Tick Float update : Msg -> Int -> (Int, Cmd Msg) update msg model = case msg of Tick t -> ( t, Cmd.none ) view : Int -> Html Msg view model = div [] [ text (model |> toString) , a [href "https://google.com"] [ text "GOOGLE IT"] ] main = program { init = ( 0, Cmd.none ) , update = update , subscriptions = \model -> Time.every Time.second Tick , view = view }

Looks pretty standard except for the funny import. Well, what does that look like?

port module MyHtml exposing (program, Html, Attribute, div, text, a, href) type Html msg = Node String (List (Attribute msg)) (List (Html msg)) | Text String type Attribute msg = Attr String String | Property String Json.Value

First, we defined some types to represent our HTML tree. Note that this is by no means complete, but can be easily extended to support things like SVG.

init : ( model, Cmd msg ) -> (model -> Html msg) -> ( model, Cmd msg ) init userInit userView = let ( initModel, initCmd ) = userInit in ( initModel, Cmd.batch [ render (userView initModel), initCmd ] ) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> msg -> model -> ( model, Cmd msg ) update userUpdate userView msg model = let ( newModel, newCmd ) = userUpdate msg model in ( newModel, Cmd.batch [ render (userView newModel), newCmd ] ) program userProgram = Platform.program { init = init userProgram.init userProgram.view , update = update userProgram.update userProgram.view , subscriptions = userProgram.subscriptions }

Next let's define our program function. We're essentially wrapping the users program, but on both init and update we're adding an extra Cmd in to the mix:

render : Html msg -> Cmd msg render = encodeHtml >> renderPort port renderPort : Json.Value -> Cmd msg encodeHtml html = case html of Node name attrs children -> Json.object [ ( "type", Json.string "node" ) , ( "name", Json.string name ) , ( "attributes", Json.list (List.map encodeAttr attrs) ) , ( "children", Json.list (List.map encodeHtml children) ) ] Text str -> Json.object [ ( "type", Json.string "text" ), ( "value", Json.string str ) ] encodeAttr attr = case attr of Attr key val -> Json.object [ ( "type", Json.string "attribute" ), ( "key", Json.string key ), ( "value", Json.string val ) ] Property key val -> Json.object [ ( "type", Json.string "property" ), ( "key", Json.string key ), ( "value", val ) ]

So here we have encoded our HTML structure into JSON and sent it over a port. I'll spare you from future encoders, but they all look follow this same pattern. Let's see what the JavaScript side looks like:

<div id="output"></div> <script> var app = Elm.Main.worker() app.ports.renderPort.subscribe(function(html) { const output = document.getElementById("output"); while (output.firstChild) { output.removeChild(output.firstChild); } render(html, output); }); function render(struct, out) { switch(struct.type) { case "node": var el = document.createElement(struct.name); struct.attributes.forEach(attr => { switch(attr.type) { case "attribute": return el.setAttribute(attr.key, attr.value); case "property": return el[attr.key] = attr.value; } }); out.appendChild(el); struct.children.forEach(child => render(child, el)); break; case "text": var el = document.createTextNode(struct.value); out.appendChild(el); break; } } </script>

So whenever the renderPort triggers, we remove all contents from our destination node and use DOM apis to build the whole structure. Note that in DOM apis there are two flavors of attribute - actual atttributes that need to be set with setAttribute and properties that have specialised setters.

Does it work? Check it out:

Supporting events

Alright, we managed to get something on the screen, but we want to build a real app and so we need to support events coming in. We'll need to modify our approach a bit.

Let's make a little counter app to demo this:

module Main exposing (main) import MyHtml exposing (Html, program, div, text, href, a, onClick) type Msg = Inc | Dec main = program { init = ( 0, Cmd.none ) , update = \msg model -> case msg of Inc -> ( model + 1, Cmd.none ) Dec -> ( model - 1, Cmd.none ) , subscriptions = \model -> Sub.none , view = \model -> div [] [ text (model |> toString) , a [ onClick Inc ] [ text "+" ] , a [ onClick Dec ] [ text "-" ] ]

Event handlers in Elm are Json.Decode.Decoders that transform JavaScript event objects into the users custom msg type. But Json.Decode.Decoders are really functions under the hood.

Now a bit of trouble presents itself. We can't encode JSON Decoders into Json.Value objects and send them over ports. So how can we set up event listeners?

To solve this, we need to build up a dispatcher, which we can store in our model. So let's make our own model, that will wrap the users model:

import Dict exposing (Dict) type alias PrivateModel model msg = { userModel : model , handlers : Dict String (Decoder msg) }

The handlers key is a dispatcher datastructure: it holds the decoders the user specified stored under string keys. But where do those come from? We'll use a simple trick and compute a path through the DOM tree as their key. So for example the Inc onClick handler above, would have the key path div.0.a:click.

Next, we'll introduce a new representation, which we'll call SafeHtml. This is exactly the same as Html msg, except it drops the type variable and inside the coresponding SafeAttribute stores these key strings instead of the decoders. Note that we will only expose Html msg to the user of our library, SafeHtml is an implementation detail. We also add a new constructor for creating event listeners:

type Node handler = Node String (List (NodeAttribute handler)) (List (Node handler)) | Text String type NodeAttribute handler = Attr String String | Property String Json.Value | Event String Options handler type alias Options = { preventDefault : Bool , stopPropagation : Bool } type alias Html msg = Node (Json.Decode.Decoder msg) type alias Attribute msg = NodeAttribute (Json.Decode.Decoder msg) type alias SafeHtml = Node String type alias SafeAttribute = NodeAttribute String

We also update our encoders to only support SafeHtml.

Now we have the key data modeling pieces in hand. We now need a function that will take a Html msg and give us back both a handlers dispatcher and a SafeHtml tree that we can use for rendering:

extractListeners : String -> Html msg -> ( Dict String (Decoder msg), SafeHtml ) extractListeners prefix html = case html of Node name attrs children -> let key = prefix ++ "." ++ name safeAttrs = List.map (makeAttrSafe key) attrs listeners = List.filterMap getListener attrs kids = List.indexedMap (\index -> extractListeners (key ++ "." ++ toString index)) children childListeners = List.foldr (\( a, _ ) b -> Dict.union a b) Dict.empty kids in ( List.foldr (\( k, fn ) d -> Dict.insert (key ++ ":" ++ k) fn d) childListeners listeners , Node name safeAttrs (List.map Tuple.second kids) ) Text s -> ( Dict.empty, Text s ) makeAttrSafe : String -> Attribute msg -> SafeAttribute makeAttrSafe prefix attr = case attr of Event key options tagger -> Event key options (prefix ++ ":" ++ key) Attr k v -> Attr k v Property k v -> Property k v getListener : Attribute msg -> Maybe ( String, Decoder msg ) getListener attr = case attr of Event key _ tagger -> Just ( key, tagger ) _ -> Nothing

Now let's build it and use it:

subscriptions : (model -> Sub msg) -> PrivateModel model msg -> Sub (Maybe msg) subscriptions userSubscribtions model = let eventDispatcher ( key, event ) = Dict.get key model.handlers |> Maybe.andThen (\decoder -> Json.Decode.decodeValue decoder event |> Result.toMaybe ) in Sub.batch [ eventPort eventDispatcher, Sub.map Just (userSubscribtions model.userModel) ] port eventPort : (( String, Json.Value ) -> msg) -> Sub msg init : ( model, Cmd msg ) -> (model -> Html msg) -> ( PrivateModel model msg, Cmd (Maybe msg) ) init userInit userView = let ( initModel, initCmd ) = userInit ( handlers, safeView ) = extractListeners "" (userView initModel) in ( { userModel = initModel , handlers = handlers } , Cmd.batch [ render safeView, Cmd.map Just initCmd ] ) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> Maybe msg -> PrivateModel model msg -> ( PrivateModel model msg, Cmd (Maybe msg) ) update userUpdate view maybeMsg model = case maybeMsg of Just msg -> let ( newModel, newCmd ) = userUpdate msg model.userModel ( handlers, safeView ) = extractListeners "" (view newModel) in ( { userModel = newModel, handlers = handlers } , Cmd.batch [ render safeView, Cmd.map Just newCmd ] ) Nothing -> ( model, Cmd.none ) program userProgram = Platform.program { init = init userProgram.init userProgram.view , update = update userProgram.update userProgram.view , subscriptions = subscriptions userProgram.subscriptions }

To handle the posibility that our dispatcher somehow gets out of sync with the DOM, we have to wrap the users message in a maybe. In the subscriptions call, we use the dispatcher to transform an incoming message (which is in the form of a (keyPath, eventObject) tuple) into the message the user expects. We handle the maybe in the new update function by simply ignoring the Nothing case. In both update and init we now call our extractListeners function to build up the dispatcher data structure.

And here's the updated JavaScript:

And that's basically all it takes to get events coming back in:

Making it Virtual

You may have noticed that we are re-rendering the entire DOM tree on every update. This is both not efficient and also potentially wrong, as it will lose focus on input elements and the like. The solution is to diff a new rendered view with the current one. This would create a bunch of patches that can then be applied to the real DOM for efficient updates.

The nice thing is that all but the application of patches can be done in Elm. So let's get started.

First let's look at describing changes between SafeHtml structures.

I've spared you the implementation of the decoders in the interest of brevity. They look just the same as the other decoders.

Now let's take a look at how we might apply these changes to the existing DOM:

This might seem like a lot of code, but it's fairly simple. The Change datastructure allows to find the node that should change (that's what the At constructor is for). Then we apply one of the 5 possible changes. Note that there are more possible mutations that could increase efficiency, like a reorder change, but we've done only these for simplicity. Facts is a term used in Elm's virtual dom to refer to attributes, properties and event listeners.

Ok, let's try to get something on the screen:

initialRender : SafeHtml -> Cmd (Maybe msg) initialRender = Insert >> Change >> encodeChange >> renderPort type alias PrivateModel model msg = { userModel : model , handlers : Dict String (Decoder msg) , view : SafeHtml } init : ( model, Cmd msg ) -> (model -> Html msg) -> ( PrivateModel model msg, Cmd (Maybe msg) ) init userInit userView = let ( initModel, initCmd ) = userInit ( handlers, safeView ) = extractListeners "" (userView initModel) in ( { userModel = initModel , handlers = handlers , view = safeView } , Cmd.batch [ initialRender safeView, Cmd.map Just initCmd ] )

Note that we store a reference to the current view. Let's do some diffing against it:

wrapAt : Int -> List Change -> List Change wrapAt i changes = case changes of [] -> [] list -> [ At i (batchIfNecessary changes) ] batchIfNecessary : List Change -> Change batchIfNecessary changes = case changes of [] -> -- This should never happen Batch [] x :: [] -> x list -> Batch list diff : SafeHtml -> SafeHtml -> List Change diff before after = if before == after then [] else case ( before, after ) of ( Text bstr, Text astr ) -> [ Change (TextChange astr) ] ( Node bName bAttrs bChildren, Node aName aAttrs aChildren ) -> if aName == bName then let attrsDiff = if aAttrs == bAttrs then [] else List.map2 diffAttrs bAttrs aAttrs |> List.concat |> Facts |> Change |> List.singleton childrenDiff = if bChildren == aChildren then [] else diffChildren 0 bChildren aChildren in [ batchIfNecessary (attrsDiff ++ childrenDiff) ] else [ Change (Redraw after) ] _ -> [ Change (Redraw after) ] diffAttrs : SafeAttribute -> SafeAttribute -> List ( Bool, SafeAttribute ) diffAttrs before after = if before == after then [] else [ ( False, before ), ( True, after ) ] diffChildren : List SafeHtml -> List SafeHtml -> List Change diffChildren index before after = case ( before, after ) of ( [], [] ) -> [] ( b :: efore, [] ) -> At index (Change Remove) :: diffChildren (index + 1) efore after ( [], a :: fter ) -> Change (Insert a) :: diffChildren (index + 1) before fter ( b :: efore, a :: fter ) -> case diff b a of [] -> diffChildren (index + 1) efore fter diffs -> At index (batchIfNecessary diffs) :: diffChildren (index + 1) efore fter

Again a fair amount of code, but the idea is pretty simple - we traverse both trees simultaneously and record changes as we see them. This code could be made more sophisticated/performant, but I tried to not make it too complicated.

I also tried to remove pointless wrapping structures, so that the changes output is easy on the eyes.

OK, let's get the diffing wired up into our update function:

render : SafeHtml -> SafeHtml -> Cmd msg -> Cmd (Maybe msg) render before after cmd = case diff before after of [] -> Cmd.map Just cmd changes -> changes |> batchIfNecessary |> At 0 |> encodeChange |> renderPort |> (\renderCmd -> Cmd.batch [ renderCmd, Cmd.map Just cmd ]) update : (msg -> model -> ( model, Cmd msg )) -> (model -> Html msg) -> Maybe msg -> PrivateModel model msg -> ( PrivateModel model msg, Cmd (Maybe msg) ) update userUpdate view maybeMsg model = case maybeMsg of Just msg -> let ( newModel, newCmd ) = userUpdate msg model.userModel ( handlers, safeView ) = extractListeners "" (view newModel) in ( { userModel = newModel, handlers = handlers, view = safeView }, render model.view safeView newCmd ) Nothing -> ( model, Cmd.none )

And that's basically it:

I've prepared a Gist version for easier reading of the completed source code.

Discussion

Now all this is neat, but so what? I wrote this for a few reasons:

It's meant as a bit more advanced tutorial on some cool ports techniques.

You are not tied to the built in rendering system. If you want to build your own rendering engine in Elm, I've just shown you how. I hope this can encourage some experimentation with cool techniques. One of the popular ones these days is to not diff against a Virtual DOM, but against the real DOM. You can try that pretty easily.

You can customise the way the rendering works. Need a crazy component (say a map library) to work within your tree? All you need to do is implement a new Node constructor.

After reading this, you should be much better equiped to go and read the real implementation. It's much harder to read, but most of the concepts are the same.

However, a few caveats are in order.

This will not beat the native version in performance anytime soon. We need to convert all of the Elm datastructures into native JS ones and that is pretty costly.

I've not widely tested this. It's meant as educational material, not a production ready library.

Hope you enjoy playing with it. Let me know on the Elm slack if you can use this for some cool experiments :)

#elm #programming

0 notes