Almost 2 months ago, we shipped Elder.js v1 and I wanted to do a recap of the lessons learned shipping it.

If you're unaware, Elder.js is Svelte static site generator that supports Partial Hydration, 0kb client side JS when not needed, and it is built with large scale SEO properties in mind.

In this post I cover:

• Demo from Svelte Summit
• Why I built Elder.js
• Why we open sourced it
• The major design decisions
• Things I'd like to improve.

Elder.js Demo Video

Yet Another Static Site Generator?

In 2019, I led the redevelopment of a non-trivial, data driven, SEO property from WordPress to Gatsby... and it was a nightmare I never want to repeat.

I'll save you the gory details but the lessons from that experience were:

• GraphQL is great, until it isn't. Using it for static sites is just silly.
• Look past the easy of setting up a project when evaluating frameworks. Instead look for how a framework manages the complexity of the project long term. Gatsby was easy to start with, painful to grow with.
• Client side routing with full hydration is just wasteful for static sites.

Coming out the other side of that project, I dreamed for a static site generator that featured:

• Clean data flow even when pulling from multiple data sources: raw SQL queries, external apis, and everything in between.
• OKB client JS if the page doesn't need it.
• No client side routing or service worker.
• "Fast" builds of sites with 10-100k pages. (I was thinking sub 2 hours back then.)
• Preview functionality during development.

Launching a New Project: Elder Guide

At the beginning of 2020, I set out with my friend Nick Lata to change the elder care industry and launched a site called Elder Guide.

At its core, Elder Guide is a data driven site that uses data to help consumers make informed decisions about senior care.

Before choosing which framework to marry this project to, I decided to build out tests projects using real data and designs across 6 different frameworks.

After 9 solid days of playing with static site generators, there wasn't a clear winner.

After testing Gatsby, Next.js, Nuxt.js, 11ty, Sapper and Hydrogen.js, I chose Sapper specifically because I fell in love with Svelte and couldn't go back to React.

Hiccups and An Afternoon

As time went on, my development speed was hamstrung by Sapper's slow builds and development reloads.

At the time, I was only working with a 5k page site, but I knew within 5 years, ElderGuide.com would likely have 100k pages or more.

Realizing that Sapper's slow builds were due to the way it "crawled" a site to render it, I gave myself an "afternoon" to explore what it would take to build an SSG from scratch.

After 4 hours of hacking, I had a proof of concept that included full SSR support for Svelte. The solution was really hack-y... but it worked and most importantly, build times and dev reload times were blazing fast.

Almost 7 months after committing an "afternoon" to building an SSG, Elder.js hit v1 and it appears that once again, I over estimated what could be done in an afternoon. ;)

Key Design Decisions

Having worked on, managed, or led the development of 5 major SEO assets and 50+ websites in total, the hardest part of any large project is managing complexity so things stay maintainable.

Elder.js is very opinionated on things, but the over arching goal is to help you and your team build maintainable, predictable websites.

Below are the major tools Elder.js offers to manage complexity along with the reasoning behind them.

• Svelte + Partial Hydration
• Shortcodes
• Hooks
• Plugins
• Builds that Scale With Resources

Svelte + Partial Hydration

If you haven't used Svelte on a project, you're missing out.

Svelte is a refreshing take on a frontend framework that makes you feel like you have super powers. It makes reactivity simple, allows you to mix your css/html/js in a single file and just makes web development fun again.

That said, the best part of Svelte is that it is compiled. This results in smaller client bundles than React/Vue and makes it a great choice for SEO.

When starting to build ElderGuide.com, the biggest problems I found with Svelte were:

1. Setting up a non-trivial Svelte SSR flow without Sapper isn't straight forward and requires a lot of experimentation and understanding of bundlers like rollup to get right.
2. There is no way to only partially hydrate SSR'd pages. This meant you couldn't sprinkle in Svelte goodness, instead your entire HTML page had to be hydrated and managed by Svelte. This meant writing 2x the data to the client and shipping a ton of unneeded JS.

Elder.js' Approach

Out of the box Elder.js ships with both a SSR express -like middleware so you can use it to power server rendered apps and a build process to statically generate all of the pages on your site. This is made possible because Elder.js internalizes the complexity of bundling the user's Svelte components with a surprisingly complex rollup configuration.

For partial hydration, Elder.js uses a preprocessor to allow users to decide which Svelte components should be hydrated and which shouldn't.

To hydrate a component, you simply add a component like so <MyComponent hydrate-client={propsHere} /> and Elder.js will mount a lazy-loaded MyComponent on the client.

Overall, the approach is pretty good but it has some rough edges:

1. Using hydrate-client isn't very Svelte-y. I'd much rather use hydrate:client but the Svelte language server doesn't support custom directives.^[1]
2. Partial hydration requires wrapping each Svelte component in a <div> with a unique ID so it can be mounted on the client. This can cause wonky layout issues with elements designed to be display:block; and display:inline-block. These are mostly fixed by CSS but they are known tradeoffs.

Results: Partial Hydration and Small Bundles

Honestly, the outcome is great. The interactivity that we're able to achieve using partial hydration is amazing considering the bundle sizes.

Sure, I wish Svelte had real support for partial hydration, but overall the outcome is very usable.

We've also found the hydrate-options to be very useful for fine grained control of how components are mounted.

Shortcodes: Power Tools For Managing Content Debt

In the SEO world, they say content is king.

What they don't tell you about content is that:

1. Content has a lifespan and when it becomes stale, it becomes content debt.
2. Content may be static, but it often need someone to reformat it to meet changing design / marketing / conversion goals.
3. Most of all, content needs to be managed. And managing it in a future proof way is hard.

Static Content Is A Problem

When you're building anything bigger than a small marketing site, you're generally going to want a system to manage your content.

Your system could be a folder full of markdown, a headless CMS such as Contentful, Prismic, WordPress or Strapi or even a home grown CMS.

Regardless of where your content is organized, it is generally static, unless you manually change it... but your site and its requirements often change over time.

This means that the amount of content on a site is generally limited to the number of people who can manage it.

When you're building 10-100k page websites, this makes costs hard to control... but the better your system for managing content and content debt, the leaner you and your team can be.

Below are real examples I've encountered that illustrate the power of shortcodes.

> Scenario 1: Youtube Embed Code

Your website publishes your Youtube videos like this one does.

Imagine that in each of your 300 posts over the past 6 years, you've copied and pasted Youtube's embed code.

You didn't think much of it, but now you want to change your design and you realize the embedded code you've been using has evolved over time.

You now have to manually update all of your prior posts to match the new desired changes.

This is content debt.

With Elder.js, you could use a shortcode like

and future proof your Youtube videos so you can change them all at once.

Here is the shortcode this site uses as of this writing:

> Scenario 2: Design Changes

You're in charge of managing your company's website. The marketing department has spearheaded a seemingly small design change to their marketing banners... but you've used a CSS framework like tailwindcss which now means you need to go and change .items-center to .items-stretch... the only problem is that this small change needs to be done 80 times because your content lives in a CMS and it is hardcoded.

As a programmer, you could dream up the regex required to fix it, but you realize things could get messy quickly so you decide to manually slog through updating 80 posts with the right class.

Next week, the marketing department calls and they need another change.

This is content debt.

With Elder.js, you could give the marketing department a shortcode like {{!banner!}}content goes here[/banner], which would allow you to update all of these banners at once.

Here is a shortcode this site uses to control the wrapping of different blocks of content:

2 Ways to Use Shortcodes to Manage Content Debt

The Elder.js docs have several other examples of how shortcodes can be used, but in general, the 2 most common used cases for shortcodes are as a placeholder for something to happen during render and a wrapper around specific blocks of content.^[2]

Shortcodes As a Placeholder

Using shortcodes as a placeholder can be as simple as a shortcode like {{!entryLevelPrice!/}} in the example above.

Another common use is to allow content teams to dynamically embed Svelte components directly within content.

As A Wrapper

Using shortcodes as a wrapper every time your content team would usually use a <div class=""> can save you from the tedious issue of needing to update all of those class names.

Elder.js' Approach to Shortcodes

When I set out to build Elder.js, I never thought I'd find a way to get a sexy shortcode implementation built into the core, but after a bunch of hacking, I'm proud of what is there.

By default, I've designed Elder.js shortcodes to have access to many of the same properties available on most hooks.

Further, Elder.js shortcodes can return a string or an object.

If you return an object, you can include arbitrary html, css, js and even head strings that will get written to the appropriate places in your html during output.

As of this writing, here are two shortcodes that fully show the api.

Results: A Real Plan for Managing Content Debt

While I doubt most Elder.js users will use shortcodes as extensively as I have on various projects, we've implemented them on ElderGuide.com to help us manage content debt.

If you are an Elder.js user, I can't encourage you enough to try and find places where shortcodes could save you a ton of time.

A useful mental model for shortcodes is to view them as a hook (covered next) that lives in your static content.

Hooks: Managing Complexity/Tech Debt

No project becomes a "ball of mud" overnight. It happens over time.

In my experience leading and being a part of dev teams, I've found when there is a clear guideline on where complexity should and shouldn't live, preventing a project from becoming a "ball of mud" gets a lot easier.

The Backstory on Hooks:

Back in 2009/2010, my partner and I managed about 10 SEO sites in 4-5 verticals.

Each site was powered by WordPress but under the hood was its own "special train wreck" as I used to say.

These projects were VERY hack-y. We didn't care about code quality, we knew shipping was all that mattered and we wanted to rank before we wasted any time making things maintainable.

As our business grew, this philosophy came back to bite us. Hard.

In no time, we were hamstrung. We knew we couldn't grow without bringing on another developer but the task just seemed impossible.

Each project had buried complexity that only we knew.

Each site was its own "ball of mud."

Around this time, I gave a WordPress theme called "Thesis" a try.

Thesis did a lot of things well, but it dramatically limited your ability to do any non-trivial customization to using hooks.

Initially, this seemed like a huge problem, but soon we saw the power and benefits of predictably knowing that all of the non-standard complexity would live in a hooks.php file.

This allowed us and our dev team to quickly jump between projects as everyone knew where the complexity was hidden.

The transformation this relatively simple change had on our business was astounding.

Elder.js' Approach to Hooks

Having seen the power of hooks, the goal with Elder.js was to expose them at all of the important parts of the build / ssr process.^[3]

To counteract some of the major debugging headaches we encountered with WordPress Hooks, Elder.js does two things:

1. Under the hood, hooks are powered by a hookInterface which is essentially a contract of what properties are props of each hook and which of those props are mutable on each hook.
2. If one of the props is mutated and isn't intended to be mutable, Elder.js will throw an error.^[4]

Results: A Predictable Location for Complexity

Having worked to migrate several properties to Elder.js, it is hard to imagine building a non-trivial site without them.

Yes, the limitations of what can be mutated where limits flexibility, but it forces you and your team to keep mutations predictable.

The end result is that we are easily able to share hooks across projects and because of a single interface for customizations, developers can shift quickly between projects without a lot of context switching.

On a practical level, here are what a couple of hooks look like:

Plugins: Bundles of Hooks To Use Across Sites

The plugin idea spawned as a 'bundle of hooks' that could be used across projects.

After a few afternoons of hacking on the idea, I decided to try and make plugins first class citizens with all of the power available to users, but with guard rails to prevent plugin developers from getting carried away.

As of this writing, there are 6 official plugins and we may release a few more in the future:

• Images Easily add and manage responsive images with your Elder.js website. (Similar to Gatsby Image. The head images on this site use this plugin.)
• Markdown An extensible markdown parser for Elder.js powered by remark.
• Critical Path CSS Quickly and easily generate and include critical path css for your Elder.js website.
• Sitemap Automatically generate the latest sitemap for your Elder.js website on build.
• Browser Reload Reload the browser when your Elder.js server restarts.
• References Easily add wikipedia style references to your content.

Elder.js' Approach to Plugins

To be honest, making plugins a first class citizen kind of happened by accident.

1. In experimenting with plugins, I was struggling with where to store data.
2. As I started to build a few plugins, I found a real need for plugins to change their behavior based on the configuration a user gave them.

While I was building a PoC for the plugin interface, I accidentally created a closure that allowed the plugin to store data between page loads.

At first, I couldn't figure out what was going on with this bug...

But when I did, I realized the power of using a closure to offer each plugin their own memory space during initialization and hook invocations.

It took a few implementations to get right, but having written 6 plugins now, I have to say the approach is incredibly powerful and fun to work with.

Basically anything that a user can do in a route.js file or via their hook.js file, a plugin can do too.

Below is the plugin as of this writing.

Results: Plugins Manage Their Own Isolated Config and Memory

Before calling v1 of Elder.js done, I wanted to write several non-trivial plugins to prove the overall model.

By far, the most challenging was the @elderjs/plugin-images as it was pretty involved, but so far I've found exactly 0 implementations that a user can do but a plugin can't.

I'm really excited to see what the Elder.js community comes up with.

Builds that Scale With Resources

Most static site generators including Gatsby, Next.js, and Sapper have painfully slow build processes on sites with over 5,000 pages. While load times are improving, waiting 5-6 hours for a Gatsby build of a 10k page site was the norm as of 2019.

In my experience, the faster the build pipeline the faster a team pushes to production. So blazing fast builds of 10-100k page sites was a focus of mine in build Elder.js. ^[5]

Which Pages To Build. A Routing Problem:

When it comes to statically exporting a site, one of the suprising problems you'll face is knowing which pages to build.

Different frameworks take different approaches to solve this problem but the solutions to this problem all stem from how a framework handles routing.

Take express for instance. A typical express route definition looks like so: /blog/:slug/. This type of routing is easy to understand and is widely used, but in an SSG conext it means your framework must crawl the entire site it is building to know all of the possible URLs and their props.

This causes a few issues:

1. This means pages without links never get built... or worse, they get built one time, then a link changes or breaks and in the next build they're missing. This is a major SEO problem.
2. Crawling is now your build bottleneck.
3. If you want complex routes like /nursing-homes/:facility_slug/, /nursing-homes/:article_slug/, and /nursing-homes/:parent_company_slug/ you must precalculate all of your possible routes ahead of time or you end up in regex hell.

Elder.js' Approach to Routing:

Solving this routing problem left me scratching my head for days. Initially, I went with the standard express-like routing, but then hit the build bottleneck I so hated. In the end I settled on two solutions:

1. Let the user define multiple entry locations for the "crawler" to start building. This had a slew of problems as you looked towards rendering in lambdas or across multiple CPUs.
2. Require "explicit routing" where you force users to explicitly define routes and their payloads instead of letting the router determine what is a valid route. This allows you to know all of the possible route variations ahead of time. It also allows them to setup the URL structure however they choose and allows Elder.js to build all of the routes (even pages without links) that need to be built.

Ultimately, I chose "explicit routing" because it had the most benefits with few downsides.

Today, I feel this decision is technically the right one, but it has been a stumbling block for some users who are used to different types of routing.

Explicit Routing in Detail

Elder.js uses the same 3 steps most routers use but instead of letting the router determine what is a valid route, it forces the user to do it ahead of time.

This has huge performance benefits in both an SSR and build context, but comes at the cost of making the user do a little more work.

The best way to understand it is to compare it to express-like routing, which most of the JS ecosystem is familiar with:

Express's Router

At a high level we:

1. Define a pattern for a URL with placeholders for params that should be extracted. eg: /blog/:slug/
2. When a request comes in, match it against known patterns extracting params. eg: req.params = {slug: 'blog-post'}
3. Pass the extracted params into whatever builds the response.

Elder.js' Explicit Router

Here is the same route using Elder.js:

1. Define how request objects are turned into urls in a permalink() function.
2. Explicitly list all possible variations of the request objects.
3. When a url is requested, match that to all known pages on the site, and pass the request on to whatever builds the response.

Explicit Routing Benefits

While Elder.js' approach to routing is a bit different, it comes with a few major benefits:

1. All of the valid URLs are known allowing for full parallelization of builds across CPUs without having to crawl a site.
2. Complete control over the URL structure. As long as there aren't duplicate URLs (you'll get an error), you're good.
3. Easily add pages with no internal links for ppc landing pages or email campaigns etc.
4. Plugins can offer functionality such as sitemap generation or internal link checking, as they can tap into all of the valid urls as well.

Results: Blazing Fast Builds That Scale

As of this writing, ElderGuide.com has about 18,700 pages.

Within 2-3 years we expect to have somewhere between 50,000 and 100,000 pages.

For ElderGuide.com, we are shooting to keep our entire build pipeline including tests, generating html, uploading to s3, cache priming, cleaning up, and setting up rollbacks under 30 minutes.

As of this writing, our build pipeline takes about 12 minutes on a beefy server or ~20 minutes on a more budget setup.

Elder.js Build Times of ElderGuide.com

• Elder.js generates 18,700 pages in ~8 minutes on a budget 4 core VM running against a budget 2 core Postgres server. (Database latency takes about ~50% of the time.)
• On a 24 core bare metal box at OVH with a local Postgres server, Elder.js generates 18,700 pages in 1.22 minutes

Other Sites

• As of this writing, one of the other projects I am invested in is building 6.5k pages in ~18 seconds.

For perspective, early versions of ElderGuide built with Sapper were taking several hours with ~5,000 pages.

Built In Perf Monitoring

One of the side effects of being obsessed with build times is that you can easily get a full print out of where Elder.js is spending its time by simply adding debug.performance: true in your elder.config.js.

This will give you a nice breakdown so you can pinpoint exactly which hook, plugin, or database call is slowing down your builds or SSR requests. Below is an example.

Deciding to Open Source Elder.js

Upon launching ElderGuide.com, I received lots of requests from the Svelte community and Hacker News to open source our SSG.

Some people even went as far as writing me emails begging me to share my solution even if I didn't open source it.

Initially, the idea of open sourcing the project felt like a huge burden.

Philosophically, I love the idea of open source... but from a business perspective, open sourcing the project had the clear disadvantage of being a huge time sink with very unknown upside.

After a few weeks of marinating on the idea, I decided that open sourcing Elder.js was only a good idea if:

1. I could commit to maintaining or facilitating for it to be maintained until the end of 2022.
2. Doing so would be a HELL YES for me.

Getting to a "HELL YES"

Initially, the decision to open source wasn't a "Hell YES", but it became one for the following reasons:

1. As a founder The main business argument for ElderGuide.com spending limited development resources, releasing and maintaining Elder.js was that, long term, it could be a great source of links to ElderGuide.com if we hosted the docs there. In many ways, this is an SEO experiment as "links regarding a JS framework" aren't really semantically related to "senior living" but hey, my background is in SEO, a link is a link. :) this alone wasn't enough of a use case to open source the project.
2. As an investor who buys/builds SEO properties, I realized that I could use Elder.js as the foundation for multiple investments. If I made it modular and highly pluggable, I could know how the sites are built, even while leading different teams across different projects. The added benefit was that if any of these investments were successful, then each would have a business case to make sure Elder.js is maintained.
3. As Founder & Investor I realized that if Elder.js did take off, it may open up the talent pool in ways that are hard to predict. Finding great programming talent and people to invest in is hard. Maybe if Elder.js gets traction, it'll be easier?
4. As a developer: Until 2018, the majority of the code I had written was hacky or front end oriented. As I move on from BroadbandNow.com, one of my goals is to truly master Javascript. By open sourcing Elder.js, I am opening up my code to further scrutiny but I am more likely to stay engaged with JS, learn things I wouldn't have otherwise, and maybe make some friends along the way.
5. As the maintainer: I realized that the scope of Elder.js was 100% in my control. If I could come up with a great system for plugins and community customizations AND limit the scope of the core Elder.js offering, then maybe Elder.js didn't need to grow into a massive codebase and instead could stay focused and lean.

With buy-in from my partner on ElderGuide.com, I went to work rewriting the core codebase and engineering the hook/plugin system Elder.js uses today.

Elder.js' Project Scope

With v1.0.0 out the door, we've nailed down Elder.js' scope statement to be:

The scope of Elder.js will be limited to building a pluggable SSG/SSR framework for Svelte where all decisions are evaluated with SEO as the first priority.

The core of Elder.js will be an SSG and basic SSR framework. All other features and use-cases should be able to be implemented by the broader Elder.js community via hooks, plugins, or as a wrapper around Elder.js.

If the core of Elder.js isn't flexible to support said use-case, then it may be worth extending Elder.js' core to support that use case.

Things To Improve:

It's hard to call your own baby ugly but if I'm candid, Elder.js does have some rough spots.

Routing

Explicit routing is definitely the right solution for the problem, but I wonder if there is a more user friendly way for Elder.js newbies. It is great for our business, but I hate that one of the most fundamental parts of Elder.js has such a learning curve.

Additionally, updating the routes in SSR mode is currently a bit hacky. We'll come up with better support in the future I'm sure.

Plugin Memory Space During Builds

I haven't encountered a problem with this yet, but because pages are generated across multiple CPUs, there is an edge case where plugins could be trying to access a shared memory state that is out of sync across processes. It is one of the trade offs of using closures for isolated memory.

The Build Implementation

As of this writing, the build implementation is pretty messy. In a perfect world, here are the improvements I'd like to see:

1. Builds should use a queue and let workers pull from the queue instead of dividing it up ahead of time.
2. In addition to spreading across processes, I think there is room to speed things up significantly by using threads within each worker.
3. The whole process needs a refactor. Initially, I was using an npm package to manage the workers, but ripped it out and haven't refactored the code because it's working and production tested.

Typescript Types

So Elder.js is my first Typescript project and to be honest, it'll probably be my last. Thus far, I've found few cases where it has really saved me time and I've probably spent more time fighting Typescript than any of the benefits it has driven.

That said, the project is young and I know that hardcore Typescript users that I respect swear by it so I'm waiting to see the light. :)

Currently, the types are a bit of a mess and I could use some direction on improving them. Now that the project is stable, this is a learning curve that I'll need to tackle at some point.

Thoughts on Open Source:

Entitlement and Generosity

Having shipped Elder.js, I have huge new respect for all of the people whose software I've used in the past.

Elder.js is still in its infancy and I've already encountered some very demanding and unforgiving users. It is amazing what some people expect of software that clearly states it is in prerelease...

... but for each user that has been difficult, I have found at least 2 people in the Svelte community to be incredibly helpful in offering critiques and ideas to improve Elder.js.

Good Docs Take Time

Great documentation is hard... but it also can be improved every day by listening to users.

Having spent the last 7 months of my life working with or on Elder.js every day, I've really struggled to communicate all that can be done with the Elder.js framework, what the best practices are, and where the pitfalls are.

Thus far, I've found 2 strategies to be very effective in helping keep the docs up to date:

1. Anytime someone hits a snag and pings me on Svelte Discord with a question, I try to respond with a few details and a link to the docs if the documentation covers it. If it isn't in the docs then I write out a detailed response and make sure the docs include details that are at least as thorough as my response to the user.
2. Hire another developer to build a site with your framework. While I know this doesn't apply in all situations, since I have another full time developer building out an investment property with Elder.js anytime he hits a snag, I make sure it is in the docs. The interesting part here is most of the snags he has hit have to do with subtleties that weren't communicated in the docs. This has been a huge blessing and I'm grateful for his effort to help improve the docs.

A Clear Scope Statement Is a Must

Writing and revisiting Elder.js' scope statement has been a huge boon. It clearly spells out which issues/bugs should be addressed and which shouldn't. It also helps make maintenance of the software feel less like I'm on a treadmill and more like I'm working towards the completion of that scope.

Without a "scope statement", I can see how it is easy for open source maintainers to burn out.

Closing

To wrap up this long write up, I want to say thank you to the Svelte Community for their support and to Filip Halas for helping write the tests that back Elder.js.