Build Tools

Nerd Sniping & the Weird State of Build Tools

The other morning I watched a video, which, like most Youtube videos I watch, covers a topic I'm already familiar with.

And I fully got nerd-sniped.

I spent at least an hour on the below codethis. Why? It's nothing that hasn't been done before (it's probably the most cliché coding challenge you can name).

The simple answer is because coding can feel like a puzzle, and puzzles can be fun. It's also because I had actual work that I didn't want to do, as I was still working on my first coffee of the day at the time.

Later the same day, I wrote a node script to automate some really boring stuff. It took me about an hour, and probably saved me about 2-3 hours of manual work (and hopefully more in the future).

The point being it can be really easy to waste time, but hard to know in the moment, and I find this to be particularly true when it comes to build tools.

Build tools are supposed to save development time and make your life easier. But because they're built by developers for developers, they have a bad habit of being poorly documented and over-engineered.

This makes it really easy to tell yourself that you're going to be saving time, only to run into an insurmountable roadblock or dead end. It takes a decent amount of experience to know when to quit Opens in a new window while you're trying to save yourself time.

Consider this a reminder to ask yourself regularly, "am I actually making my work easier?"

The (weird) state of build tools.

When people talk about JavaScript fatigue Opens in a new window, at least some part of that is referring to the sometimes painful process of evaluating, learning, and configuring build tools, most of which are both written in, and focussed on managing JS.

Adding to this is the fact that build tools are kind of in a weird place right now. The older generation, while still in active development and sporting a strong plugin ecosystem, is being challenged by the newer generation which are immature, buggy, incredibly fast, and built with single-page-applications like React, Vue, and Svelte in mind.

Before we get going, I think it's important to talk about some of the problems that build tools try to solve, and establish some categories for the tools that are trying to solve them.

Problems build tools solve

We've talked a fair bit about the "pipeline" - the series of automated actions that happen when you push your code: tests get run, code gets rebuilt, assets get optimized, etc.

Build tools similarly try to automate actions that a developer would otherwise have to do manually.

Here is a short list of automations that build tools can offer:

Minification

Removing unnecessary whitespace in your HTML, CSS and JS, and potentially renaming your JS variables and functions with shorter names.

Image optimization

Squashing your jpegs, pngs, gifs and svgs, and potentially converting to webp Opens in a new window.

Rewriting file paths

Changing localhost:8000/assets/images/hero-image.jpg to mydomain.com/images/hero-image.jpg when you push to production.

This is also helpful for cache busting Opens in a new window - when you've sped up your website by telling browsers "save /style.css in memory for one year", but you've made a change to /style.css, so you rename it to /style.7ca9d3db0013f3ea9ba05b9dcda5ede0.css.

Compiling code

SASS makes writing CSS easier. Typescript makes writing JS better. ES6 makes writing JavaScript simpler. But you can't deliver SASS, Typescript, or sometimes even ES6 to the user. You need a way to turn your SASS into CSS and your Typescript into JavaScript before you publish.

Bundling

We love how ES6 modules keep things simple and organized, and we love how we can grab whole libraries for our application with a few keypresses through npm. Wouldn't it be great if we could deliver that code to the user without a lot of copying and pasting?

Bundling packs together all your modules, including the production dependencies from node_modules, and delivers them to the user in a single file without you having to worry about browser support.

Serving files locally

You need to see what you're working on while you code, preferrably served as they would be in production, rather than as flat files. This means we need a dev server, which we've done in this course, but is an extra step to set up every time you work on a new project.

Hot re-loading

Tired of reloading the browser every time you write a new block of code? Hot re-loading detects changes to files, and reloads the browser for you. With the newer generation of build tools, hot module re-loading (HMR) re-loads only the files that have changed, making reloading basically instantaneous, even with very large applications.

Toolchaining

As you've probably noticed, some of the above are necessary while you're writing code, and some just need to be run when you're ready to move your code to production. Wouldn't it be cool if you could just have one command that you run for each type of work, whether it's development, testing, or building for production? And don't you want it to run as fast as possible?

So that's a list of tasks that we can automate with build tools. But most things that are called "build tools" are not made for the purpose of solving all these problems. So let's talk about some of the categories of build tools, and the popular tools that fall into those categories.

Categories of build tools: Task Runners, Toolchains & Bundlers

Task runners

Task runners are meant to let you create short, easy-to-remember commands for performing complex actions. They typically don't actually do the work of performing those actions, delegating instead to 'plugins' - sub-libraries that specialize in a particular task.

Beyond simple commands, the value offering of task running is speed. Gulp, for example, uses NodeJS "streams" when passing data from one task to the next. These "streams" mean that when, for example, you turn your SASS into CSS, and then apply browser-specific vendor prefixes, and then minify it, instead of reading the entire file every time, Gulp will send parts of the file, much like when you stream a video, and the browser starts playing before the whole video has loaded.

Toolchains

Toolchains are "next-generation" task runners, which, in addition to task run configuration (i.e. doing the stuff that task runners do), typically also include built-in features like an HTTP development server, and/or may do their own tasks like bundling JS files, and are very friendly towards Single-Page Application frameworks.

They usually have "sane defaults", meaning they're fairly easy to work with "out of the box", with minimal configuration and plugin installation for most projects.

Bundlers

Too many plugins

I've avoided talking too much about the plugins that drive these task runners and toolchains, because there's an absolute ton of them - 4204 for Gulp Opens in a new window at last count, 6250 for Grunt Opens in a new window, and the plugins themselves have plugins! "imagemin" is a popular image compression library that requires plugins for different image types, and "PostCSS" is a popular CSS-transformation library that has about 350 plugins that range from useful (minification, auto-prefixing) to very cool (removing unused CSS, generating a style guide, generating gradients as image fallbacks by analyzing an image) to a bit silly.

However, I feel we should talk about JS module bundlers. A JavaScript module bundler has one job - find all the JavaScript required to make your application work in the browser, and put it into a file together.

However, at some point (actually circa 2012 Opens in a new window), the most popular bundler, webpack, decided to include a development server, and a lot of people decided that the would use this module bundler as their build tool Opens in a new window.

This has lead to a lot of confusion, since you can use gulp to run webpack Opens in a new window, and create-react-app is built on top of webpack Opens in a new window.

I mention this because when you are configuring your development environment, and you Google "snowpack vs." or you look at the stateOfJs list of build tools Opens in a new window, you will see task runners, toolchains, and bundlers, all lumped together. Bundlers are not toolchains (except sometimes they are).

Some toolchains come with bundlers built-in (like Vite, which uses esbuild in development, and Rollup for production), and some (like Snowpack) ask you to choose a bundler as a plugin.

Let's try out a toolchain

Out of all the task runners and toolchains, I find Vite to be the least painful to start a project with, so let's try it out.

In your console, run this command:

npm init vite@latest

You'll be asked to make some choices, about what you want to call your project (call it whatever!), what framework you'd like to use (let's go with no framework, also known as 'vanilla'), and whether you'd like to use TypeScript ('ts'), which we won't use today.

Then it'll tell you a few commands to run to get started:

# change directories into your 
#   project folder
cd {your project name}
# install dependencies
npm install
# start using vite, including
#   the development server
npm run dev

As you can probably guess, at this point we can visit localhost:3000 Opens in a new window in our browser and see... something!

If we open up the code in our project folder, this should all look pretty familiar - an HTML file, a CSS file, and a JavaScript file, along with our node_modules folder and package.json.

When we look at our index.html file things look... a little different. Where's most of the text and styles that show up on the page in localhost?

Hmm, so where could those things be coming from?

That JavaScript is just a normal old innerHTML property, nothing fancy about that, but what's going on with that CSS file?

One feature Vite offers is the ability to inject CSS into a style tag on the page, rather than making an HTTP request for the CSS file. Any page that loads this JavaScript file will have that CSS injected. Go ahead and make changes to the CSS file, and watch the browser reload with those changes!

Neat, and also totally optional - you can still load CSS files the old-fashioned way (without sacrificing the hot-reloading).

Alright, so thus far we have that CSS feature, and a dev server that reloads our page whenever we change a file.

As a front-end nerd, though, I want to write some SASS.

SASS Opens in a new window, if I haven't told you already, is a "superset" of CSS (meaning all CSS is valid SASS), and a "preprocessor language" (meaning it's meant to be turned into another language - CSS in this case). It's kind of like TypeScript is with JavaScript, only Typescript is meant to make JavaScript stricter, whereas SASS is meant to make CSS faster to write. With SASS you can use functions, for loops, nesting and variables to compose your CSS. Let's get Vite to support it!

Step 1: install sass.

npm install --save-dev sass

Step 2: in any CSS files where you want to write SASS, change the file extension from `.css` to `.scss` (and update any references in your HTML or JS files).

Step 3: There is no step 3!

Hey, it works!

I mean, it works in development, but that's not how we'd want to deploy to users.

The first thing you'll notice is that a folder called "dist" (short for "distribution") is created. This is your production-ready application.

A bunch of cool stuff just happened:

• hash strings were added to our asset file names for cache busting (hashes aren't just for passwords, they're great any time you need a unique string!),
• the JavaScript function we imported, and the data from our JSON file, were added (bundled) to a single 'main.js' file,
• our SASS was compiled into a CSS file,
• all the the references in our HTML were updated to reflect our newly hashed, compiled, and bundled files,
• our CSS and JavaScript files were minified and compressed.

And all of this took about... one second.

Previewing production

Want to make sure that everything behaves as it should in production?

# This will only work after you've 
# run the `build` command
npm run preview

And that's how you build with a build tool!

Et cetera.

There's a couple things we mentioned today that build tools can do that Vite doesn't do by default (well, basically just image compression - you need to install a plugin Opens in a new window, or do it elsewhere in your pipeline, like as part of your Netlify deploy).

There's also some cool stuff built-in (like PostCSS Opens in a new window, or extended browser support through the vite config Opens in a new window) that we didn't have time for.