Optimizing š¦ Webpack-built bundles: a case study
hive-169321Ā·@instytutfiĀ·
0.000 HBDOptimizing š¦ Webpack-built bundles: a case study
__This is the story on how I managed to optimize Webpack build process in many ways, resulting in generated bundle size thatās 12 times smaller than what I started with!__ The front-end ecosystem is growing continuously, cool new tools and packages are released constantly. While this is great, it also mean itās harder and harder to be up-to-date with everything. Recently I spent a few days working on improving performance of an web app. Itās a rather large app built using Django & Django REST Framework to create APIs and a few āļø React apps that make use of these APIs. While my performance optimizations were done on both back-end and front-end, in this article Iāll recap the steps I took to improve performance of the front-end part built with React and at the end Iāll draw important conclusions from it. ## The problem As I mentioned in the intro above, the main problem our users experienced was the **low performance of the system**. Unsurprisingly, the most complaints that we got were from users using mobile phones and mobile Internet connection. They would complain that page load times were often times uncomfortably long and that there was a noticeable delay when navigating between different views. ## The investigation My immediate bet was that our Webpack-built bundles are too heavy and it takes a lot of time for browser to load them from the server. After these bundles are loaded, only then React is booted and starts rendering the webpage ā which also takes time. This means that user has to wait a long time before anything appears on the page. This would be most noticeable on initial loads (when there are no cached bundles yet) and on slower network connections (on a mobile Internet connection). Okay, so thatās the suspicion. Now how do I confirm it? ### Webpack Bundle Analyzer Meet the awesome [š Webpack Bundle Analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer). Itās a command-line tool that allows developers to visualize size of Webpack output files with an interactive zoomable treemap. Itās great for checking what makes up the size of the bundles, especially for such a visual person that I am. This was a natural starting point for me. I quickly added Bundle Analyzer to the dependencies: $ yarn add -D webpack-bundle-analyzer Once the analyzer finished installing, I ran profiling command. It generates stats.json file that stores information about the bundles that Webpack builds. $ npx webpack --profile --json > stats.json It took a moment and once the file was generated I was ready to dive into the generated statistics: $ npx webpack-bundle-analyzer stats.json ./dist First argument is the stats.json file that was just generated, and the second one is a path to the folder where the built files are outputted by Webpack. This is the dist folder by default.   ### Analyzing the data As you can see above, the front-end that needs optimization consists of few separate React apps (**chunks**). Each of them is mounted at a different URL endpoint. Before we start digesting the graph, letās look at the file sizes⦠All bundles weight **almost 20 MB**! Thatās a lot. And the average 2-3 MB for a single app is also way too much ā thatās how much the browser will have to load before it even starts rendering anything. React suggests that chunks do excess 250KB. But why do these chunks even weight so much? We can find out by investigating composition of these files. These are the problems that arise from this investigation and that I will try to handle in this article: 1. Firstly, the **bundled code is not minified** for production! 2. Around 80% of each app is `node_modules` dependencies. 3. The commons package that I mentioned at the start **is not rendered into a separate chunk** and so its copy is bundled with each app. 4. Some libraries can be easily **replaced with lightweight alternatives**, other can be removed altogether. 5. One of the apps is still bundled, even though **itās no longer used**. Of course, itās possible and even wise to go on and try to find more issues at some point, but letās not go over the top with too much at once ā letās see how much can be optimized by dealing with just these five problems! ## Step 1: Working with Webpack First step that I took was to update Webpack configuration and ensure that files are properly minimized for production. While development code shouldnāt be minified to be easier to š debug, **minimizing production code is a standard procedure** that allows to lower the size of generated packages by up to few dozen percent. Unfortunately, the project was still using Webpack 3. Upgrading it to version 4 took a while and I had to replace few Webpack plugins for Webpack 4-compatible versions. But with that out of the way, I was ready. I wanted to update to Webpack 4, because from this version itās no longer necessary to set up minification manually if you make use of the [**mode configuration option**](https://webpack.js.org/configuration/mode/). This was also applicable in my case, so simply adding --mode=production to the already existing build command did the trick! š ### A side note: npm audit If youāre playing with dependencies anyway, remember to use npm audit as well! This command will report you dependencies that have security vulnerabilities in them. npm audit fix will try to automatically update these packages to newer minor or patch versions that have these vulnerabilities fixed. This command is generally safe to use, because it will not do any major updates, and minor updates shouldnāt include breaking changes anyway ā but you can never be sure. In my case it managed to automatically fix over 150 serious vulnerabilities and did not break anything! š That was a long time since the last time anyone updated something, I guess⦠š¤·āā ## Step 2: Replacing and removing dependencies With the first step completed, we can now go into the code itself. Right away itās easy to spot few dependencies that can be removed or replaced with better alternatives. ### Unused app One of the app bundles was a no longer used , yet it somehow remained in the code base. I started looking through the rest of the apps and quickly discovered, that there were maybe 2 or 3 files that imported something from this unused appā few simple utility functions. **It makes no sense to keep so much legacy code just because of that**. Luckily, it was really simple to move these utilities to common module and replace imports so that they point to the new location. At this point this unnecessary chunk could be safely removed without consequences. š ### jQuery The other thing that draw my attention right away was this huge word repeating⦠Yes, Iām looking at you, jQuery! Unfortunately, simply removing jQuery was no option ā there was a lot of code that made have use of it and rewriting it would take days if not more. I started thinking about alternative solutions⦠I opened the app in the browser and started analyzing the code server by the server⦠I quickly discovered, that pages that served these React apps already had jQuery included using good old `<script>` tag! **jQuery works by adding itself as a property to the object window**. This means that in React apps I could remove it completely and instead load it from the window. // Old imports of jQuery in React apps: import $ from 'jquery' // I replaced them with this line: const $ = window.$ The above method will only work if you ensure that jQuery is loaded **before **the React app. Otherwise `window.$` will be undefined. This small change **saved over 100KB** on each chunk. Yay! š ### Luxon & moment.js Another thing that I spotted on the initial stats is that both [Luxon](https://moment.github.io/luxon/) and [moment.js](https://momentjs.com/) libraries are used ā both to create, manipulate and format DateTime instances. Itās **a huge mistake to have them both** at the same time. Since Luxon is a more modern and lighter replacement for moment.js (it even is created by the same people), I decided to ditch moment entirely. This required some small changes, like changing momentās .format() methods to Luxonās .toFormat(), etc. But all in all, it was relatively easy. **A side note**: You should probably **think twice before using Luxon or moment.js** anyway. These libraries are great and loaded with various features, but truth be: you might not even need them! If youāre simply trying to manipulate and display dates in pretty format, **š [Day.js](https://day.js.org/) might be just enough for you**! For more, see this comparison: [You donāt need moment.js š](https://github.com/you-dont-need/You-Dont-Need-Momentjs#brief-comparison). I later replaced Luxon with Day.js as well to save even more. ### lodash The last library that I decided to optimize was lodash. Itās a library full of different utilities delivering modularity, performance & extras. It saves a lot of time for developers, but this time it came with a price⦠As seen in the initial graph, **entire lodash library is included with each chunk** ā thatās at least few dozen KB times 6, even though only few helpers were used throughout the app. Thatās because the **lodash package does not support the [š³ tree-shaking mechanism](https://webpack.js.org/guides/tree-shaking/)** and has to be bundled in its entirety. Fortunately, thereās the lodash-es library to the rescue! Itās the same as original lodash, but repacked as set of ES2015 modules. This means that for every chunk Webpack will be able to only bundle what is in fact used in the code, and not the entire library. The replacement is as simple as changing your imports to use lodash-es instead of lodash. The rest is just the same, all the exports have the same names as previously. import _ from 'lodash-es' To achieve even better results, I combined this with two additional plugins tha allow to minimize bundles even more: * [babel-plugin-lodash](https://www.npmjs.com/package/babel-plugin-lodash) automatically cherry picks only those lodash modules that are actually used * [lodash-webpack-plugin](https://www.npmjs.com/package/lodash-webpack-plugin) works on top of the previous plugin to minimize modules even more ### Regenerating the graph After removing all of the unnecessary dependencies and replacing others with better alternatives, I generated new graph with Webpack Bundle Analyzer.   **Weāre down to just 3.39MB**āand we started with 20MB! That means itās **almost 6 times less** than what we started with š. ## Step 3: Using common chunks Weāve made great progress so far, but thereās still a huge problem. Letās look at the graph above and youāll quickly see **a lot of duplicated dependencies**. Each app requires react, react-dom, mobx and mobx-react, luxon, lodash-es and so on. This is problematic, because **userās browser has to download those shared dependencies separately for each app**. I might go to one page that will load this super-heavy 100KB dependency, and then go to another page that will load it once again. Fortunately, we have the **Webpackās [SplitChunksPlugin](https://webpack.js.org/plugins/split-chunks-plugin/)**! Itās a great tool for separating those shared dependencies into separate files, so that other chunks can reuse them instead of bundling copies of them. If youāre lucky, you can use the default behavior of the plugin. It generates a diagram of your app dependencies and automatically chooses how it can be best optimized by splitting it to common chunks. You donāt have to do anything and you get great results. In my case, however, **automatically generating chunks was not an option**. Thatās because they are created dynamically and can easily change if you modify the original appās dependencies. If you use plugins like [HtmlWebpackPlugin](https://webpack.js.org/plugins/html-webpack-plugin/) to inject the chunks, thatās not a problem. But in this case, Webpack bundles have to be added manually to templates and if they changed at any point, many other templates would have to be updates as well. Because of the reasons above, I decided to **create common chunks by hand**, and to further simplify things, I only created chunks that are **reused by all of the apps**. This took some time, as I manually analyzed the graph and wrote down the biggest shared dependencies. I came up with this list: react react-dom lodash-es dayjs mobx mobx-react axios react-ga react-tooltip react-portal Also, I decided to **create a separate chunk for common module**, so that it doesnāt get bundled with each app separately. Below is the Webpack configuration that handles creating these chunks: { ..., optimization: { splitChunks: { maxInitialRequests: Infinity, minSize: 0, cacheGroups: { vendors: { name: 'vendors', chunks: 'all', reuseExistingChunk: *true*, test: /[\\/]node_modules[\\/](react|react-dom|lodash-es|dayjs|mobx|mobx-react|axios|react-ga|react-tooltip|react-portal)[\\/]/ }, common: { name: 'common', chunks: 'all', reuseExistingChunk: *true*, test: /[\\/]apps[\\/]common[\\/]/ } } } } } As you can see, under the optimization key I added splitChunks config. I overwrite the default maxInitialRequests and minSize values so that **Webpack doesnāt try o mess up with my chunks**. Then I declared two cacheGroups: vendors for shared node_modules packages and common for the module of the same name. The most important part there is the test option, which tells Webpack how to decide on what to insert into the chunk. In my case, I just created simple Regexp lookups with the name of the packages that I wanted to be inserted into the chunk. After this, I only had to update the templates which included React apps. Previously, they would only include the app itself: <script src="dist/app_chunk.js"> **Now they also have to include common chunks, otherwise React app will not load**. Also, note the order in which we include these files. vendors are first, because code in common depends on them, and then marketplaces, which depends on both common and vendors. <script src="dist/vendors.js"> <script src="dist/common.js"> <script src="dist/app_chunk.js"> Thatās it! š I was ready to run Webpack Bundle Analyzer again to see the result. ## The result After all these fixes, the total size of all bundles dropped from almost 20MB to⦠**just 1.65MB**! This is **12 times lighter** than the initial size! š We started with node_modules taking up around 80% of each app size. **Now itās around 50%**, and for few apps itās even less.   What also matters is that we now have vendors and common chunks, so they only have to be loaded by user once. If other app has to be loaded, it can now **use cached vendors and common** instead of requesting them again from the server. ## Conclusions As shown in this article, by taking just few simple steps and spending few hours, I was able to optimize bundles created by Webpack by a lot. The overall weight of bundles **dropped from 20MB to 1.65MB**. Using chunks also allows browsers to better **leverage the caching mechanism**. This all resulted in a **huge performance gain for users** of the app. Also, I think itās worth noting that the above investigation and optimization was **not a š©āš¬ rocket science**. The problems that I have listed here were pretty easy to spot once one spends as much as **2ā5 minutes** looking at the Webpack Bundle Analyzer graphs. It also only took **few hours to implement** all the optimization that I have described. Yes, I hear your thoughts: > But why werenāt these steps taken before? Itās important to also understand **how did the project end up in this place**? As I mentioned at the start: front-end ecosystem is growing all the time, getting more and more complex. However, the team responsible for the app is mostly consisting of **back-end-focused full-stack developers**. While they are able to write JavaScript code and extend the existing React apps, they are nowhere near being front-end experts or at least hotheads. Nowadays, **the concept of full-stack developer is becoming less and less important and meaningful**. Growing complexity of front-end apps makes it very hard if even possible for a person to expertise in both back-end and front-end technologies at the same time. While itās important to have some understanding of back-end technologies when youāre a front-end developer and *vice versa*, this is not the same as being expert in both fields. **People who have great command of both the back and front-end are really rare specialists**. This is a trend thatās growing stronger in recent years and I expect this division to extend even moreā **these days teams have to be built with diversity of skills in mind**, consisting of people who specialize in various areas. <center> \* \* \* </center> This post was previously published on [Medium.com](https://medium.com/@mciszczon/optimizing-webpack-built-bundles-a-case-study-e41616afc04d) platform back in Feb 2020 under @mciszczon username. It is my intention to go back to publishing similar content, but on Hive instead of Medium.com and as @instytutfi. I decided to copy the post here as well for the visibility. Stay around for more content to come!