Recently I read an email talking about Google's "Core Web Vitals", and I decided to check what they were for Bots on Discord.
Not very good, but that's not unexpected. I only have one server and Bots on Discord is quite a complex site. In addition to all the stylesheets and libraries, the site has trackers and ads. Because of all of this I've always seen loading speed as an impossible battle. It's nice to have a website that loads instantly, but it's not realistic.
However, I did start to do a little chatting and thinking about ways I could try to reduce this. There's nothing I can do about how ads and the third-party code to load them, so that's out of the question. Google Analytics was also staying since it's a useful tool to have, but there was another tracker on the site. Yandex Metrica has been on the site for a while, mainly since it allows me to replay sessions and see issues people have in using the site. I never really look at it, but it was included with the ad code, so it didn't really matter... or so I thought. Either it never was included or it got removed, because I was the only one including the file. I decided to remove Metrica due to my lack of use of it, and by the time these screenshots were taken it had already been removed for around two weeks.
Before I get into the main content I'll make a place for these measurements taken before any of the following changes. These are Google's estimated measurements of the Web Vitals for the home page and the bot page for Mirai as of the time I started writing this:
Home page
Bot page
Moving away from loading time issues caused by the world being big and ads, what could I change in the actual code of Bots on Discord to reduce the size of files? Bots on Discord is already processed and bundled with Webpack, making it as small as possible without human input. Luckily there's a plugin that analyzes this bundle so I can easily see what makes up each js file that gets built. Here is the result of running nuxt build with build.analyze = true:
Relative sizes are determined using the gzip size of the file or folder.
With this helpful graphic generated we can very easily see what makes up most of the download size for pages. Here are the things that stand out:
For now I'll be working on three of these: css processing, moment, and highlight.js. There are possibilities for others which I will discuss at the end.
Bots on Discord allows users to define their own CSS for certain elements. There are three packages that make this easily possible: PostCSS, PostCSS-Scopify, and Autoprefixer. PostCSS allows me to apply post-processing to CSS. Scopify is a plugin that scopes the CSS to a specific selector (the markdown container). Autoprefixer isn't required, but it adds prefixes to make the CSS work across more browsers.
Unfortunately postcss-scopify adds unnecessary weight by requiring an older version of PostCSS for itself. The code itself is very small, but migrating it to the new plugin API is not something I'm willing to do. Autoprefixer is larger than both with a size of 89.2KB. In total these CSS post-processing libraries add around 130KB to the size of any page using markdown. That's a lot for something that only needs to run one time, or never if there is no user CSS. These tools are really only intended to be used during build anyways.
I thought about how I could remove them from pages. On lists and the markdown guide there's no need for them at all, and on bot pages it only might be needed when first loading. I can't just completely remove it from the site though, since editing pages still need to process CSS as the user changes it, and I don't want to make an API route for that.
The first thing to do was decouple css processing from the markdown component, since it isn't needed everywhere markdown is. I moved the CSS code to an importable processUserCss function. The processing would be run outside of the markdown component and then passed to it for use. For bot pages I added an option to the API to replace the bot.customizations.styles field server-side, so the CSS arrives to the client already ready to use.
Just this simple change removed 136KB of JavaScript from bot pages, the markdown guide, and lists. The bot edit and submit pages still have them, but there's no reason to worry about a fraction of a second on loading those pages.
Next up on the list is Moment.js. Luckily replacing moment is a common occurrence, so You Don't Need has put together a short comparison / guide to moment. The alternatives to moment were Luxon, date-fns, and dayjs. Replacing moment with the built-in Date was not an option because Bots on Discord uses moment's format and calendar methods a lot, which don't exist with Node's Date. Luxon is much larger than the other libraries and the API differs quite a bit from moment. date-fns also differs from moment's API and and has a larger size in total. dayjs offers an API equal to moment's with a tiny size, making it an easy choice. To show you how easy it was to migrate from moment to dayjs, here's the code:
import moment from 'moment';
moment(record.time).format('LLLL');
moment(this.bot.dates.lastSubmitted).calendar();
moment(bot.dates.lastSubmitted).add(2, 'weeks').isBefore();
moment().subtract(days, 'days').toISOString();
moment.utc().subtract(2, 'weeks').toDate();
moment().add(3, 'months').valueOf();
moment.utc().startOf('day').toDate();
moment.utc().isSame(bot.invites[bot.invites.length - 1].date, 'day');
moment().endOf('hour').diff(moment());
import dayjs from 'dayjs';
import localizedFormat from 'dayjs/plugin/localizedFormat';
import calendar from 'dayjs/plugin/calendar';
import utcPlugin = from 'dayjs/plugin/utc';
dayjs.extend(localizedFormat);
dayjs.extend(calendar);
dayjs.extend(utcPlugin);
dayjs(record.time).format('LLLL');
dayjs(this.bot.dates.lastSubmitted).calendar();
dayjs(bot.dates.lastSubmitted).add(2, 'weeks').isBefore(dayjs());
dayjs().subtract(days, 'days').toISOString();
dayjs().utc().subtract(2, 'weeks').toDate();
dayjs().add(3, 'months').valueOf();
dayjs().utc().startOf('day').toDate();
dayjs().utc().isSame(bot.invites[bot.invites.length - 1].date, 'day');
dayjs().endOf('hour').diff(dayjs());
This small change removed a ton of data from the site without changing the functionality at all. Look how much smaller dayjs is!
moment: 252.56 KB minified, 69.89 KB gzipped.
dayjs: 6.15 KB minified, 2.7 KB gzipped.
Highlight.js is by-far the largest part of the bundle. Like the CSS processing code above, it is sent to any page that has markdown: bot pages, edit and submit pages for bots, some info pages, and lists. Being able to reduce this would greatly reduce the size of those pages.
I was already aware of one alternative to highlight.js: Prismjs. Prism weighs in at 6.3KB minified and gzipped. It supports many languages, and appeared like an easy replacement for hljs. I was also told about shiki, which weights in at 37.6KB. Shiki currently supports most languages, but not nearly as many as hljs and prismjs. I wanted to avoid reducing support if possible, so I decided against it.
I decided to try using prismjs. There's one important limitation of prismjs that its website made me aware of going in: It fails to highlight correctly in certain cases. It uses regex to highlight which makes it smaller, but adds that limitation. I was ok with that in exchange for the massively smaller size though.
Once I began adding prismjs to the code more problems presented themselves. First of all, the documentation for usage in Node amounted to a section of the website the height of your screen. According to the website there was no way to load all languages without manually listing every language supported. You also can't use the provided loadLanguages() function if you don't want to include every language and plugin in your bundle. I would have to add a webpack plugin and list every language there. I tried using some code I found online to programmatically load everything in the languages folder, but that doesn't work with Webpack either. After a bit more googling and a lot of looking at source code, I decided that I didn't think prismjs was made very well, and that it certainly wasn't designed with Nodejs in mind. I tabled the task of replacing hljs.
Fast-forward a bit and my mind was once again on hljs. I thought about it and thought "Do I really even need a lot of languages supported for Bots on Discord?" I exported the bots collection to a json file and searched for all the codeblock languages. The results were surprising:
ini: 2
sh: 3
ts: 4
python: 2
bash: 1
js: 10
css: 31
yaml: 2
diff: 1
fix: 1
markdown: 1
Those are the only languages being highlighted on bot pages. I looked through the supported languages in hljs and added 42 more that didn't take up a lot of space. I manually imported these languages one-by-one (although I'd rather not, Webpack really requires it). This allowed me to keep my existing code while removing definition files like mathematica.js, taking up 29.5KB by itself. In the end this reduced the hljs size from 256KB gzipped to 48KB. I'm interested in looking more into shiki in the future, but for now it would only be saving 10KB.
After making these three changes this is the resulting bundle:
The total size of the bundle was reduced from 760.2KB to 494.7KB, with no noticeable change in functionality!
I hope this can help some of you reduce the size of your distributed files. If you have time try inspecting your own website's bundle and see where you could remove things!
I could reduce the size of the markdown-it-emoji plugin by using a short list of frequently-used shortcuts, instead of the full list.
I could also reduce the size of the Buefy distributable by importing only what is used. I currently use nuxt-buefy which adds everything to vue for me, but I can selectively import the parts I use with the following code. I'll likely include this in the next update.
import { ConfigProgrammatic, Table, Input } from 'buefy';
Vue.use(Table);
Vue.use(Input);
ConfigProgrammatic.setOptions({
defaultIconPack: 'fa',
defaultModalScroll: 'keep'
});