Skip to content

G
gitlab-ce
Switch branch/tag
  1. 28 Jan, 2021 1 commit
    • Lukas 'Eipi' Eipert's avatar
      Add incremental compiler to webpack dev server · 76f9be20
      Lukas 'Eipi' Eipert authored 
      In order to have quicker compilations while developing we are adding an
incremental compiler to webpack to render routes on demand.

When the developer is working on the dev server and enables the
incremental compiler with DEV_SERVER_INCREMENTAL=true, the new
functionality is enabled.

The biggest problem to solve here: How can we _know_ which entry point
to render and which not? Our current webpack integration with rails
requires the webpack manifest to have a list of all existing entry
points. So our incremental compiler takes the following approach:

Every compilation of webpack will run `generateEntries` which generates
a list of all our entry points. In that function we are able to replace
all page specific entrypoints and point them to an empty file,
unless we explicitly want them to compile.

In the webpack-dev-server itself we register a middleware which keeps
track of all the page specific bundles requested. Whenever a page
specific bundle is requested that hasn't been requested before, we add
it to the list of bundles we want compiled.

This approach allows us to dynamically change the entry points without a
need to restart webpack alltogether _and_ it works with hot module
reloading.

Rather than pointing to a blank javascript we are pointing to one which
renders an overlay to let the user know that webpack compiles the page
for the first time.

Additionally we keep a history of requested routes in `tmp/cache` in
order to keep the list of compiled entry points between sessions. In a
next iteration we can add a bit of logic and e.g. remove entry points
the developer hasn't been visiting in a week.

First results are really promising (on my machine):
 - Memory consumption when idling: 1600MB => 340MB
 - Max memory: ~2200MB => ~1000MB
 - Initial Compilation time: 58s => 15s
 - Recompile afer a file change: 13s => 3s
 - Visiting a new page that hasn't been visited before, it takes about
   four seconds to reload the page, seven seconds to completely load the
   page.

Currently the technique still watches all of the source files, so
changing an unrelated file will trigger a recompilation. This is however
a minor caveat and the same behavior that we currently have, maybe we
can optimize in the future.
      76f9be20
  3. 26 Jan, 2021 39 commits
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7