koa和egg项目webpack内存编译和热更新实现

系统结
• 阅读 14567

背景

在用Node.js+Webpack构建的方式进行开发时, 我们希望能实现修改代码能实时刷新页面UI的效果.
这个特性webpack本身是支持的, 而且基于koa也有现成的koa-webpack-hot-middleware 和 koa-webpack-dev-middleware 封装好的组件支持.
不过这里如果需要支持Node.js服务器端修改代码自动重启webpack自动编译功能就需要cluster来实现.

今天这里要讲的是如何在koa和egg应用实现Node.js应用重启中的webpack热更新功能. 要实现egg项目中webpack友好的开发体验, 需要解决如下三个问题.

问题

  • 如何解决Node.js服务器端代码修改应用重启避免webpack重新编译.

  • 如何访问js,css,image等静态资源.

  • 如何处理本地开发webpack热更新内存存储读取和线上应用本机文件读取逻辑分离.

基于koa的webpack编译和热更新实现

在koa项目中, 通过koa-webpack-dev-middleware和koa-webpack-hot-middleware可以实现webpack编译内存存储和热更新功能, 代码如下:

const compiler = webpack(webpackConfig);
const devMiddleware = require('koa-webpack-dev-middleware')(compiler, options);
const hotMiddleware = require('koa-webpack-hot-middleware')(compiler, options);
app.use(devMiddleware);
app.use(hotMiddleware);

如果按照上面实现, 可以满足修改修改客户端代码实现webpack自动变编译和UI界面热更新的功能, 但如果是修改Node.js服务器端代码重启后就会发现webpack会重新编译,
这不是我们要的效果.原因是因为middleware是依赖app的生命周期, 当app销毁时, 对应webpack compiler实例也就没有了, 重启时会重新执行middleware初始化工作.
针对这个我们可以通过Node.js cluster实现, 大概思路如下:

通过cluster worker 启动App应用

if (cluster.isWorker) {
  const koa = require('koa');
    app.listen(8888, () =>{
        app.logger.info('The server is running on port: 9999');
    });
}

通过cluster master 启动一个新的koa应用, 并启动 webpack 编译.

const cluster = require('cluster');
const chokidar = require('chokidar');

if (cluster.isMaster) {
  const koa = require('koa');
  const app = koa();
  const compiler = webpack([clientWebpackConfig,serverWebpackConfig]);
  const devMiddleware = require('koa-webpack-dev-middleware')(compiler);
  const hotMiddleware = require('koa-webpack-hot-middleware')(compiler);
  app.use(devMiddleware);
  app.use(hotMiddleware);

  let worker = cluster.fork();
  chokidar.watch(config.dir, config.options).on('change', path =>{
    console.log(`${path} changed`);
    worker.kill();
    worker = cluster.fork().on('listening', (address) =>{
      console.log(`[master] listening: worker ${worker.id}, pid:${worker.process.pid} ,Address:${address.address } :${address.port}`);
    });
  });
}

通过chokidar库监听文件夹的文件修改, 然后重启worker, 这样就能保证webpack compiler实例不被销毁.

const watchConfig = {
        dir: [ 'controller', 'middleware', 'lib', 'model', 'app.js', 'index.js' ],
        options: {}
    };
    let worker = cluster.fork();
    chokidar.watch(watchConfig.dir, watchConfig.options).on('change', path =>{
        console.log(`${path} changed`);
        worker && worker.kill();
        worker = cluster.fork().on('listening', (address) =>{
            console.log(`[master] listening: worker ${worker.id}, pid:${worker.process.pid} ,Address:${address.address } :${address.port}`);
        });
});

worker 通过process.send 向 master 发现消息, process.on 监听 master返回的消息

  • 首先我们看看本地文件读取的实现, 在context上面挂载readFile方法, 进行view render时, 调用app.context.readFile 方法.

app.context.readFile = function(fileName){
  const filePath = path.join(config.baseDir, config.staticDir, fileName);
  return new Promise((resolve, reject) =>{
    fs.readFile(filePath, CHARSET, function(err, data){
      if (err) {
        reject(err);
      } else {
        resolve(data);
      }
    });
  });
};

  • 通过覆写worker app.context.readFile 方法, 这样进行本地开发时,开启该插件就可以无缝的从webpack编译内存系统里面读取文件

app.context.readFile = (fileName) =>{
  return new Promise((resolve, reject) =>{
    process.send({ action: Constant.EVENT_FILE_READ, fileName });
    process.on(Constant.EVENT_MESSAGE, (msg) =>{
      resolve(msg.content);
    });
  });
};

master 通过监听worker发过来的消息, 获取webpack编译进度和读取webpack compiler内存系统文件内容

cluster.on(Constant.EVENT_MESSAGE, (worker, msg) =>{
        switch (msg.action) {
            case Constant.EVENT_WEBPACK_BUILD_STATE: {
                const data = {
                    action: Constant.EVENT_WEBPACK_BUILD_STATE,
                    state: app.webpack_client_build_success && app.webpack_server_build_success
                };
                worker.send(data);
                break;
            }
            case Constant.EVENT_FILE_READ: {
                const fileName = msg.fileName;
                try {
                    const compiler = app.compiler;
                    const filePath = path.join(compiler.outputPath, fileName);
                    const content = app.compiler.outputFileSystem.readFileSync(filePath).toString(Constant.CHARSET);
                    worker.send({ fileName, content });
                } catch (e) {
                    console.log(`read file ${fileName} error`, e.toString());
                }
                break;
            }
            default:
                break;
        }
});

基于egg的webpack编译和热更新实现

通过上面koa的实现思路, egg实现就更简单了. 因为egg已经内置了worker和agent通信机制以及自动重启功能.

app.js (worker) 通过 检测webpack 编译进度

  • 通过app.messenger.sendToAgent 向agent发送消息

  • 通过app.messenger.on 监听agent发送过来的消息

app.use(function* (next) {
  if (app.webpack_server_build_success && app.webpack_client_build_success) {
    yield* next;
  } else {
    const serverData = yield new Promise(resolve => {
      this.app.messenger.sendToAgent(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, {
        webpackBuildCheck: true,
      });
      this.app.messenger.on(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, data => {
        resolve(data);
      });
    });
    app.webpack_server_build_success = serverData.state;

    const clientData = yield new Promise(resolve => {
      this.app.messenger.sendToAgent(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, {
        webpackBuildCheck: true,
      });
      this.app.messenger.on(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, data => {
        resolve(data);
      });
    });

    app.webpack_client_build_success = clientData.state;

    if (!(app.webpack_server_build_success && app.webpack_client_build_success)) {
      if (app.webpack_loading_text) {
        this.body = app.webpack_loading_text;
      } else {
        const filePath = path.resolve(__dirname, './lib/template/loading.html');
        this.body = app.webpack_loading_text = fs.readFileSync(filePath, 'utf8');
      }
    } else {
      yield* next;
    }
  }
});

app.messenger.on(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, data => {
  app.webpack_server_build_success = data.state;
});

app.messenger.on(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, data => {
  app.webpack_client_build_success = data.state;
});

agent.js 启动koa实例和webpack编译流程

这里client和server编译单独启动koa实例, 而不是一个是因为在测试时发现编译会导致热更新冲突.

  • 启动webpack client 编译模式, 负责编译browser运行文件(js,css,image等静态资源)

'use strict';

const webpack = require('webpack');
const koa = require('koa');
const cors = require('kcors');
const app = koa();
app.use(cors());
const Constant = require('./constant');
const Utils = require('./utils');

module.exports = agent => {

  const config = agent.config.webpack;
  const webpackConfig = config.clientConfig;
  const compiler = webpack([webpackConfig]);

  compiler.plugin('done', compilation => {
    // Child extract-text-webpack-plugin:
    compilation.stats.forEach(stat => {
      stat.compilation.children = stat.compilation.children.filter(child => {
        return child.name !== 'extract-text-webpack-plugin';
      });
    });
    agent.messenger.sendToApp(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, { state: true });
    agent.webpack_client_build_success = true;
  });

  const devMiddleware = require('koa-webpack-dev-middleware')(compiler, {
    publicPath: webpackConfig.output.publicPath,
    stats: {
      colors: true,
      children: true,
      modules: false,
      chunks: false,
      chunkModules: false,
    },
    watchOptions: {
      ignored: /node_modules/,
    },
  });

  const hotMiddleware = require('koa-webpack-hot-middleware')(compiler, {
    log: false,
    reload: true,
  });

  app.use(devMiddleware);
  app.use(hotMiddleware);

  app.listen(config.port, err => {
    if (!err) {
      agent.logger.info(`start webpack client build service: http://127.0.0.1:${config.port}`);
    }
  });

  agent.messenger.on(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, () => {
    agent.messenger.sendToApp(Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, { state: agent.webpack_client_build_success });
  });

  agent.messenger.on(Constant.EVENT_WEBPACK_READ_CLIENT_FILE_MEMORY, data => {
    const fileContent = Utils.readWebpackMemoryFile(compiler, data.filePath);
    if (fileContent) {
      agent.messenger.sendToApp(Constant.EVENT_WEBPACK_READ_CLIENT_FILE_MEMORY_CONTENT, {
        fileContent,
      });
    } else {
      agent.logger.error(`webpack client memory file[${data.filePath}] not exist!`);
      agent.messenger.sendToApp(Constant.EVENT_WEBPACK_READ_CLIENT_FILE_MEMORY_CONTENT, {
        fileContent: '',
      });
    }
  });
};

  • 启动webpack server 编译模式, 负责编译服务器端Node运行文件

'use strict';

const webpack = require('webpack');
const koa = require('koa');
const cors = require('kcors');
const app = koa();
app.use(cors());
const Constant = require('./constant');
const Utils = require('./utils');

module.exports = agent => {
  const config = agent.config.webpack;
  const serverWebpackConfig = config.serverConfig;
  const compiler = webpack([serverWebpackConfig]);

  compiler.plugin('done', () => {
    agent.messenger.sendToApp(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, { state: true });
    agent.webpack_server_build_success = true;
  });

  const devMiddleware = require('koa-webpack-dev-middleware')(compiler, {
    publicPath: serverWebpackConfig.output.publicPath,
    stats: {
      colors: true,
      children: true,
      modules: false,
      chunks: false,
      chunkModules: false,
    },
    watchOptions: {
      ignored: /node_modules/,
    },
  });

  app.use(devMiddleware);

  app.listen(config.port + 1, err => {
    if (!err) {
      agent.logger.info(`start webpack server build service: http://127.0.0.1:${config.port + 1}`);
    }
  });

  agent.messenger.on(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, () => {
    agent.messenger.sendToApp(Constant.EVENT_WEBPACK_SERVER_BUILD_STATE, { state: agent.webpack_server_build_success });
  });

  agent.messenger.on(Constant.EVENT_WEBPACK_READ_SERVER_FILE_MEMORY, data => {
    const fileContent = Utils.readWebpackMemoryFile(compiler, data.filePath);
    if (fileContent) {
      agent.messenger.sendToApp(Constant.EVENT_WEBPACK_READ_SERVER_FILE_MEMORY_CONTENT, {
        fileContent,
      });
    } else {
      // agent.logger.error(`webpack server memory file[${data.filePath}] not exist!`);
      agent.messenger.sendToApp(Constant.EVENT_WEBPACK_READ_SERVER_FILE_MEMORY_CONTENT, {
        fileContent: '',
      });
    }
  });
};

  • 挂载 webpack 内存读取实例到app上面, 方便业务扩展实现, 代码如下:

我们通过worker向agent发送消息, 就可以从webpack内存获取文件内容, 下面简单封装一下:

class FileSystem {

  constructor(app) {
    this.app = app;
  }

  readClientFile(filePath, fileName) {
    return new Promise(resolve => {
      this.app.messenger.sendToAgent(Constant.EVENT_WEBPACK_READ_CLIENT_FILE_MEMORY, {
        filePath,
        fileName,
      });
      this.app.messenger.on(Constant.EVENT_WEBPACK_READ_CLIENT_FILE_MEMORY_CONTENT, data => {
        resolve(data.fileContent);
      });
    });
  }

  readServerFile(filePath, fileName) {
    return new Promise(resolve => {
      this.app.messenger.sendToAgent(Constant.EVENT_WEBPACK_READ_SERVER_FILE_MEMORY, {
        filePath,
        fileName,
      });
      this.app.messenger.on(Constant.EVENT_WEBPACK_READ_SERVER_FILE_MEMORY_CONTENT, data => {
        resolve(data.fileContent);
      });
    });
  }
}

在app/extend/application.js 挂载webpack实例

const WEBPACK = Symbol('Application#webpack');
module.exports = {
  get webpack() {
    if (!this[WEBPACK]) {
      this[WEBPACK] = new FileSystem(this);
    }
    return this[WEBPACK];
  },
};

本地开发webpack热更新内存存储读取和线上应用文件读取逻辑分离

基于上面编译流程实现和webpack实例, 我们很容易实现koa方式的本地开发和线上运行代码分离. 下面我们就以vue 服务器渲染render实现为例:

在egg-view插件开发规范中,我们会在ctx上面挂载render方法, render方法会根据文件名进行文件读取, 模板与数据编译, 从而实现模板的渲染.如下就是controller的调用方式:

exports.index = function* (ctx) {
  yield ctx.render('index/index.js', Model.getPage(1, 10));
};

其中最关键的一步是根据文件名进行文件读取, 只要view插件设计时, 把文件读取的方法暴露出来(例如上面的koa的readFile),就可以实现本地开发webpack热更新内存存储读取.

  • vue view engine设计实现:

const Engine = require('../../lib/engine');
const VUE_ENGINE = Symbol('Application#vue');

module.exports = {

  get vue() {
    if (!this[VUE_ENGINE]) {
      this[VUE_ENGINE] = new Engine(this);
    }
    return this[VUE_ENGINE];
  },
};
class Engine {
  constructor(app) {
    this.app = app;
    this.config = app.config.vue;
    this.cache = LRU(this.config.cache);
    this.fileLoader = new FileLoader(app, this.cache);
    this.renderer = vueServerRenderer.createRenderer();
    this.renderOptions = Object.assign({
      cache: this.cache,
    }, this.config.renderOptions);
  }

  createBundleRenderer(code, renderOptions) {
    return vueServerRenderer.createBundleRenderer(code, Object.assign({}, this.renderOptions, renderOptions));
  }

  * readFile(name) {
    return yield this.fileLoader.load(name);
  }

  render(code, data = {}, options = {}) {
    return new Promise((resolve, reject) => {
      this.createBundleRenderer(code, options.renderOptions).renderToString(data, (err, html) => {
        if (err) {
          reject(err);
        } else {
          resolve(html);
        }
      });
    });
  }
}

  • ctx.render 方法

class View {
  constructor(ctx) {
    this.app = ctx.app;
  }

  * render(name, locals, options = {}) {
    // 我们通过覆写app.vue.readFile即可改变文件读取逻辑
    const code = yield this.app.vue.readFile(name);
    return this.app.vue.render(code, { state: locals }, options);
  }

  renderString(tpl, locals) {
    return this.app.vue.renderString(tpl, locals);
  }
}

module.exports = View;

服务器view渲染插件实现 egg-view-vue

  • 通过webpack实例覆写app.vue.readFile 改变从webpack内存读取文件内容.

if (app.vue) {
  app.vue.readFile = fileName => {
    const filePath = path.isAbsolute(fileName) ? fileName : path.join(app.config.view.root[0], fileName);
    if (/\.js$/.test(fileName)) {
      return app.webpack.fileSystem.readServerFile(filePath, fileName);
    }
    return app.webpack.fileSystem.readClientFile(filePath, fileName);
  };
}

app.messenger.on(app.webpack.Constant.EVENT_WEBPACK_CLIENT_BUILD_STATE, data => {
  if (data.state) {
    const filepath = app.config.webpackvue.build.manifest;
    const promise = app.webpack.fileSystem.readClientFile(filepath);
    promise.then(content => {
      fs.writeFileSync(filepath, content, 'utf8');
    });
  }
});

webpack + vue 编译插件实现 egg-webpack-vue

egg+webpack+vue工程解决方案

点赞
收藏
评论区
推荐文章
blmius blmius
•
3年前
MySQL:[Err] 1292 - Incorrect datetime value: ‘0000-00-00 00:00:00‘ for column ‘CREATE_TIME‘ at row 1
文章目录问题用navicat导入数据时，报错：原因这是因为当前的MySQL不支持datetime为0的情况。解决修改sql\mode:sql\mode:SQLMode定义了MySQL应支持的SQL语法、数据校验等，这样可以更容易地在不同的环境中使用MySQL。全局s
5338
0
0
Wesley13 Wesley13
•
3年前
VBox 启动虚拟机失败
在Vbox（5.0.8版本）启动Ubuntu的虚拟机时，遇到错误信息：NtCreateFile(\\Device\\VBoxDrvStub)failed:0xc000000034STATUS\_OBJECT\_NAME\_NOT\_FOUND(0retries) (rc101)Makesurethekern
1724
0
0
Wesley13 Wesley13
•
3年前
FLV文件格式
1.        FLV文件对齐方式FLV文件以大端对齐方式存放多字节整型。如存放数字无符号16位的数字300（0x012C），那么在FLV文件中存放的顺序是：|0x01|0x2C|。如果是无符号32位数字300（0x0000012C），那么在FLV文件中的存放顺序是：|0x00|0x00|0x00|0x01|0x2C。2.  
1644
0
0
Stella981 Stella981
•
3年前
SpringBoot整合Redis乱码原因及解决方案
问题描述：springboot使用springdataredis存储数据时乱码rediskey/value出现\\xAC\\xED\\x00\\x05t\\x00\\x05问题分析：查看RedisTemplate类!(https://oscimg.oschina.net/oscnet/0a85565fa
1185
0
0
Wesley13 Wesley13
•
3年前
mysql设置时区
mysql设置时区mysql\_query("SETtime\_zone'8:00'")ordie('时区设置失败，请联系管理员！');中国在东8区所以加8方法二：selectcount(user\_id)asdevice,CONVERT\_TZ(FROM\_UNIXTIME(reg\_time),'08:00','0
1796
0
0
Wesley13 Wesley13
•
3年前
PHP创建多级树型结构
<!lang:php<?php$areaarray(array('id'1,'pid'0,'name''中国'),array('id'5,'pid'0,'name''美国'),array('id'2,'pid'1,'name''吉林'),array('id'4,'pid'2,'n
1660
0
0
Easter79 Easter79
•
3年前
SpringBoot整合Redis乱码原因及解决方案
问题描述：springboot使用springdataredis存储数据时乱码rediskey/value出现\\xAC\\xED\\x00\\x05t\\x00\\x05问题分析：查看RedisTemplate类!(https://oscimg.oschina.net/oscnet/0a85565fa
1119
0
0
Wesley13 Wesley13
•
3年前
Java日期时间API系列36
　　十二时辰，古代劳动人民把一昼夜划分成十二个时段，每一个时段叫一个时辰。二十四小时和十二时辰对照表：时辰时间24时制子时深夜11：00凌晨01：0023：0001：00丑时上午01：00上午03：0001：0003：00寅时上午03：00上午0
1830
0
0
Wesley13 Wesley13
•
3年前
MySQL部分从库上面因为大量的临时表tmp_table造成慢查询
背景描述Time:20190124T00:08:14.70572408:00User@Host:@Id:Schema:sentrymetaLast_errno:0Killed:0Query_time:0.315758Lock_
2081
0
0
Python进阶者 Python进阶者
•
1年前
Excel中这日期老是出来00：00：00，怎么用Pandas把这个去除
大家好，我是皮皮。一、前言前几天在Python白银交流群【上海新年人】问了一个Pandas数据筛选的问题。问题如下：这日期老是出来00：00：00，怎么把这个去除。二、实现过程后来【论草莓如何成为冻干莓】给了一个思路和代码如下：pd.toexcel之前把这
925
0
0
美凌格栋栋酱 美凌格栋栋酱
•
5个月前
Oracle 分组与拼接字符串同时使用
SELECTT.,ROWNUMIDFROM(SELECTT.EMPLID,T.NAME,T.BU,T.REALDEPART,T.FORMATDATE,SUM(T.S0)S0,MAX(UPDATETIME)CREATETIME,LISTAGG(TOCHAR(
2009
0
0
系统结
系统结
Lv1
去年花里逢君别，今日花开已一年。
文章
8
粉丝
0
获赞
0
热门文章
使用SharePoint App-Only获得访问权限
service worker 是什么？看这篇就够了
探秘 Spring 的 PropertyEditor
2018 所思所感
一位小厂前端 Leader 的年度总结