Tree Shaking
配置项
从官方提供的 文档可知,Rollup 主要的 tree-shaking 配置项如下:
annotations
默认值为 true,表示 rollup 会根据注解来判断是否保留某些代码。注解的格式为 @__PURE__、#__PURE__ 或 @__NO_SIDE_EFFECTS__、#__NO_SIDE_EFFECTS__,rollup 会根据注解来判断是否保留某些代码。
@__PURE__或#__PURE__的注释标记特定的函数调用或构造函数调用为无副作用。这意味着 Rollup 将执行tree-shaking优化,即移除调用,除非返回值在一些未tree-sharking优化的代码中被使用。这些注解需要紧跟在调用调用之前才能生效。以下代码将完全除屑优化,除非将该选项设置为 false,否则它将保持不变。js/*@__PURE__*/ console.log('side-effect'); class Impure { constructor() { console.log('side-effect'); } } /*@__PURE__ There may be additional text in the comment */ new Impure();@__NO_SIDE_EFFECTS__或者#__NO_SIDE_EFFECTS__的注释标记函数声明本身是无副作用的。当一个函数被标记为没有副作用时,所有对该函数的调用都将被认为是没有副作用的。下面的代码将被完全tree-sharking优化。js/*@__NO_SIDE_EFFECTS__*/ function impure() { console.log('side-effect'); } /*@__NO_SIDE_EFFECTS__*/ const impureArrowFn = () => { console.log('side-effect'); }; impure(); // <-- call will be considered as side effect free impureArrowFn(); // <-- call will be considered as side effect free
manualPureFunctions
默认值为 true,表示 rollup 会根据手动设置的纯函数来判断是否保留某些代码。
moduleSideEffects
当设置为 false 时,对于未引用的其他模块和外部依赖模块,会进行 tree-shaking。
// 输入文件
import { unused } from 'external-a';
import 'external-b';
console.log(42);// treeshake.moduleSideEffects === true 时的输出
import 'external-a';
import 'external-b';
console.log(42);// treeshake.moduleSideEffects === false 时的输出
console.log(42);若应用了其他模块的应用变量,则会扫描重新导出模块是否存在副作用的问题取决于变量的重新导出方式。
// 输入文件 a.js
import { foo } from './b.js';
import { demo } from 'demo';
console.log(foo, demo);
// 输入文件 b.js
// 直接重新导出将忽略副作用
export { foo } from './c.js';
console.log('this side-effect is ignored');
// 输入文件 c.js
// 非直接重新导出将包含副作用
import { foo } from './d.js';
foo.mutated = true;
console.log('this side-effect and the mutation are retained');
export { foo };
// 输入文件 d.js
console.log('d.js');
export const foo = 42;也就是说对于直接重新导出的模块,rollup 会忽略其副作用,而非直接重新导出的模块,rollup 会保留其副作用。这与 moduleSideEffects 的默认值为 true 是不一样的,后者会保留所有可达模块的副作用。
// treeshake.moduleSideEffects === false 时的输出
import { demo } from 'demo';
console.log('d.js');
const foo = 42;
foo.mutated = true;
console.log('this side-effect and the mutation are retained');
console.log(foo, demo);
// treeshake.moduleSideEffects === true 时的输出
import { demo } from 'demo';
console.log('d.js');
const foo = 42;
foo.mutated = true;
console.log('this side-effect and the mutation are retained');
console.log('this side-effect is ignored');
console.log(foo, demo);小结 moduleSideEffects
moduleSideEffects 的配置项很常见,代表着如果未从中导入任何内容,则是否包含此模块的导入。可以通过 resolveId、load、transform 钩子来进行指定当前模块自身是否具备副作用(默认为 true)。
moduleSideEffects为false时的特性- 会预先假设所有依赖项模块(包括依赖模块和外部依赖模块)均没有副作用。
- 若没有使用依赖项模块引入的变量,那么不会保留依赖项模块。
- 若使用了依赖项模块引入的变量,那么会保留依赖项模块中的副作用语句。但此时需要注意对于重导入的模块副作用的处理,若重导入模块为直接重新导出,那么不会保留重导入模块的副作用语句;若重导入模块为非直接重新导出,那么会保留重导入模块的副作用语句。
moduleSideEffects为true时的特性(rollup的默认配置)- 会预先假设所有依赖项模块(包括依赖模块和外部依赖模块)均存在副作用。
- 即使没有使用依赖项模块引入的变量,也会保留依赖项模块的副作用语句。
propertyReadSideEffects
默认值为 true
表示 rollup 会根据属性读取的副作用来判断是否保留某些代码。
tryCatchDeoptimization
默认值为 true
依赖于抛出错误的特征检测工作流,Rollup 将默认禁用 try 语句中的 tree-shaking 优化,也就是说 Rollup 会保留 try 语句中的所有代码。如果函数参数在 try 语句中被使用,那么该参数也不会被优化处理。
function a(d, w) {
try {
const b = 1;
const c = w;
} catch (_) {}
}
const param1 = 1;
const param2 = 2;
a(param1, param2);function a(d, w) {
try {
const b = 1;
const c = w;
} catch (_) {}
}
const param1 = 1;
const param2 = 2;
a(param1, param2);Ast 树的生成
Rollup 的所有 tree-shaking 操作均在 AST 上进行,因此首先需要生成 AST。生成 AST 的逻辑如下:
class ModuleLoader {
private async addModuleSource(
id: string,
importer: string | undefined,
module: Module
): Promise<void> {
let source: LoadResult;
try {
source = await this.graph.fileOperationQueue.run(async () => {
const content = await this.pluginDriver.hookFirst('load', [id]);
if (content !== null) return content;
this.graph.watchFiles[id] = true;
return await readFile(id, 'utf8');
});
} catch (error_: any) {
let message = `Could not load ${id}`;
if (importer) message += ` (imported by ${relativeId(importer)})`;
message += `: ${error_.message}`;
error_.message = message;
throw error_;
}
const sourceDescription =
typeof source === 'string'
? { code: source }
: source != null &&
typeof source === 'object' &&
typeof source.code === 'string'
? source
: error(logBadLoader(id));
const code = sourceDescription.code;
if (code.charCodeAt(0) === 0xfe_ff) {
sourceDescription.code = code.slice(1);
}
const cachedModule = this.graph.cachedModules.get(id);
if (
cachedModule &&
!cachedModule.customTransformCache &&
cachedModule.originalCode === sourceDescription.code &&
!(await this.pluginDriver.hookFirst('shouldTransformCachedModule', [
{
ast: cachedModule.ast,
code: cachedModule.code,
id: cachedModule.id,
meta: cachedModule.meta,
moduleSideEffects: cachedModule.moduleSideEffects,
resolvedSources: cachedModule.resolvedIds,
syntheticNamedExports: cachedModule.syntheticNamedExports
}
]))
) {
if (cachedModule.transformFiles) {
for (const emittedFile of cachedModule.transformFiles)
this.pluginDriver.emitFile(emittedFile);
}
await module.setSource(cachedModule);
} else {
module.updateOptions(sourceDescription);
await module.setSource(
await transform(
sourceDescription,
module,
this.pluginDriver,
this.options.onLog
)
);
}
}
}可以清晰的看到在加载完代码(source)后,会执行 module.setSource 方法,将代码转换为 AST。当然此处有涉及到 Ast 的缓存优化,后续会介绍。
Ast 树的缓存
从缓存的实现逻辑可以得知
const cachedModule = this.graph.cachedModules.get(id);
if (
cachedModule &&
!cachedModule.customTransformCache &&
cachedModule.originalCode === sourceDescription.code &&
!(await this.pluginDriver.hookFirst('shouldTransformCachedModule', [
{
ast: cachedModule.ast,
code: cachedModule.code,
id: cachedModule.id,
meta: cachedModule.meta,
moduleSideEffects: cachedModule.moduleSideEffects,
resolvedSources: cachedModule.resolvedIds,
syntheticNamedExports: cachedModule.syntheticNamedExports
}
]))
) {
if (cachedModule.transformFiles) {
for (const emittedFile of cachedModule.transformFiles)
this.pluginDriver.emitFile(emittedFile);
}
await module.setSource(cachedModule);
}需要满足以下几个条件才可以复用 Ast 缓存:
在
watch模式下,cachedModule的Map结构中存在Module的JSON结构缓存,缓存的模块信息如下。tsclass Module { toJSON(): ModuleJSON { return { ast: this.info.ast!, attributes: this.info.attributes, code: this.info.code!, customTransformCache: this.customTransformCache, dependencies: Array.from(this.dependencies, getId), id: this.id, meta: this.info.meta, moduleSideEffects: this.info.moduleSideEffects, originalCode: this.originalCode, originalSourcemap: this.originalSourcemap, resolvedIds: this.resolvedIds, sourcemapChain: this.sourcemapChain, syntheticNamedExports: this.info.syntheticNamedExports, transformDependencies: this.transformDependencies, transformFiles: this.transformFiles }; } }模块没有设置
customTransformCache。模块的代码没有发生变更
执行所有插件的
shouldTransformCachedModule钩子,均没有返回true的情况,即没有插件干预缓存的命中。
紧接着来看一下 module.setSource 方法中是如何处理缓存的:
class Module {
async setSource({
ast,
code,
customTransformCache,
originalCode,
originalSourcemap,
resolvedIds,
sourcemapChain,
transformDependencies,
transformFiles,
...moduleOptions
}: TransformModuleJSON & {
resolvedIds?: ResolvedIdMap;
transformFiles?: EmittedFile[] | undefined;
}): Promise<void> {
// 仅展示生成 AST 的逻辑
if (ast) {
this.ast = new nodeConstructors[ast.type](
programParent,
this.scope
).parseNode(ast) as Program;
this.info.ast = ast;
} else {
// Measuring asynchronous code does not provide reasonable results
timeEnd('generate ast', 3);
const astBuffer = await parseAsync(code, false);
timeStart('generate ast', 3);
this.ast = convertProgram(astBuffer, programParent, this.scope);
// Make lazy and apply LRU cache to not hog the memory
Object.defineProperty(this.info, 'ast', {
get: () => {
if (this.graph.astLru.has(fileName)) {
return this.graph.astLru.get(fileName)!;
} else {
const parsedAst = this.tryParse();
// If the cache is not disabled, we need to keep the AST in memory
// until the end when the cache is generated
if (this.options.cache !== false) {
Object.defineProperty(this.info, 'ast', {
value: parsedAst
});
return parsedAst;
}
// Otherwise, we keep it in a small LRU cache to not hog too much
// memory but allow the same AST to be requested several times.
this.graph.astLru.set(fileName, parsedAst);
return parsedAst;
}
}
});
}
}
}在 Rollup 中,Module 实例中存储着两种 ast 结构,一种是 Estree 规范的 ast,存储在 module.info.ast 变量中;另一种是 Rollup 根据 estree 结构生成的的 ast 类实例,存储在 module.ast 变量中。
Module 实例中缓存的信息如下:
class Module {
toJSON(): ModuleJSON {
return {
ast: this.info.ast!,
attributes: this.info.attributes,
code: this.info.code!,
customTransformCache: this.customTransformCache,
dependencies: Array.from(this.dependencies, getId),
id: this.id,
meta: this.info.meta,
moduleSideEffects: this.info.moduleSideEffects,
originalCode: this.originalCode,
originalSourcemap: this.originalSourcemap,
resolvedIds: this.resolvedIds,
sourcemapChain: this.sourcemapChain,
syntheticNamedExports: this.info.syntheticNamedExports,
transformDependencies: this.transformDependencies,
transformFiles: this.transformFiles
};
}
}可以发现 Rollup 仅在 Module 实例中缓存了 Estree 规范的 ast,而 Rollup 实现的 ast 类实例并没有进行缓存。因此这里需要做 parseNode 操作,将标准的 Estree 规范的 ast 转换为 Rollup 实现的 ast 类。后续的 tree-shaking 操作均在 Rollup 实现的 ast 类上进行。
若没有命中缓存(首次构建时),Rollup 会借助 swc 的能力将代码转译为 swc 的 ast。
const astBuffer = await parseAsync(code, false);需要注意的是 swc 转译的 ast 结构与标准 Estree 规范的 ast 结构并不完全一致,Rollup 需要将 swc 的 ast 结构转换为 Rollup 的 ast 类实例,存储在 module.ast 变量中。同时还需要将 swc 的 ast 结构转换为 Estree 规范的 ast 结构,并存储在 module.info.ast 变量中。
小结 ast 缓存
不管是转换为 Rollup 的 AST 类实例还是 Estree 规范的 AST,均需要借助 swc 的能力,将源码转译为 AST 二进制流。

针对 acorn parser 切换到 swc parser 的改动,可以参考 Switch Parser To Swc。
Ast 树构建前的预备
astContext 的初始化
每一个模块中均初始化了 astContext 对象,在构建 Rollup 的 Ast 实例树时,会调用 astContext 对象中的方法来收集当前模块的信息。例如通过 addDynamicImport 方法收集 dynamic import 信息,通过 addImport 方法收集 import 信息,通过 addExport 方法收集 export 和 reexports 的信息,通过 addImportMeta 方法收集 import.meta 信息,通过判断 Ast 的结构来判断是否当前模块使用顶层 await 等信息。
this.astContext = {
addDynamicImport: this.addDynamicImport.bind(this),
addExport: this.addExport.bind(this),
addImport: this.addImport.bind(this),
addImportMeta: this.addImportMeta.bind(this),
code, // Only needed for debugging
deoptimizationTracker: this.graph.deoptimizationTracker,
error: this.error.bind(this),
fileName, // Needed for warnings
getExports: this.getExports.bind(this),
getModuleExecIndex: () => this.execIndex,
getModuleName: this.basename.bind(this),
getNodeConstructor: (name: string) => nodeConstructors[name] || nodeConstructors.UnknownNode,
getReexports: this.getReexports.bind(this),
importDescriptions: this.importDescriptions,
includeAllExports: () => this.includeAllExports(true),
includeDynamicImport: this.includeDynamicImport.bind(this),
includeVariableInModule: this.includeVariableInModule.bind(this),
log: this.log.bind(this),
magicString: this.magicString,
manualPureFunctions: this.graph.pureFunctions,
module: this,
moduleContext: this.context,
options: this.options,
requestTreeshakingPass: () => (this.graph.needsTreeshakingPass = true),
traceExport: (name: string) => this.getVariableForExportName(name)[0],
traceVariable: this.traceVariable.bind(this),
usesTopLevelAwait: false
};接下来介绍在构建 Rollup 的 Ast 实例树时,是如何收集模块信息的。
收集模块信息
dynamic import当初始化ImportExpression节点时(解析完子节点后)。jsclass ImportExpression extends NodeBase { initialise() { super.initialise(); this.scope.context.addDynamicImport(this); } parseNode(esTreeNode: GenericEsTreeNode): this { this.sourceAstNode = esTreeNode.source; return super.parseNode(esTreeNode); } }会调用
addDynamicImport方法,将ImportExpression节点添加到当前模块的dynamicImports中。tsclass Module { private addDynamicImport(node: ImportExpression) { let argument: AstNode | string = node.sourceAstNode; if (argument.type === NodeType.TemplateLiteral) { if ( (argument as TemplateLiteralNode).quasis.length === 1 && typeof (argument as TemplateLiteralNode).quasis[0].value .cooked === 'string' ) { argument = (argument as TemplateLiteralNode).quasis[0].value .cooked!; } } else if ( argument.type === NodeType.Literal && typeof (argument as LiteralStringNode).value === 'string' ) { argument = (argument as LiteralStringNode).value!; } this.dynamicImports.push({ argument, id: null, node, resolution: null }); } }import当初始化
ImportDeclaration节点时。jsclass ImportDeclaration extends NodeBase { initialise(): void { super.initialise(); this.scope.context.addImport(this); } }会调用
addImport方法,将ImportDeclaration节点添加到当前模块的importDescriptions中。tsclass Module { private addImport(node: ImportDeclaration): void { const source = node.source.value; this.addSource(source, node); for (const specifier of node.specifiers) { const localName = specifier.local.name; if ( this.scope.variables.has(localName) || this.importDescriptions.has(localName) ) { this.error( logRedeclarationError(localName), specifier.local.start ); } const name = specifier instanceof ImportDefaultSpecifier ? 'default' : specifier instanceof ImportNamespaceSpecifier ? '*' : specifier.imported instanceof Identifier ? specifier.imported.name : specifier.imported.value; this.importDescriptions.set(localName, { module: null as never, // filled in later name, source, start: specifier.start }); } } }此时会通过导入方式的不同来存储特定的信息
ImportDefaultSpecifier存储defaultjsimport demo from 'demo.js'; module.importDescriptions.set('demo', { module: null, name: 'default', source: 'demo.js', start: 7 });ImportNamespaceSpecifier存储*jsimport * as demo from 'demo.js'; module.importDescriptions.set('demo', { module: null, name: '*', source: 'demo.js', start: 36 });ImportSpecifier存储imported.name或imported.valuejsimport { demo } from 'demo.js'; import { next as demo1 } from 'demo.js'; module.importDescriptions.set('demo', { module: null, // filled in later name: 'demo', source: 'demo.js', start: 72 }); module.importDescriptions.set('demo1', { module: null, // filled in later name: 'next', source: 'demo.js', start: 80 });export&&reexportsexport 涉及到的 Node 节点包括
ExportDefaultDeclaration、ExportAllDeclaration、ExportNamedDeclaration。js// case 1: ExportDefaultDeclaration export default 'Y'; // case 2: ExportAllDeclaration export * as name from './foo.js'; // case 3: ExportAllDeclaration export * from './foo.js'; // case 4: ExportNamedDeclaration export const a = 123; // case 5: ExportNamedDeclaration const demoA = 123; const demoB = 123; export { demoA as _a, demoB as _b }; // case 6: ExportNamedDeclaration export function foo() { console.log('foo'); } // case 7: ExportNamedDeclaration export { foo as _foo } from './foo.js';三个与 export 相关的节点在初始化 Node 阶段会调用
addExport方法,将当前模块的exports信息添加到exportDescriptions中。tsclass Module { private addExport( node: | ExportAllDeclaration | ExportNamedDeclaration | ExportDefaultDeclaration ): void { if (node instanceof ExportDefaultDeclaration) { // export default foo; this.assertUniqueExportName('default', node.start); this.exports.set('default', { identifier: node.variable.getAssignedVariableName(), localName: 'default' }); } else if (node instanceof ExportAllDeclaration) { const source = node.source.value; this.addSource(source, node); if (node.exported) { // export * as name from './other' const name = node.exported instanceof Literal ? node.exported.value : node.exported.name; this.assertUniqueExportName(name, node.exported.start); this.reexportDescriptions.set(name, { localName: '*', module: null as never, // filled in later, source, start: node.start }); } else { // export * from './other' this.exportAllSources.add(source); } } else if (node.source instanceof Literal) { // export { name } from './other' const source = node.source.value; this.addSource(source, node); for (const { exported, local, start } of node.specifiers) { const name = exported instanceof Literal ? exported.value : exported.name; this.assertUniqueExportName(name, start); this.reexportDescriptions.set(name, { localName: local instanceof Literal ? local.value : local.name, module: null as never, // filled in later, source, start }); } } else if (node.declaration) { const declaration = node.declaration; if (declaration instanceof VariableDeclaration) { // export var { foo, bar } = ... // export var foo = 1, bar = 2; for (const declarator of declaration.declarations) { for (const localName of extractAssignedNames(declarator.id)) { this.assertUniqueExportName(localName, declarator.id.start); this.exports.set(localName, { identifier: null, localName }); } } } else { // export function foo () {} const localName = (declaration.id as Identifier).name; this.assertUniqueExportName(localName, declaration.id!.start); this.exports.set(localName, { identifier: null, localName }); } } else { // export { foo, bar, baz } for (const { local, exported } of node.specifiers) { // except for reexports, local must be an Identifier const localName = (local as Identifier).name; const exportedName = exported instanceof Identifier ? exported.name : exported.value; this.assertUniqueExportName(exportedName, exported.start); this.exports.set(exportedName, { identifier: null, localName }); } } } }上述样例解析结果如下:
js// case 1: ExportDefaultDeclaration export default foo; // case 1 output module.exports.set('default', { identifier: null, localName: 'default' }); // case 2: ExportAllDeclaration export * as name from './foo.js'; // case 2 output module.reexportDescriptions.set('name', { localName: '*', module: null, // filled in later source: './foo.js', start: 72 }); // case 3: ExportAllDeclaration export * from './foo.js'; // case 3 output module.exportAllSources.add('./foo.js'); // case 4: ExportNamedDeclaration export const a = 123; // case 4 output module.exports.set('a', { identifier: null, localName: 'a' }); // case 5: ExportNamedDeclaration const demoA = 123; const demoB = 123; export { demoA as _a, demoB as _b }; // case 5 output module.exports.set('_a', { identifier: null, localName: 'demoA' }); module.exports.set('_b', { identifier: null, localName: 'demoB' }); // case 6: ExportNamedDeclaration export function foo() { console.log('foo'); } // case 6 output module.exports.set('foo', { identifier: null, localName: 'foo' }); // case 7: ExportNamedDeclaration export { foo as _foo } from './foo.js'; // case 7 output module.reexportDescriptions.set('_foo', { localName: 'foo', module: null, // filled in later source: './foo.js', start: 289 });Export重复声明校验解析 export 不同与 import 的点在于,export 需要验证是否在模块中重复声明变量,而 import 是允许重复声明的,无需做校验。
jsclass Module { assertUniqueExportName(name, nodeStart) { if (this.exports.has(name) || this.reexportDescriptions.has(name)) { this.error(parseAst_js.logDuplicateExportError(name), nodeStart); } } addExport(node: ExportAllDeclaration | ExportNamedDeclaration | ExportDefaultDeclaration): void { // 其他逻辑省略 this.assertUniqueExportName(name, node.exported.start); } }若 export 出现重复声明变量,则会抛出错误,并终止打包。
小结
可以发现在构建 Ast 的过程中收集的 import 和 export 相应的节点内容为在当前模块作用域中使用的 localName 与依赖方中所导出的 localName 的映射关系。同时需要关注的是所对应依赖项的映射值在 Ast 构建过程中并没有进行填充,而是在后续子依赖项解析完成后进行填充。
jsclass Module { addModulesToImportDescriptions(importDescription) { for (const specifier of importDescription.values()) { const { id } = this.resolvedIds[specifier.source]; specifier.module = this.graph.modulesById.get(id); } } linkImports() { this.addModulesToImportDescriptions(this.importDescriptions); this.addModulesToImportDescriptions(this.reexportDescriptions); const externalExportAllModules = []; for (const source of this.exportAllSources) { const module = this.graph.modulesById.get( this.resolvedIds[source].id ); if (module instanceof ExternalModule) { externalExportAllModules.push(module); continue; } this.exportAllModules.push(module); } this.exportAllModules.push(...externalExportAllModules); } }由于在执行
linkImports方法时,当前模块的所有依赖项模块都已经解析完成,所以可以在this.graph.modulesById中获取到所有依赖项模块的引用,此时对当前模块的importDescriptions和reexportDescriptions的依赖项模块引用进行填充。对于exportAllSources中存储的source,也通过this.graph.modulesById找到对应的依赖项模块,并将其添加到this.exportAllModules中。至此,Ast 构建阶段未填充的依赖项模块已经全部填充完毕。import.meta与
import.meta相关的 Node 节点是MetaProperty。这是声明式的节点,由于无法 ast 构建阶段判断是否需要import.meta节点信息,所以节点信息并没有在初始化节点进行处理,而是在tree-sharking后确认需要执行import.meta节点后才会通过addImportMeta方法收集import.meta的信息。jsclass MateProperty { include() { if (!this.included) { this.included = true; if (this.meta.name === IMPORT) { this.scope.context.addImportMeta(this); const parent = this.parent; const metaProperty = (this.metaProperty = parent instanceof MemberExpression && typeof parent.propertyKey === 'string' ? parent.propertyKey : null); if (metaProperty?.startsWith(FILE_PREFIX)) { this.referenceId = metaProperty.slice(FILE_PREFIX.length); } } } } } class Module { addImportMeta(node) { this.importMetas.push(node); } }usesTopLevelAwait这是为了标记当前模块是否包含了
Top Level Await。针对Top Level Await在不同 bundler 的实现在 文章 中已经详细说明了,这里不再赘述。与
Top Level Await相关的节点是AwaitExpression。jsclass AwaitExpression extends NodeBase { include(context: InclusionContext, includeChildrenRecursively: IncludeChildren): void { if (!this.deoptimized) this.applyDeoptimizations(); if (!this.included) { this.included = true; checkTopLevelAwait: if (!this.scope.context.usesTopLevelAwait) { let parent = this.parent; do { if (parent instanceof FunctionNode || parent instanceof ArrowFunctionExpression) break checkTopLevelAwait; } while ((parent = (parent as Node).parent as Node)); this.scope.context.usesTopLevelAwait = true; } } this.argument.include(context, includeChildrenRecursively); } }与
import.meta节点一样,AwaitExpression节点信息也是在tree-sharking后确认需要执行AwaitExpression节点后才会通过addAwaitExpression方法收集AwaitExpression的信息。判断模块中是否包含AwaitExpression节点的方式很简单,即往父级遍历,直到遇到FunctionNode或ArrowFunctionExpression节点为止,若均没有遇到,则将模块的usesTopLevelAwait标记为true。js// usesTopLevelAwait: true await 1; // usesTopLevelAwait: false (async () => { await 1; })(); // usesTopLevelAwait: false (function () { await 1; })();
以上这些信息均在 Ast 构建阶段收集完毕,而 includeAllExports、includeDynamicImport、includeVariableInModule 这些方法在 tree-sharking 阶段会进一步确定。
明确 Ast Node 的作用域
class Scope {
readonly children: ChildScope[] = [];
readonly variables = new Map<string, Variable>();
hoistedVariables?: Map<string, LocalVariable>;
}
class ChildScope extends Scope {
constructor(
readonly parent: Scope,
readonly context: AstContext
) {
super();
parent.children.push(this);
}
}
class ModuleScope extends ChildScope {
declare parent: GlobalScope;
constructor(parent: GlobalScope, context: AstContext) {
super(parent, context);
this.variables.set(
'this',
new LocalVariable(
'this',
null,
UNDEFINED_EXPRESSION,
context,
'other'
)
);
}
}
class Module {
async setSource({
ast,
code,
customTransformCache,
originalCode,
originalSourcemap,
resolvedIds,
sourcemapChain,
transformDependencies,
transformFiles,
...moduleOptions
}: TransformModuleJSON & {
resolvedIds?: ResolvedIdMap;
transformFiles?: EmittedFile[] | undefined;
}): Promise<void> {
this.scope = new ModuleScope(this.graph.scope, this.astContext);
}
}可以看到在 Rollup 为当前模块创建一个 ModuleScope 实例(即模块作用域),其父集为 GlobalScope(即全局作用域,也就是 this.graph.scope 的值)。同时为当前的模块作用域中添加 this 变量,作为当前模块作用域中第一个变量。
接下来举一个例子来讲解 Scope 在 Ast 构建过程中是如何发挥作用的。
const localVariable = 123;
function scopeDemo() {
const scopeDemoLocalVariable = 345;
const scopeDemoLocalFunction = function (params) {
const d = 4445;
return params + d;
};
console.log(scopeDemoLocalFunction(scopeDemoLocalVariable));
}
scopeDemo();
console.log(localVariable);因为 Ast Node 节点均继承与 NodeBase,实例化 Ast Node 节点时会调用 NodeBase 的构造函数,为当前的 Ast Node 节点创建 scope 作用域。
class NodeBase extends ExpressionEntity {
constructor(parent, parentScope) {
super();
this.parent = parent;
this.scope = parentScope;
this.createScope(parentScope);
}
createScope(parentScope) {
this.scope = parentScope;
}
}我们先拿 const localVariable = 123; 这条语句来举例,VariableDeclaration 节点初始化完成后(即 VariableDeclaration 节点的所有 Ast Node 节点均已经初始化完成),则会调用 VariableDeclaration 节点的 initialise 方法。
class Identifier extends NodeBase {
declare(kind, init) {
let variable;
const { treeshake } = this.scope.context.options;
switch (kind) {
case 'var': {
variable = this.scope.addDeclaration(
this,
this.scope.context,
init,
kind
);
if (treeshake && treeshake.correctVarValueBeforeDeclaration) {
// Necessary to make sure the init is deoptimized. We cannot call deoptimizePath here.
variable.markInitializersForDeoptimization();
}
break;
}
case 'function': {
// in strict mode, functions are only hoisted within a scope but not across block scopes
variable = this.scope.addDeclaration(
this,
this.scope.context,
init,
kind
);
break;
}
case 'let':
case 'const':
case 'using':
case 'await using':
case 'class': {
variable = this.scope.addDeclaration(
this,
this.scope.context,
init,
kind
);
break;
}
case 'parameter': {
variable = this.scope.addParameterDeclaration(this);
break;
}
/* istanbul ignore next */
default: {
/* istanbul ignore next */
throw new Error(
`Internal Error: Unexpected identifier kind ${kind}.`
);
}
}
return [(this.variable = variable)];
}
}
class VariableDeclarator extends NodeBase {
declareDeclarator(kind, isUsingDeclaration) {
this.isUsingDeclaration = isUsingDeclaration;
this.id.declare(kind, this.init || UNDEFINED_EXPRESSION);
}
}
class VariableDeclaration extends NodeBase {
initialise() {
super.initialise();
this.isUsingDeclaration =
this.kind === 'await using' || this.kind === 'using';
for (const declarator of this.declarations) {
declarator.declareDeclarator(this.kind, this.isUsingDeclaration);
}
}
}可以看到通过 const localVariable = 123; 这条语句,VariableDeclaration 节点会调用 VariableDeclarator 节点的 declareDeclarator 方法,而 VariableDeclarator 节点又会调用 Identifier 节点的 declare 方法,通过 this.scope.addDeclaration 方法将 Identifier 节点添加到 scope.variables 中。
class Scope {
addDeclaration(identifier, context, init, kind) {
const name = identifier.name;
const existingVariable =
this.hoistedVariables?.get(name) || this.variables.get(name);
if (existingVariable) {
const existingKind = existingVariable.kind;
if (kind === 'var' && existingKind === 'var') {
existingVariable.addDeclaration(identifier, init);
return existingVariable;
}
context.error(
parseAst_js.logRedeclarationError(name),
identifier.start
);
}
const newVariable = new LocalVariable(
identifier.name,
identifier,
init,
context,
kind
);
this.variables.set(name, newVariable);
return newVariable;
}
}
class ModuleScope extends ChildScope {
addDeclaration(identifier, context, init, kind) {
if (this.context.module.importDescriptions.has(identifier.name)) {
context.error(
parseAst_js.logRedeclarationError(identifier.name),
identifier.start
);
}
return super.addDeclaration(identifier, context, init, kind);
}
}可以发现此时会检测当前的变量声明是否与 import 导入的变量声明重复,若出现重复则直接抛出异常。同时还会对 var 关键字声明的变量进行检测,若非 var 关键字声明的变量重复声明,则抛出异常。检测通过后会根据 Identifier Ast Node 节点信息实例化 LocalVariable 类,并将实例化的变量添加到当前作用域的 scope.variables 中。
当遇到函数声明语句时,会在 FunctionNode 节点中调用 createScope 方法为当前的 FunctionNode 节点创建新的 scope 作用域。
class Scope {
constructor() {
this.children = [];
this.variables = new Map();
}
}
class ChildScope extends Scope {
constructor(parent, context) {
super();
this.parent = parent;
this.context = context;
this.accessedOutsideVariables = new Map();
parent.children.push(this);
}
}
class ParameterScope extends ChildScope {
constructor(parent, isCatchScope) {
super(parent, parent.context);
this.parameters = [];
this.hasRest = false;
this.bodyScope = isCatchScope
? new CatchBodyScope(this)
: new FunctionBodyScope(this);
}
}
class ReturnValueScope extends ParameterScope {
constructor() {
super(...arguments);
this.returnExpression = null;
this.returnExpressions = [];
}
}
class FunctionScope extends ReturnValueScope {
constructor(parent) {
const { context } = parent;
super(parent, false);
this.variables.set(
'arguments',
(this.argumentsVariable = new ArgumentsVariable(context))
);
this.variables.set(
'this',
(this.thisVariable = new ThisVariable(context))
);
}
}
class FunctionNode extends FunctionBase {
constructor() {
super(...arguments);
this.objectEntity = null;
}
createScope(parentScope) {
this.scope = new FunctionScope(parentScope);
this.constructedEntity = new ObjectEntity(
Object.create(null),
OBJECT_PROTOTYPE
);
// This makes sure that all deoptimizations of "this" are applied to the
// constructed entity.
this.scope.thisVariable.addEntityToBeDeoptimized(
this.constructedEntity
);
}
}可以看到在 FunctionScope 中为当前函数节点创建一个父集为 ModuleScope 新的函数 scope 作用域。并在新的 scope 作用域中创建 this 变量和 arguments 变量作为当前作用域中的初始变量。同时为父集 ModuleScope 收集新的函数作用域存储在父集 ModuleScope 的 children 数组中。
注意
函数作用域中声明的变量并不是添加到 FunctionScope 的 scope.variables 中,而是添加到 FunctionScope.bodyScope 的 scope.variables 中。
function functionDeclaration(node: FunctionDeclaration, position, buffer) {
const { scope } = node;
const flags = buffer[position];
node.async = (flags & 1) === 1;
node.generator = (flags & 2) === 2;
const annotations = (node.annotations = convertAnnotations(buffer[position + 1], buffer));
node.annotationNoSideEffects = annotations.some(comment => comment.type === 'noSideEffects');
const idPosition = buffer[position + 2];
node.id =
idPosition === 0 ? null : convertNode(node, scope.parent as ChildScope, idPosition, buffer);
const parameters = (node.params = convertNodeList(node, scope, buffer[position + 3], buffer));
scope.addParameterVariables(
parameters.map(
parameter => parameter.declare('parameter', UNKNOWN_EXPRESSION) as ParameterVariable[]
),
parameters[parameters.length - 1] instanceof RestElement
);
node.body = convertNode(node, scope.bodyScope, buffer[position + 4], buffer);
}那么 bodyScope 是哪里来的呢?
class ParameterScope extends ChildScope {
constructor(parent, isCatchScope) {
super(parent, parent.context);
this.parameters = [];
this.hasRest = false;
this.bodyScope = isCatchScope ? new CatchBodyScope(this) : new FunctionBodyScope(this);
}
}
class ReturnValueScope extends ParameterScope {}
class FunctionScope extends ReturnValueScope {}可以看到在创建函数作用域的时候会创建 FunctionBodyScope 作为当前函数作用域的 bodyScope,同时 FunctionBodyScope 的父集为 FunctionScope。后续在 函数体中声明的变量 均会添加到 FunctionBodyScope 的 scope.variables 中。
函数表达式的参数会通过 scope.addParameterVariables 方法添加到 FunctionScope 的 scope.variables 中。
function functionExpression(node: FunctionExpression, position, buffer) {
const { scope } = node;
const flags = buffer[position];
node.async = (flags & 1) === 1;
node.generator = (flags & 2) === 2;
const annotations = (node.annotations = convertAnnotations(buffer[position + 1], buffer));
node.annotationNoSideEffects = annotations.some(comment => comment.type === 'noSideEffects');
const idPosition = buffer[position + 2];
node.id = idPosition === 0 ? null : convertNode(node, node.idScope, idPosition, buffer);
const parameters = (node.params = convertNodeList(node, scope, buffer[position + 3], buffer));
scope.addParameterVariables(
parameters.map(
parameter => parameter.declare('parameter', UNKNOWN_EXPRESSION) as ParameterVariable[]
),
parameters[parameters.length - 1] instanceof RestElement
);
node.body = convertNode(node, scope.bodyScope, buffer[position + 4], buffer);
}同时还会将参数添加到函数作用域(FunctionScope)下的函数体作用域(bodyScope,bodyScope.parent 为 FunctionScope)的 hoistedVariables 中。
class Scope {
addHoistedVariable(name: string, variable: LocalVariable) {
(this.hoistedVariables ||= new Map()).set(name, variable);
}
}
class ParameterScope extends ChildScope {
addParameterDeclaration(identifier: Identifier): ParameterVariable {
const { name, start } = identifier;
const existingParameter = this.variables.get(name);
if (existingParameter) {
return this.context.error(logDuplicateArgumentNameError(name), start);
}
const variable = new ParameterVariable(name, identifier, this.context);
this.variables.set(name, variable);
// We also add it to the body scope to detect name conflicts with local
// variables. We still need the intermediate scope, though, as parameter
// defaults are NOT taken from the body scope but from the parameters or
// outside scope.
this.bodyScope.addHoistedVariable(name, variable);
return variable;
}
}hoistedVariables 的作用是为了在函数体中检测标识符是否与参数变量声明冲突。
function logRedeclarationError(name: string): RollupLog {
return {
code: REDECLARATION_ERROR,
message: `Identifier "${name}" has already been declared`
};
}
class FunctionBodyScope extends ChildScope {
// There is stuff that is only allowed in function scopes, i.e. functions can
// be redeclared, functions and var can redeclare each other
addDeclaration(
identifier: Identifier,
context: AstContext,
init: ExpressionEntity,
kind: VariableKind
): LocalVariable {
const name = identifier.name;
const existingVariable =
this.hoistedVariables?.get(name) || (this.variables.get(name) as LocalVariable);
if (existingVariable) {
const existingKind = existingVariable.kind;
if (
(kind === 'var' || kind === 'function') &&
(existingKind === 'var' || existingKind === 'function' || existingKind === 'parameter')
) {
existingVariable.addDeclaration(identifier, init);
return existingVariable;
}
context.error(logRedeclarationError(name), identifier.start);
}
const newVariable = new LocalVariable(identifier.name, identifier, init, context, kind);
this.variables.set(name, newVariable);
return newVariable;
}
}可以看到若存在非 var 和 function 关键字声明的变量与参数变量同名,则抛出异常: Identifier xxx has already been declared。因此在构建 Ast 树的过程中,就可以检测到变量声明的重复问题。
Ast 树的构建
上述提到 module.ast 是 Rollup 实现的 ast 类实例,结构与 estree 规范的 ast 结构一致,后续 tree-shaking 操作均在 module.ast 上进行。那么核心问题来了,Rollup 是如何将标准的 Estree 规范的 ast 转换为 rollup 实现的 ast 类实例?
标准 Estree 规范的 ast 结构解析为 Rollup 实现的 ast 类实例的逻辑如下:
import type * as estree from 'estree';
type OmittedEstreeKeys =
| 'loc'
| 'range'
| 'leadingComments'
| 'trailingComments'
| 'innerComments'
| 'comments';
interface AstNodeLocation {
end: number;
start: number;
}
type RollupAstNode<T> = Omit<T, OmittedEstreeKeys> & AstNodeLocation;
type AstNode = RollupAstNode<estree.Node>;
interface GenericEsTreeNode extends AstNode {
[key: string]: any;
}
type AnnotationType = 'pure' | 'noSideEffects';
interface RollupAnnotation {
start: number;
end: number;
type: AnnotationType;
}
const nodeConstructors: Record<string, typeof NodeBase> = {
ArrayExpression,
ArrayPattern,
ArrowFunctionExpression,
AssignmentExpression,
AssignmentPattern,
AwaitExpression,
BinaryExpression,
BlockStatement,
BreakStatement,
CallExpression,
CatchClause,
ChainExpression,
ClassBody,
ClassDeclaration,
ClassExpression,
ConditionalExpression,
ContinueStatement,
DebuggerStatement,
Decorator,
DoWhileStatement,
EmptyStatement,
ExportAllDeclaration,
ExportDefaultDeclaration,
ExportNamedDeclaration,
ExportSpecifier,
ExpressionStatement,
ForInStatement,
ForOfStatement,
ForStatement,
FunctionDeclaration,
FunctionExpression,
Identifier,
IfStatement,
ImportAttribute,
ImportDeclaration,
ImportDefaultSpecifier,
ImportExpression,
ImportNamespaceSpecifier,
ImportSpecifier,
LabeledStatement,
Literal,
LogicalExpression,
MemberExpression,
MetaProperty,
MethodDefinition,
NewExpression,
ObjectExpression,
ObjectPattern,
PanicError,
ParseError,
PrivateIdentifier,
Program,
Property,
PropertyDefinition,
RestElement,
ReturnStatement,
SequenceExpression,
SpreadElement,
StaticBlock,
Super,
SwitchCase,
SwitchStatement,
TaggedTemplateExpression,
TemplateElement,
TemplateLiteral,
ThisExpression,
ThrowStatement,
TryStatement,
UnaryExpression,
UnknownNode,
UpdateExpression,
VariableDeclaration,
VariableDeclarator,
WhileStatement,
YieldExpression
};
class NodeBase {
parseNode(esTreeNode: GenericEsTreeNode): this {
for (const [key, value] of Object.entries(esTreeNode)) {
// Skip properties defined on the class already.
// This way, we can override this function to add custom initialisation and then call super.parseNode
// Note: this doesn't skip properties with defined getters/setters which we use to pack wrap booleans
// in bitfields. Those are still assigned from the value in the esTreeNode.
if (this.hasOwnProperty(key)) continue;
if (key.charCodeAt(0) === 95 /* _ */) {
if (key === ANNOTATION_KEY) {
this.annotations = value as RollupAnnotation[];
} else if (key === INVALID_ANNOTATION_KEY) {
(this as unknown as Program).invalidAnnotations =
value as RollupAnnotation[];
}
} else if (typeof value !== 'object' || value === null) {
(this as GenericEsTreeNode)[key] = value;
} else if (Array.isArray(value)) {
(this as GenericEsTreeNode)[key] = [];
for (const child of value) {
(this as GenericEsTreeNode)[key]
.push(
child === null ? null : new nodeConstructors[child.type]()
)(this, this.scope)
.parseNode(child);
}
} else {
(this as GenericEsTreeNode)[key] = new nodeConstructors[value.type](
this,
this.scope
).parseNode(value);
}
}
// extend child keys for unknown node types
childNodeKeys[esTreeNode.type] ||=
createChildNodeKeysForNode(esTreeNode);
this.initialise();
return this;
}
}可以看出 Rollup 内部实现了具有 estree 规范的 Ast Node 类 nodeConstructors。通过 DFS 遍历 estree 规范的 ast 树,实例化对应的 Rollup Ast Node 类。
Ast 的节点操作
getVariableForExportName
注意
在 generateModuleGraph 阶段会执行实例化 Module 的操作,同时根据 source code 解析对应的 Ast 树。需要注意的是此时模块的依赖项模块还没有开始实例化,因此无法在解析 Ast 树时获取到子依赖模块的引用,所以在 importDescriptions 和 reexportDescriptions 中收集到的导入变量和重导出变量没有填充依赖模块项的模块引用,处于 未填充状态。
当依赖项模块均已经实例化完成后,就会执行 module.linkImports() 方法为 importDescriptions 和 reexportDescriptions 填充依赖模块项的引用。
class Module {
linkImports(): void {
this.addModulesToImportDescriptions(this.importDescriptions);
this.addModulesToImportDescriptions(this.reexportDescriptions);
const externalExportAllModules: ExternalModule[] = [];
for (const source of this.exportAllSources) {
const module = this.graph.modulesById.get(this.resolvedIds[source].id)!;
if (module instanceof ExternalModule) {
externalExportAllModules.push(module);
continue;
}
this.exportAllModules.push(module);
}
this.exportAllModules.push(...externalExportAllModules);
}
}tree-sharking 阶段, importDescriptions 和 reexportDescriptions 的依赖模块项的模块引用已经填充完毕,因此在 tree-sharking 阶段可以通过以下方式来获取依赖项的 module 引用。
// main.js
// export * as d from 'demo.js';
const reexportDeclaration = this.reexportDescriptions.get(name);
const childDependenciesReexportModule = reexportDeclaration.module;
// import { d } from 'demo.js';
const importDescription = this.importDescriptions.get(name);
const childDependenciesImportModule = importDescription.module;继续分析 getVariableForExportName 方法的实现。
class Module {
getVariableForExportName(
name: string,
{
importerForSideEffects,
isExportAllSearch,
onlyExplicit,
searchedNamesAndModules
}: {
importerForSideEffects?: Module;
isExportAllSearch?: boolean;
onlyExplicit?: boolean;
searchedNamesAndModules?: Map<string, Set<Module | ExternalModule>>;
} = EMPTY_OBJECT
): [variable: Variable | null, indirectExternal?: boolean] {
if (name[0] === '*') {
if (name.length === 1) {
// export * from './other'
return [this.namespace];
}
// export * from 'external'
const module = this.graph.modulesById.get(
name.slice(1)
) as ExternalModule;
return module.getVariableForExportName('*');
}
// export { foo } from './other'
const reexportDeclaration = this.reexportDescriptions.get(name);
if (reexportDeclaration) {
const [variable] = getVariableForExportNameRecursive(
reexportDeclaration.module,
reexportDeclaration.localName,
importerForSideEffects,
false,
searchedNamesAndModules
);
if (!variable) {
return this.error(
logMissingExport(
reexportDeclaration.localName,
this.id,
reexportDeclaration.module.id
),
reexportDeclaration.start
);
}
if (importerForSideEffects) {
setAlternativeExporterIfCyclic(
variable,
importerForSideEffects,
this
);
if (this.info.moduleSideEffects) {
getOrCreate(
importerForSideEffects.sideEffectDependenciesByVariable,
variable,
getNewSet<Module>
).add(this);
}
}
return [variable];
}
const exportDeclaration = this.exports.get(name);
if (exportDeclaration) {
if (exportDeclaration === MISSING_EXPORT_SHIM_DESCRIPTION) {
return [this.exportShimVariable];
}
const name = exportDeclaration.localName;
const variable = this.traceVariable(name, {
importerForSideEffects,
searchedNamesAndModules
})!;
if (importerForSideEffects) {
setAlternativeExporterIfCyclic(
variable,
importerForSideEffects,
this
);
getOrCreate(
importerForSideEffects.sideEffectDependenciesByVariable,
variable,
getNewSet<Module>
).add(this);
}
return [variable];
}
if (onlyExplicit) {
return [null];
}
if (name !== 'default') {
const foundNamespaceReexport =
this.namespaceReexportsByName.get(name) ??
this.getVariableFromNamespaceReexports(
name,
importerForSideEffects,
searchedNamesAndModules
);
this.namespaceReexportsByName.set(name, foundNamespaceReexport);
if (foundNamespaceReexport[0]) {
return foundNamespaceReexport;
}
}
if (this.info.syntheticNamedExports) {
return [
getOrCreate(
this.syntheticExports,
name,
() =>
new SyntheticNamedExportVariable(
this.astContext,
name,
this.getSyntheticNamespace()
)
)
];
}
// we don't want to create shims when we are just
// probing export * modules for exports
if (!isExportAllSearch && this.options.shimMissingExports) {
this.shimMissingExport(name);
return [this.exportShimVariable];
}
return [null];
}
}可以看到实现上针对不同类型的导出方式获取的方式有些差异化,以下逐一介绍针对不同方式的导出,其获取 Rollup Ast Node 的不同实现方式。
重导出(reexport)变量的
Rollup Ast节点获取可以看出针对重导出的
Rollup Ast变量声明节点的获取会调用getVariableForExportNameRecursive方法tsconst reexportDeclaration = this.reexportDescriptions.get(name); if (reexportDeclaration) { const [variable] = getVariableForExportNameRecursive( // 重导出依赖模块 reexportDeclaration.module, reexportDeclaration.localName, importerForSideEffects, false, searchedNamesAndModules ); // 其他逻辑省略 }在
getVariableForExportNameRecursive方法内部递归调用 重导出的依赖模块 的getVariableForExportName方法来检索依赖模块中是否声明了目标变量,若没有则继续递归调用直到找到目标变量的Rollup Ast Node节点为止。tsfunction getVariableForExportNameRecursive( target: Module | ExternalModule, name: string, importerForSideEffects: Module | undefined, isExportAllSearch: boolean | undefined, searchedNamesAndModules = new Map< string, Set<Module | ExternalModule> >() ): [variable: Variable | null, indirectExternal?: boolean] { const searchedModules = searchedNamesAndModules.get(name); if (searchedModules) { if (searchedModules.has(target)) { return isExportAllSearch ? [null] : error(logCircularReexport(name, target.id)); } searchedModules.add(target); } else { searchedNamesAndModules.set(name, new Set([target])); } return target.getVariableForExportName(name, { importerForSideEffects, isExportAllSearch, searchedNamesAndModules }); }导出(export)变量的
Rollup Ast节点获取获取
export的变量的Rollup Ast Node节点tsclass Module { getVariableForExportName( name: string, { importerForSideEffects, isExportAllSearch, onlyExplicit, searchedNamesAndModules }: { importerForSideEffects?: Module; isExportAllSearch?: boolean; onlyExplicit?: boolean; searchedNamesAndModules?: Map< string, Set<Module | ExternalModule> >; } = EMPTY_OBJECT ): [variable: Variable | null, indirectExternal?: boolean] { // 其他逻辑省略 const exportDeclaration = this.exports.get(name); if (exportDeclaration) { if (exportDeclaration === MISSING_EXPORT_SHIM_DESCRIPTION) { return [this.exportShimVariable]; } const name = exportDeclaration.localName; const variable = this.traceVariable(name, { importerForSideEffects, searchedNamesAndModules })!; if (importerForSideEffects) { setAlternativeExporterIfCyclic( variable, importerForSideEffects, this ); getOrCreate( importerForSideEffects.sideEffectDependenciesByVariable, variable, getNewSet<Module> ).add(this); } return [variable]; } } // 其他逻辑省略 }可以看到会通过
traceVariable方法获取目标模块中声明的导出变量。jsclass Module { traceVariable( name: string, { importerForSideEffects, isExportAllSearch, searchedNamesAndModules }: { importerForSideEffects?: Module; isExportAllSearch?: boolean; searchedNamesAndModules?: Map<string, Set<Module | ExternalModule>>; } = EMPTY_OBJECT ): Variable | null { const localVariable = this.scope.variables.get(name); if (localVariable) { return localVariable; } const importDescription = this.importDescriptions.get(name); if (importDescription) { const otherModule = importDescription.module; if (otherModule instanceof Module && importDescription.name === '*') { return otherModule.namespace; } const [declaration] = getVariableForExportNameRecursive( otherModule, importDescription.name, importerForSideEffects || this, isExportAllSearch, searchedNamesAndModules ); if (!declaration) { return this.error( logMissingExport(importDescription.name, this.id, otherModule.id), importDescription.start ); } return declaration; } return null; } }但可能需要考虑以下两种特殊情况:
导出的是当前模块中声明的变量
js// target.js export const a = 1;这是一个递归出口。对于这种情况直接通过
this.scope.variables.get(name)获取当前作用域中声明的Rollup Ast Node节点即可。jsconst localVariable = this.scope.variables.get(name); if (localVariable) { return localVariable; }导出的是依赖模块中声明的变量
js// target.js import { a } from './other'; export { a };对于这种本质上和重导出的性质一致,因此继续通过
getVariableForExportNameRecursive方法递归其他模块获取目标模块作用域中包含的变量声明。jsif (importDescription) { const otherModule = importDescription.module; if (otherModule instanceof Module && importDescription.name === '*') { return otherModule.namespace; } const [declaration] = getVariableForExportNameRecursive( otherModule, importDescription.name, importerForSideEffects || this, isExportAllSearch, searchedNamesAndModules ); if (!declaration) { return this.error( logMissingExport(importDescription.name, this.id, otherModule.id), importDescription.start ); } return declaration; }举一个例子
js// main.js export { a } from './other';js// other.js export { a } from './other-next';js// other-next.js import { a } from './other-next-next.js'; export { a };js// other-next-next.js export const a = 123;若在
main.js中通过getVariableForExportName方法获取a变量的Rollup Ast Node节点时,会递归调用getVariableForExportNameRecursive方法,直到找到a变量为止。也就是最后会获取到other-next-next模块中声明的a变量的Rollup Ast Node节点,同时该Rollup Ast Node节点的module属性为other-next-next模块的引用。:::
确定 statements 是否具有副作用
提前说明
每一个模块中都会包含 moduleSideEffects 属性(module.info.moduleSideEffects),该属性用于标记当前的模块是否具有副作用,从而影响 Rollup 对模块的 tree-sharking 行为。
interface ModuleOptions {
attributes: Record<string, string>;
meta: CustomPluginOptions;
moduleSideEffects: boolean | 'no-treeshake';
syntheticNamedExports: boolean | string;
}moduleSideEffects 值分为以下三种:
moduleSideEffects = true表示该模块具有副作用,这是 Rollup 的默认行为,
Rollup认为所有模块都具有副作用。tsclass ModuleLoader { constructor( private readonly graph: Graph, private readonly modulesById: Map<string, Module | ExternalModule>, private readonly options: NormalizedInputOptions, private readonly pluginDriver: PluginDriver ) { this.hasModuleSideEffects = options.treeshake ? options.treeshake.moduleSideEffects : () => true; } private getResolvedIdWithDefaults( resolvedId: NormalizedResolveIdWithoutDefaults | null, attributes: Record<string, string> ): ResolvedId | null { if (!resolvedId) { return null; } const external = resolvedId.external || false; return { attributes: resolvedId.attributes || attributes, external, id: resolvedId.id, meta: resolvedId.meta || {}, moduleSideEffects: resolvedId.moduleSideEffects ?? this.hasModuleSideEffects(resolvedId.id, !!external), resolvedBy: resolvedId.resolvedBy ?? 'rollup', syntheticNamedExports: resolvedId.syntheticNamedExports ?? false }; } }执行
Rollup的resolveId插件钩子时,可以通过钩子返回的moduleSideEffects属性来指定模块是否有副作用,从而影响Rollup对模块的tree-sharking行为。钩子没有返回moduleSideEffects属性时,则默认模块具有副作用(即moduleSideEffects = true),当然也可以通过options.treeshake.moduleSideEffects属性来指定上述的默认行为。tree-sharking阶段会根据statements是否有副作用(hasEffects)执行tree-sharking操作。tsclass Module { include(): void { const context = createInclusionContext(); if (this.ast!.shouldBeIncluded(context)) this.ast!.include(context, false); } }moduleSideEffects = false:表示该模块没有副作用,tree-sharking阶段会直接删掉该模块。moduleSideEffects = 'no-treeshake':表示该模块不需要执行tree-sharking操作,完全保留该模块的内容及其所有依赖项。tsclass Graph { private includeStatements(): void { // 省略上方逻辑 if (this.options.treeshake) { let treeshakingPass = 1; do { timeStart(`treeshaking pass ${treeshakingPass}`, 3); this.needsTreeshakingPass = false; for (const module of this.modules) { if (module.isExecuted) { if (module.info.moduleSideEffects === 'no-treeshake') { module.includeAllInBundle(); } else { module.include(); } } } if (treeshakingPass === 1) { // We only include exports after the first pass to avoid issues with // the TDZ detection logic for (const module of entryModules) { if (module.preserveSignature !== false) { module.includeAllExports(false); this.needsTreeshakingPass = true; } } } timeEnd(`treeshaking pass ${treeshakingPass++}`, 3); } while (this.needsTreeshakingPass); } // 省略下方逻辑 } }可以看到如果需要执行的模块,若其
moduleSideEffects属性值设置为'no-treeshake'时,Rollup会通过执行module.includeAllInBundle()方法来保留该模块中的所有内容,includeAllInBundle的详细说明可见下方。
moduleSideEffects 属性值还可以通过插件的 load、transform 钩子来指定当前模块是否具有副作用,插件执行完后 Rollup 会通过 updateOptions 方法来更新模块的 moduleSideEffects 属性。
class Module {
updateOptions({
meta,
moduleSideEffects,
syntheticNamedExports
}: Partial<PartialNull<ModuleOptions>>): void {
if (moduleSideEffects != null) {
this.info.moduleSideEffects = moduleSideEffects;
}
if (syntheticNamedExports != null) {
this.info.syntheticNamedExports = syntheticNamedExports;
}
if (meta != null) {
Object.assign(this.info.meta, meta);
}
}
}执行完 生成模块依赖图、排序模块执行顺序 之后,会正式进入 tree-sharking 阶段,具体的逻辑实现在 includeStatements 方法中。
class Graph {
async build(): Promise<void> {
timeStart('generate module graph', 2);
await this.generateModuleGraph();
timeEnd('generate module graph', 2);
timeStart('sort and bind modules', 2);
this.phase = BuildPhase.ANALYSE;
this.sortModules();
timeEnd('sort and bind modules', 2);
timeStart('mark included statements', 2);
this.includeStatements();
timeEnd('mark included statements', 2);
this.phase = BuildPhase.GENERATE;
}
}includeStatements
rollup 以用户的配置项(即 options.input) 和 implictlyLoadedBefore 作为入口,通过 BFS 算法遍历各个入口模块的所有依赖项,遍历到的依赖项模块需要执行(即 module.isExecuted = true)。
function markModuleAndImpureDependenciesAsExecuted(
baseModule: Module
): void {
baseModule.isExecuted = true;
const modules = [baseModule];
const visitedModules = new Set<string>();
for (const module of modules) {
for (const dependency of [
...module.dependencies,
...module.implicitlyLoadedBefore
]) {
if (
!(dependency instanceof ExternalModule) &&
!dependency.isExecuted &&
(dependency.info.moduleSideEffects ||
module.implicitlyLoadedBefore.has(dependency)) &&
!visitedModules.has(dependency.id)
) {
dependency.isExecuted = true;
visitedModules.add(dependency.id);
modules.push(dependency);
}
}
}
}
class Graph {
private includeStatements(): void {
const entryModules = [
...this.entryModules,
...this.implicitEntryModules
];
for (const module of entryModules) {
markModuleAndImpureDependenciesAsExecuted(module);
}
// 省略下方逻辑
}
}标记完哪些模块需要执行后,则要正式进入 tree-sharking 阶段。
class Graph {
private includeStatements(): void {
// 省略上方逻辑
if (this.options.treeshake) {
// 省略 tree-sharking 逻辑
} else {
for (const module of this.modules) module.includeAllInBundle();
}
// 省略下方逻辑
}
}根据用户配置项(即 options.treeshake)判断是否开启 tree-sharking。若不需要执行 tree-sharking,则对所有模块执行 includeAllInBundle 方法。先来看一下 includeAllInBundle 方法的逻辑。
class Module {
includeAllInBundle(): void {
this.ast!.include(createInclusionContext(), true);
this.includeAllExports(false);
}
}includeAllInBundle 方法做了两件事情,在此之前需要明确一件事情,若 Ast Node 确认为需要包含在最后的产物中则会执行 node.include,后续 ast.render 阶段会跳过这些 未包含 的 Ast 节点,从而达到 tree-sharking 的效果。
调用
ast.include方法,将当前模块的Ast节点标记为 包含。tsclass Program { include( context: InclusionContext, includeChildrenRecursively: IncludeChildren ): void { this.included = true; for (const node of this.body) { if (includeChildrenRecursively || node.shouldBeIncluded(context)) { node.include(context, includeChildrenRecursively); } } } }调用
ast.include的第二个参数includeChildrenRecursively标记为true,意味着在递归模块的Ast节点,会递归包含(确认执行node.include)所有的子孙Ast Node节点。若没有这个标记位,那么Ast Node是否需要包含在最终的代码中是由Ast Node是否存在 副作用 而决定的。jsclass NodeBase { shouldBeIncluded(context: InclusionContext): boolean { return this.included || (!context.brokenFlow && this.hasEffects(createHasEffectsContext())); } }调用
includeAllExports方法。
includeAllExports 方法主要也做了两件事:
function markModuleAndImpureDependenciesAsExecuted(
baseModule: Module
): void {
baseModule.isExecuted = true;
const modules = [baseModule];
const visitedModules = new Set<string>();
for (const module of modules) {
for (const dependency of [
...module.dependencies,
...module.implicitlyLoadedBefore
]) {
if (
!(dependency instanceof ExternalModule) &&
!dependency.isExecuted &&
(dependency.info.moduleSideEffects ||
module.implicitlyLoadedBefore.has(dependency)) &&
!visitedModules.has(dependency.id)
) {
dependency.isExecuted = true;
visitedModules.add(dependency.id);
modules.push(dependency);
}
}
}
}
class Module {
getReexports(): string[] {
if (this.transitiveReexports) {
return this.transitiveReexports;
}
// to avoid infinite recursion when using circular `export * from X`
this.transitiveReexports = [];
const reexports = new Set(this.reexportDescriptions.keys());
for (const module of this.exportAllModules) {
if (module instanceof ExternalModule) {
reexports.add(`*${module.id}`);
} else {
for (const name of [
...module.getReexports(),
...module.getExports()
]) {
if (name !== 'default') reexports.add(name);
}
}
}
return (this.transitiveReexports = [...reexports]);
}
includeAllExports(includeNamespaceMembers: boolean): void {
if (!this.isExecuted) {
markModuleAndImpureDependenciesAsExecuted(this);
this.graph.needsTreeshakingPass = true;
}
for (const exportName of this.exports.keys()) {
if (
includeNamespaceMembers ||
exportName !== this.info.syntheticNamedExports
) {
const variable = this.getVariableForExportName(exportName)[0];
if (!variable) {
return error(logMissingEntryExport(exportName, this.id));
}
variable.deoptimizePath(UNKNOWN_PATH);
if (!variable.included) {
this.includeVariable(variable);
}
}
}
for (const name of this.getReexports()) {
const [variable] = this.getVariableForExportName(name);
if (variable) {
variable.deoptimizePath(UNKNOWN_PATH);
if (!variable.included) {
this.includeVariable(variable);
}
if (variable instanceof ExternalVariable) {
variable.module.reexported = true;
}
}
}
if (includeNamespaceMembers) {
this.namespace.setMergedNamespaces(
this.includeAndGetAdditionalMergedNamespaces()
);
}
}
}- 若模块未标记为已执行,则标记当前模块及其依赖的模块为已执行(即
module.isExecuted = true)。 - 遍历当前模块的
exports和reexports,将所有 导出变量 和 重导出变量 对应的Ast Node节点标记为 包括 (即variable.included = true)。
这里有一个有意思的点是通过 getVariableForExportName 来获取导出和重导出变量所对应的 Ast Node,来看看是如何实现的吧。
tree-sharking
tree-sharking 的执行是在 Chunk 的 render 阶段,此时已经明确了哪些模块需要被执行,哪些 statements 需要保留在最终的产物中。
class Chunk {
// This method changes properties on the AST before rendering and must not be async
private renderModules(fileName: string) {
// 省略其他逻辑
for (const module of orderedModules) {
let renderedLength = 0;
let source: MagicString | undefined;
if (module.isIncluded() || includedNamespaces.has(module)) {
const rendered = module.render(renderOptions);
// 省略其他逻辑
}
}
// 省略其他逻辑
}
async render(): Promise<ChunkRenderResult> {
const {
accessedGlobals,
indent,
magicString,
renderedSource,
usedModules,
usesTopLevelAwait
} = this.renderModules(preliminaryFileName.fileName);
// 省略其他逻辑
}
}由 生成 chunks 这篇文章可知,一个 chunk 对应至少一个 module。因此上述的源码逻辑中可以看出 Rollup 会依次遍历 chunk 中所有的 module,然后依次调用每一个模块的 render 方法来生成最终的代码。
注意
chunk 中所包含的 所有模块 是按照模块执行顺序依次排序的,存储在 orderedModules 数组中。
const compareExecIndex = <T extends OrderedExecutionUnit>(
unitA: T,
unitB: T
) => (unitA.execIndex > unitB.execIndex ? 1 : -1);
function sortByExecutionOrder(units: OrderedExecutionUnit[]): void {
units.sort(compareExecIndex);
}
class Bundle {
private async generateChunks(): Promise<void> {
// 省略其他逻辑
for (const { alias, modules } of inlineDynamicImports
? [{ alias: null, modules: includedModules }]
: preserveModules
? includedModules.map(module => ({
alias: null,
modules: [module]
}))
: getChunkAssignments(
this.graph.entryModules,
manualChunkAliasByEntry,
experimentalMinChunkSize,
this.inputOptions.onLog
)) {
sortByExecutionOrder(modules);
const chunk = new Chunk(
modules,
this.inputOptions,
this.outputOptions,
this.unsetOptions,
this.pluginDriver,
this.graph.modulesById,
chunkByModule,
externalChunkByModule,
this.facadeChunkByModule,
this.includedNamespaces,
alias,
getHashPlaceholder,
bundle,
inputBase,
snippets
);
chunks.push(chunk);
}
// 省略其他逻辑
}
// 省略其他逻辑
}可以看到 tree-sharking 的执行时机是在 module.render(renderOptions) 函数中。再深入研究一下 module.render(renderOptions) 函数的逻辑。
class Module {
render(options: RenderOptions): {
source: MagicString;
usesTopLevelAwait: boolean;
} {
const source = this.magicString.clone();
this.ast!.render(source, options);
source.trim();
const { usesTopLevelAwait } = this.astContext;
if (
usesTopLevelAwait &&
options.format !== 'es' &&
options.format !== 'system'
) {
return error(logInvalidFormatForTopLevelAwait(this.id, options.format));
}
return { source, usesTopLevelAwait };
}
}从这里可以了解到 Rollup 中的 TLA 语法并不支持非 ESM 和 system 的输出格式。关于 TLA 语法的具体实现方案可以参考 TLA 的详细讲解与实现。
除此之外 module.render(renderOptions) 函数中会调用 module.ast.render(source, options) 方法,从而执行 tree-sharking 操作。
this.ast!.render(source, options) 方法的逻辑如下:
class Program {
render(code: MagicString, options: RenderOptions): void {
let start = this.start;
if (code.original.startsWith('#!')) {
start = Math.min(code.original.indexOf('\n') + 1, this.end);
code.remove(0, start);
}
if (this.body.length > 0) {
// Keep all consecutive lines that start with a comment
while (
code.original[start] === '/' &&
/[*/]/.test(code.original[start + 1])
) {
const firstLineBreak = findFirstLineBreakOutsideComment(
code.original.slice(start, this.body[0].start)
);
if (firstLineBreak[0] === -1) {
break;
}
start += firstLineBreak[1];
}
renderStatementList(this.body, code, start, this.end, options);
} else {
super.render(code, options);
}
}
}逻辑很简单,依次 render 当前模块对应的 Ast 的 body 节点,通过 renderStatementList 方法来渲染每一条语句。换句话说 renderStatementList 方法就是 tree-sharking 的核心逻辑所在。
function renderStatementList(
statements: readonly StatementNode[],
code: MagicString,
start: number,
end: number,
options: RenderOptions
): void {
let currentNode, currentNodeStart, currentNodeNeedsBoundaries, nextNodeStart;
let nextNode = statements[0];
let nextNodeNeedsBoundaries = !nextNode.included || nextNode.needsBoundaries;
if (nextNodeNeedsBoundaries) {
nextNodeStart =
start +
findFirstLineBreakOutsideComment(
code.original.slice(start, nextNode.start)
)[1];
}
for (let nextIndex = 1; nextIndex <= statements.length; nextIndex++) {
currentNode = nextNode;
currentNodeStart = nextNodeStart;
currentNodeNeedsBoundaries = nextNodeNeedsBoundaries;
nextNode = statements[nextIndex];
nextNodeNeedsBoundaries =
nextNode === undefined
? false
: !nextNode.included || nextNode.needsBoundaries;
if (currentNodeNeedsBoundaries || nextNodeNeedsBoundaries) {
nextNodeStart =
currentNode.end +
findFirstLineBreakOutsideComment(
code.original.slice(
currentNode.end,
nextNode === undefined ? end : nextNode.start
)
)[1];
if (currentNode.included) {
if (currentNodeNeedsBoundaries) {
currentNode.render(code, options, {
end: nextNodeStart,
start: currentNodeStart
});
} else {
currentNode.render(code, options);
}
} else {
treeshakeNode(currentNode, code, currentNodeStart!, nextNodeStart);
}
} else {
currentNode.render(code, options);
}
}
}renderStatementList 方法中会根据 statement 的 Ast Node 是否需要(node.included) 来决定是否调用 node.render 方法还是 treeshakeNode 方法。
若节点不需要包裹在最终产物中(即
node.included = false)那么就会通过
treeshakeNode方法来执行删除这块代码的逻辑操作。tsfunction treeshakeNode( node: Node, code: MagicString, start: number, end: number ): void { code.remove(start, end); node.removeAnnotations(code); }若节点需要包裹在最终产物中(即
node.included = true)那么就会通过
node.render方法递归 Ast 树并做细节处理。逻辑主要包含对于源代码的细节调整,比如替换变量名、删除未使用的代码块(递归调用renderStatementList)、去除所需的export ast节点对应的源码的export关键字、句末添加分号等。
