Firebase functions emulator does not work with Yarn 2

[cjmyles]

[REQUIRED] Environment info

firebase-tools:
8.2.0

Platform:
macOS

Node:
10.16.0

Yarn:
2.0.0-rc.33

[REQUIRED] Test case

When generating, installing and running a fresh Typescript version of a functions project using the firebase-tools CLI tool and Yarn 2, 20 errors are produced when executing yarn serve - 3 unique:

Cannot find module 'express-serve-static-core'.
Cannot find type definition file for 'node'.
'functions' is declared but its value is never read.

I can resolve these in the following way, but the resolution causes other errors which are not curable:

1. yarn add @types/node -D
2. yarn add @types/express-serve-static-core -D
3. Modify src/index.ts to export the helloWorld function present in the comments.

This causes the following CLI output:

i  Starting emulators: ["functions"]
✔  functions: Emulator started at http://localhost:5001
i  functions: Watching "/Users/craig/dev/sandbox/firebase-functions-yarn-2" for Cloud Functions...
⚠  Error: Cannot find module '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip/node_modules/firebase-admin/package.json'
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
Require stack:
- /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:957:15)
    at Function.module_1.Module._resolveFilename (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12283:54)
    at Module._load (internal/modules/cjs/loader.js:840:27)
    at Function.module_1.Module._load (/Users/craig/dev/sandbox/firebase-functions-yarn-2/.pnp.js:12160:36)
    at Module.require (internal/modules/cjs/loader.js:1019:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at /usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:123:32
    at Generator.next (<anonymous>)
    at fulfilled (/usr/local/lib/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:4:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
⚠  Your function was killed because it raised an unhandled error.

I can confirm that the zip file '/Users/craig/dev/sandbox/firebase-functions-yarn-2/.yarn/cache/firebase-admin-npm-8.11.0-40c4d739ef-3.zip' contains the path 'node_modules/firebase-admin/package.json'.

I tried downgrading to firebase-tools v8.0.3 as per this issue to no avail.

I tried upgrading Yarn to use the latest sources version to no avail.

I tried using a newer version of Node (v12.16.3) to no avail.

I tried using a newer version of tslint (v6.1.2) to no avail.

Could be an issue with the PnP resolver (https://classic.yarnpkg.com/en/docs/pnp/troubleshooting)? Although I tried running the emulator using the following command, and the error still prevailed.

NODE_OPTIONS="--require ./.pnp.js" firebase emulators:start --only functions

I've created a repository containing the generated project so it's as minimal as possible: https://bitbucket.org/cjmyles/firebase-functions-yarn-2/src. I've included firebase-debug.log as well as the Yarn 2 Zero Install files.

[REQUIRED] Steps to reproduce

1. Install Yarn 2.
2. Execute firebase init functions from the CLI - choose a Firebase project; Typescript (Y); TSLint (Y); install dependencies (N).
3. Execute yarn to install dependencies.
4. Execute yarn serve to see the first set of errors.

[REQUIRED] Expected behavior

Generated project should execute without issues, same as if the project's dependencies were installed and the project executed using Yarn 1.

[REQUIRED] Actual behavior

See test case above.

[samtstern]

Thank you @cjmyles for the really detailed feedback. This is all way over my level of JS knowledge, I'm hoping @abeisgoat can chime in.

[cjmyles]

Thanks @samtstern. Interestingly, as soon as I add express to the mix (as per this basic example), I get another error:

../../.yarn/$$virtual/firebase-functions-virtual-bbad2b487f/0/cache/firebase-functions-npm-3.6.1-9aae15f59c-3.zip/node_modules/firebase-functions/lib/handler-builder.d.ts:21:123 - error TS2307: Cannot find module 'express-serve-static-core'.

Running yarn add @types/express @types/express-serve-static-core -D doesn't fix the issue.

So a few Yarn 2 related issues here. I'd like to use Yarn 2 as it resolves transitive dependencies correctly (I have a React/Functions monorepo for example using Yarn workspaces, which references "core" packages on my machine and Yarn 1 has link issues with libraries such as React which need to exist singularly but still be referenced in both), it's a lot quicker than Yarn 1 and offers some other nice features. I'd imagine a lot of people will be migrating over.

[cjmyles]

@samtstern @abeisgoat Any ideas?

[abeisgoat]

Will take a look this week and report back. I'm not familiar with the differences between yarn and yarn@2 so it'll take a moment I'm sure

[cjmyles]

@abeisgoat did you get a chance to take a look? Yarn 2 is a life saver for me in terms of the transitive dependency resolution feature - I have a core React library that can be imported into multiple other React projects no problem, all running within Workspaces. The missing piece of the jigsaw is getting Firebase to run properly in the same setup, so any help would be greatly appreciated.

[cjmyles]

@samtstern @abeisgoat any update on this? Or is Yarn 2 something that shouldn't be used with Firebase?

[samtstern]

@cjmyles I'll take a look at this today. We do a lot of manual module resolution in the emulator and I expect that PNP is tripping us up.

[samtstern]

Yikes PNP is doing some really nasty stuff, like patching fs so it looks inside ZIPs:

https://yarnpkg.com/features/pnp#packages-are-stored-inside-zip-archives-how-can-i-access-their-files

[samtstern]

Ok I got realllllly far with this but yarn got in my way at the very end:
yarnpkg/berry#1467

I have a few workarounds possible but none are great so I would like for the Yarn2 team to get back to me first.

[samtstern]

@cjmyles I have a work in progress here:
#2356

Could you try this:

1. Clone that branch
2. Run npm install && npm build && npm link to install my branch of the Firebase CLI as the firebase command on your machine
3. See if you're able to load your Yarn2 project in the emulator?

[samtstern]

After a bunch of discussion in #2356 we realized that there's no safe way for us to support Yarn2 with PnP in the local emulators that won't break in production. But we're going to add a helpful warning to make this less confusing.