Testing with Vitest Browser Mode
I’m going to say admit something personal… I’ve never written a test in Playwright. I know, I know… embarrasing (not!).
I’ve always liked the idea of Playwright, but when it comes times to spin up a new project, I always find myself opting out. Here’s why:
npx playwright install installs 3.5GB worth of browser binaries.
This is a minor nitpick, but it’s always rubbed me the wrong way. Why is this the default behavior? Why not just ask which ones I want? –and where are they going?? Somewhere in the bowels of my hard drive, I’m sure. I don’t want to know.
To top it off, these browsers don’t install with npm install, so that’s an extra step in the setup process.
Speed
(I’ll be the first to admit, this is kinda crap.)
Just about every “Jest vs Playwright” thread I’ve seen felt it was necessary to state that Playwright tests are inherently slower. This is obvious because “chrome eat memory go brrr”, and only an nincompoop would use it for unit tests.
…Is this actually true, though? Sure, the overhead of spinning up a headless browser vs. Node is probably… non-negligible, but you’re only spinning up one, right?
Since starting my current job, I’ve come to the heart-breaking realization that most people’s tests suck anyway. They’re slow, brittle, and probably stuck somewhere between “mocking hell” and “doesn’t run in parallel”. Be real, Chromium is rarely the bottleneck.
Portability
It’s fairly easy to write a test runner for whatever environment you want using Jest or Vitest. While the whole point of Playwright is to run tests in your browser, you still need Jest/Vitest if you want to run tests in Node.
Enter Browser Mode
A couple months ago, I was going down a doom-spiral with @testing-library/svelte. The problem is that, in short, it doesn’t work well. In this case, the fireEvent API seems to be completely broken with Svelte. Whilst parting the ocean of stale GitHub issues, I stumbled upon Vitest’s browser mode.
TL;DR: @vitest/browser wraps Playwright (but supports other platforms too), and lets you write Playwright-like tests while integrating with your existing Vitest setup (which integrates with your existing Vite setup).
This… isn’t really much of a selling point over @testing-library/svelte. Sure enough, with a little elbow grease, I was able to get my Svelte tests running.
The Good
Configuration in Monorepos
I hesitated to put this under “good”, because it’s still a pain in the ass, and documentation leaves something to be desired. Nonetheless, not needing yet another config file for every… single… package is a small victory. Vite actually does a fantastic job handling nested configs, and even lets you combine package-level vite.config.tss with a root-level vitest.config.ts without any issues (again, with enough fussing around). Why might you want to do that? Well, Scott Spence has a great article on his Vitest browser testing setup, where he sets up distinct environments based on file extension (.svelte.test.ts/.test.tsx vs. .test.ts). I absolutely love this idea. Combine that with some root-level TSConfig magic (omit tests files at package level, re-include them at root), and you’ve got a perfectly type-safe testing suite! How often does that happen??? Never, that’s how often. Now, your mind can rest, knowing you’ll never risk seeing TypeError: Cannot read properties of undefined (reading 'window') ever again.
(This pattern also works great if you have packages that don’t rely on Vite; you can still run them in Vitest without any exta configuration, and still run npx vitest at the package level)
Thank You, Todd Howard
No more mocking browser APIs, period.
Since you’re not relying on JSDOM and whatnot, things tend to work without a lot of configuration. If you’re using a fullstack framework, you’re still going to hit the wall of “how do I get my server running for tests?”, but at least you’ve got one less thing to worry about.
Speed
150 tests in <2 seconds. It’s totally fine. Maybe this becomes more of an issue on dinosaur hardware, but GitHub actions handles it just fine.
Screenshots
Vitest boasts features like “visual regression testing” (which… sounds cool, but I’m not sure how I feel about it). However, I love the test failure screenshots. That’s so cool. No more guessing which dependency you forgot to mock, or remembering “how to read console log from Playwright cli output”.
I had an “ah-ha!” moment for this when I was running a test on a form. I had put “2.5” into a number input. Previously this passed, but now it failed. Before my very eyes, a screenshot appeared, showing input alongside a popup:
Please enter a valid value. The two nearest valid values are 2 and 3.
Who knew it could be this easy?
The Bad
Configuration
I know I said this was a “good”, but it’s still a pain in the ass. The documentation is OK, but fairly sparse. It also suffers from a lot of package churn, so most tutorials are slightly out-of-date. It’s manageable, a time waster.
The default configuration feels a bit opinionated, with these defaults:
- Browsers run in UI mode
- Failure screenshots enabled
- Screenshots go in seemingly-arbitrarily placed
__screenshots__folders
Visual Regression Testing
This isn’t really Vitest’s fault, but I felt it was important to mention.
Despite my earlier comment, this feature is really cool in the right place. Unfortunately, it’s a tad awkward to set up. You might be shocked to hear that each browser/platform renders things slightly differently (I was shocked and appalled, too). Vitest is aware of this, and suffixes these screenshot files with this metadata. So, unless your entire team (you) only ever use one system/platform, these tests are useless for local dev. As well, seeding these screenshots for a CI environment can be a pain.
Locator API
This isn’t that bad. In fact, I prefer the limitations of the locators, as they force you to make your tests more resilient. However, it is a separate API with considerable buy-in.
Integration with Other Environments
Since Vitest supports different platforms here, I was really hoping to see setups for Chrome Extensions, Electron, and whatnot. Unfortunately, this is still WIP, and I haven’t seen any serviceable examples in the wild yet. I’m currently neck-deep in a VSCode extension, and have been forced to deal with the abysmal testing situation there. Something like this would be a godsend.
The Ugly
Honestly, I don’t have anything to put here.
Conclusion
Over 9 months since launch, I’m happy to see browser mode remain stable. I’m not 100% sold on it yet, but using it is a lot more pleasant than the typical JSDOM shenanigans. I could definitely see this becoming the testing standard in the future (especially with VoidZero going all-in on the ecosystem).