Todd Wolfson

Software Engineer

November 22, 2014

My website has a test suite with visual regression tests; screenshot different webpages and compare against a set of expected images:

https://github.com/twolfson/twolfson.com/tree/3.40.0/test/perceptual-tests/expected_screenshots

This weekend, I switched it from PhantomJS to node-webkit and am happy with the results:

Comparison

Benefits

  • node_modules/ always work (e.g. no process is undefined issues)
  • node_modules/ are automatically resolved via require
  • Rendering is more accurate (e.g. web fonts load)

Downsides

  • node-webkit has a slow startup time
  • Need to define index.html and package.json to load JS in node-friendly context
    • This can be worked around with a script wrapper (e.g. build a node module for it)

Gotchas

  • There were concurrency issues, we had to throttle it to 2 screenshot runs at the same time
    • Otherwise, we would see white screens sometimes
  • node-webkit@0.10.5 would occasionally draw an underline for a link when all others didn't have one
    • Upon upgrading to node-webkit@0.11.1, these issues went away
  • Need to use Xvfb to exceed maximum desktop screenshot size (e.g. 2000 height on 1080 tall resolution)
  • Need to keep on using Vagrant to match Travis CI renders

Why not atom-shell?

My first attempt was with atom-shell. However, there was an issue with screenshotting in the latest release (0.19.3):

https://github.com/atom/atom-shell/issues/847

The latest version that would capture screenshots (0.17.2) didn't have a mechanism for reading data from a website when node-integration was disabled. This was disabled to verify the website was accurate as possible.

Why not Selenium?

Selenium is would yield the most accurate set of screenshots. However setup, installation, and maintenance are not as easy as node-webkit.

Additionally, I wanted to verify that node-webkit was a viable replacement for future PhantomJS scripts for the benefits listed earlier.

Top articles

Develop faster

Removing the tedium from creating, developing, and publishing repos.

Sexy bash prompt

A bash prompt with colors, git statuses, and git branches.

Recent articles

Parallel visual testing

Comparison of parallelization techniques for visual testing