We have good automated testing, but it’s not visual. It just checks for certain elements and text on the page. We always perform manual testing as well which can be time consuming and still doesn’t always find every little display error.
One visual testing tool is https://percy.io/, a Californian company. It works by comparing screenshots of predefined pages. All pages that change in appearance have to be approved by one our developers or testers. Unexpected side effects are more likely to be noticed at this stage.
- Can reduce manual testing time.
- Can decrease shipped bugs and display errors.
- Can make testing more fun.
- Requires a tester to think which pages would be best to track.
- Requires a developer to mark the pages to track. This is one task in the beginning and is later maintained with the rest of the automated tests.
- The service has a monthly fee.
- It still can’t catch all bugs since we can’t capture every page and they may be depending on production data.
- Testers have to be careful not to rely on the tool too much since it can’t catch all cases.
The basic version costs $149 per month and comes with 10,000 screenshots per month. Last month we had around 250 automated test builds. Which would mean that we can define 40 pages to take screenshots of. We can start with a 14 day trial. Update: We could get a free plan, see below.
I asked the support about a discount for charities or free software.
Yes! We’d be happy to make something work for you. What is the project?
Our open source plan is pretty new, but we plan to sponsor some open source applications if they meet some qualifications, namely not-for-profit and without corporate sponsorship. If the organization has some reasonable way to pay for Percy, we’d ask that they do so and we’d offer a discounted plan.
We don’t actively advertise it, but we do have a “free” plan for tiny users—once you complete the 14-day trial of the Growth plan, things will keep working and you’ll be moved to our limited-use Free plan that includes 500 visual diffs per month.
So, feel free to start a trial and give it a go! If you decide you want to use Percy in your open source app at the end, you might fit into the free plan or we can set something up for you depending on your usage.