This is an interesting post on the value crowdsourced testing can offer.
As any software developer or tester knows, it’s impossible to identify every user issue in a controlled test environment. For mobile apps especially, there are a near-limitless number of permutations― combinations of devices, firmware, operating systems, and networks. As a result, it simply isn’t feasible to test every possible scenario. It’s also impossible for lab testers who are already familiar with an app’s anticipated behavior to experience it from the perspective of a neophyte.
To help resolve this problem, I can attest that one of the most effective ways is through crowdsourced testing. Crowdsourced testing, also called crowd testing, has been around for many years. Now, though, the power of databases and analytics makes it possible for vendors to create highly targeted pools of up to thousands testers who meet specific criteria. It also allows building and maintaining accurate profiles and ratings on these testers over time.
In such a scenario, crowd testers are neither random nor unpredictable. Because crowd testers are usually paid or remunerated in some fashion, and because their future work hinges on producing accurate results, they have a vested interest in performing the tests or demonstrations expediently and correctly. (Testers usually report the majority of defects within the first few hours of the test.) This approach gives enterprises that engage in crowdsourced testing greater confidence in the reliability of the outcomes.