I've built this kind of system before for more than one company. What you want to build is a continuous integration system (and this is what my company specializes in). This allows you to trigger tests each time a developer submits code. The test you describe would fire up several environments in...
Just like most things, the answer is a bit of an "it depends" situation. Generally speaking the more your app functionality/features/experience is tied to calls and SMS then the more probabilistic it is you should need to test the interruptions. FWIW my firm has helped launch dozens of mobile ...
I have been in this situation as well as almost every other web application developer and I learned my lesson along the way. You said you and 3 others tested, but I feel that 4 people may not be enough. I think at least 50 to 75 people is a much more sufficient number and here is how I do it wit...