In a recent article posted by Amol Ker of Netflix, he shared his struggles with performing device testing of their mobile apps and websites. Since they operated on a hybrid approach using HTML5 based coding. In the article Amol shared the following “when Netflix decided to enter the Android ecosystem, we faced a daunting set of challenges: a) We wanted to release rapidly every 6-8 weeks, b) There were hundreds of Android devices of different shapes, versions, capacities and specifications which need to playback audio and video and c) We wanted to keep the team small and happy. Of course, the seasoned tester in you has to admit that these are the sort of problems you like to wake up to every day and solve. Doing it with a group of other software engineers who are passionate about quality is what made overcoming those challenges even more fun.”
He continues… “To put device diversity in context, we see almost around 1000 different devices streaming Netflix on Android every day. We had to figure out how to categorize these devices in buckets so that we can be reasonably sure that we are releasing something that will work properly on these devices.”
And later mentions “We keep our team lean by focusing our full time employees on building solutions that scale and automation is a key part of this effort. ”
Netflix’ issues are not new and are not specific to Netflix. Anyone looking to test their apps or websites MUST perform real-world, real device testing before release AND after it’s launched to support forthcoming updates. Netflix has chosen a way that works for them that has potential flaws, in that it can easily miss most of the device OS ecosystem if they don’t test on a certain device or platform. And as budgets and efficient-driven teams demand, utlizing mobile test automation software is key to their strategy.
Keynote, with it’s acquisition of DeviceAnywhere has developed several tools to address these issues helping thousands of companies become more efficient and enabling them to centralize their testing efforts while maintaining dispersed, cost-efficient development teams around the world. With DeviceAnywhere Test Planner, QA managers and developers can get free assistance on determining which devices they should test on that will provide them the widest coverage while testing on the lowest number of devices, providing the ultimate in efficiency and planning of any new rollout.
Keynote also offers automated mobile testing with it’s TCE – Automation platform – leveraging the latest HTML5 web-based coding for ease-of-use giving you the flexibility of running the same test across any HTML5-enabled device or tablet. No more need to ship devices around or leverage agent-based systems whereby one does not get a true experience from the consumer’s perspective, resulting in potential pitfalls for the developer and in result potentially affecting revenues.