On Feb 26, Testflight in it's original form will shut down as a means to send out builds to your iOS test group. Here's what you need to know:
Why
Apple bought Testflight to take the pain out of being an App tester. It's a long, technical and traditionally very complicated process. Downloading an App to test and provide feedback is now pretty darn simple and similar to how things work in the App store. Testers accept an invite, download the Testflight app on their device, and use their iTunes credentials to login, send feedback, etc. No new accounts, registration, or fancy provisioning permissions goodness. Bish, bash, bosh.
Internal Testers vs Beta Testers
You get 25 internal testers. Internal testers are resources on your development, admin and marketing teams that will have itunesconnect access with permissions. When a testing build is submitted, this internal testing group will have immediate access to your app (like you're used to).
You get 1000 beta testers. This is where your external testers will go. The catch here is that when you submit a build for them to test, Apple must approve your build to be sent to testers. Yes, you read that right. Apple has to approve your beta to testers, which is subject to the normal 4-9 day approval times. That may be a deal breaker for many of you. HOWEVER, there is a silver lining. Once the initial Beta is approved, some app developers/apps will be granted permissions to send immediate builds to their beta team. Translation: If your app isn't sketchy, in theory you'll get to bypass this approval process.
30 day expiration
New builds to the test group only live for 30 days before they expire. You can submit a new build before that expiration but your testers and your team must be active to keep a steady stream of logs. For apps that are looking to track long-term user behavior, clocks or lifestyle tracking, this may be a deal breaker.
iOS 8.0+ only
If you need to regression test for devices that are not iOS 8.0 compatible, you'll need to work with either a different or second testing suite. For iPad applications, the iPad 1 is the only tablet not supported. For iPhones and iPods this is going to be a much different story. iOS 8 has a 71% marketshare on iOS devices but know that some of your testers will have to upgrade their devices to be able to test your apps. That may be conclusive for many. This may be a good time to look at your analytics to see what your the makeup of your userbase is. iOS 9.0 so you can delay your panic for another 9 months or so.
Next steps for teams:
- Export any .ipa and log files you want to keep for legacy purposes
- Export your testing group in Testflight
- Import your testing group into the Beta Testers Tab in iTunes Connect
- Create a new distribution provisioning profile with the beta tester designation (this is new radio button). Once this is done, simply build against it and submit the app to iTunes Connect per usual
- Inform your users about the change in process and that they'll be getting a new invite in their email
The BIG Takeaway
Decide on a testing strategy NOW. If you don't, you'll lose all your testing group contact information, your current app builds and logs. That could be a huge setback in the effeciency of your team. There are other cross-platform solutions you could use to continue your immediate build workflow like http://hockeyapp.net/features/ across both Android, iOS and Microsoft (Disclaimer: Microsoft just bought this company so it's future may also be a little fuzzy).
Keep in mind you balance any solution with it's ease of use to setup users, send them builds and get effective feedback. At Eight Bit, on average 1 in 10 testers follow through with signing up and installing apps from our startups. We're always looking for ways to optimize this workflow and for iOS, this may still be the drink of choice. Just keep in mind that you're not just making a technical decision, you're making an experience decision for your testing group.
A short history lesson on testing iOS apps
When iPhone app development first started, to test apps with any size team used to be a long and complicated process:
A tester had to plug in your device to a computer, go into a settings area of itunes, right click on the name of the device, copy and paste the UDID and send it to the developmer. Then all that went into a spreadsheet that was imported or entered one-by-one into iTunes Connect. Provisioning profiles had to be emailed, installed on the devices often involving deleting old profiles, restarting devices and mountains of troubleshooting. Oh you want crash logs? Ugh, here we go again. It was insane.
After a year or so, companies began to emerge that helped ease this process. Testflight was one of them. Finally, you had a place where users could send testers to register their device that would automatically do most of this. It still required installing some things and registered but it was nothing compared to the old system. For a little while Testflight was handling both Android and iOS development and developers were loving their lives. Apple then purchased the company and immediately discontinued support for Android. #shocking
About the author
John W Ostler (@seahostler) is Co-Founder and Principal of UX & UI at Eight Bit Studios (@eightbitstudios), an award winning mobile strategy, design and development studio that focuses on strategizing and executing the visions of Chicago's most promising startups. He has helped lead and produce technical and interaction design engagements with brands like Cadbury Adams, Burger King, Motorola, Career Builder, Groupon, Exelon, Sidley Austin, and HSBC. His work has been featured in the USA Today, Brandweek, The Daily Beast, NewYork Times Tech blog, Mashable.com, featured on multiple CSS design blogs, and had multiple apps reach in the Top 25 and receive a "New and Noteworthy" nod from Apple. He is also Co-Founder of Bughouse (@bughousekids) a kids app, toy, game, books startup.
