mardi 31 mai 2016

Best way to test apps against test environments

Not sure if this is the right forum and struggling to articulate this questions but here goes:

We have a process in place where we regularly release testflight app builds in order to test new functionality and fixes in our app. We always point these builds at our production environment (API’s, databases, etc). This causes us some issues like:

  • We are regularly trying to test stuff which has not yet been released to production (i.e. backend stuff may have not been updated yet)
  • When we do move code from our backend test environment to our production environment it has potentially not been tested as the app never points there.
  • We dont see what our real production users see as we are always on more advanced builds.

I am sure there are a number of ways we could do this a lot better but a key requirement we have going forward is to be able to release versions of our app which point to our test environment so we can correctly test things end-to-end to help alleviate the above issues. At the same time we also need versions of our app pointing at production as we use the app ourselves day to day.

The two obvious choices that I can come up with are:

Option 1 - Have separate test and production apps

By this I mean we have two apps released with two separate bundle id’s and names (like app_test and app_prod). You can then load both of these apps on your phone at the same time.

Option 2 - Same app built to either point at test or prod

This is basically where I would release the same app pointing at either production or test. You can only have one or the other on your phone at any one time.

Option 1 feels like the right thing to do but I have a number of issues with it:

  • The configuration of push and provisioning certificates will be more complicated

  • Inbound push notifications will be hard to understand and debug on phones if you have the two apps running

  • We would need to either submit a second app to app stores or we will need to use a third party release tool like hockeyapp or whatever.

  • Our semi automated build process would need reworking to support multiple bundle ids and stuff

Option 2 on the other hand presents itself with problems too:

  • You can only have test or production apps on a single device at any one time. This implies we would need to get more devices to test from

  • There is a risk of releasing an app pointing to test environments to our production users.

I guess a 3rd option would be to have the ability to reconfigure the app to use either environment (by way of a hidden dev menu or something). This concerns us in regards to caching settings but maybe that could be worked around. I guess it also means we could only run one version of the app on a device at any one time.

Anyway, this is quite an open question for advice. I am really keen to hear what other people are doing when it comes to testing apps. I have read up a bit on this and spoken to a number of people to get input (which is what leads me to the above two options) but would appreciate more input and advice. How do you solve this problem? Maybe I am just approaching this from the completely wrong angle!

Aucun commentaire:

Enregistrer un commentaire