Curious about this groups opinion on this topic. In researching automation tools for our iOS mobile project many of the major players in the industry require you to jailbreak the apple device in order to use the automation tool on it (eggPlant, PerfectoMobile, etc). Many at my office feel it does invalidate the testing but it appears that many big mobile players in the industry don't seem to care based on the listed clients of these tools and services.

Thoughts?

You need to be a member of Mobile QA Zone to add comments!

Join Mobile QA Zone

Email me when people reply –

Replies

  • Thanks for the insight Lee. My main concern is the FDA as our solution will have be 510K approved. I'm inquiring with the FDA on this as their draft guidance on mobile medical devices doesn't go that deep yet. I'll post anything I hear back here.

    Lee Barnes said:

    Greg,

    This conversation reminds me of a similar issue in the early 90's when the automated testing vendors often required that special code be compiled in to the application client to allow communication with their tool. Many organizations BLINDLY refused to test their applications this way because it wasn’t being tested in the exact same configuration as it was to be deployed. Many more chose to release their applications with the test tool code present. In the end, the presence of the code had no effect for MOST applications.

    For better or worse, jailbreaking/rooting is likely required to provide any meaningful functionality from an automated testing perspective – especially for iOS devices. You need to understand the impact of a jail-broke device on YOUR application in YOUR environment to determine if the risk level outweighs the advantages of automation. The impact should be examined from more than a technical perspective. For example, regulatory requirements would likely preclude testing on a jailbroke device for a healthcare app requiring validation under FDA regulations.

    The worst thing you could do is to dismiss these tools solely because they require jailbreaking. I’m assuming that since you’re looking into automated tools, the organization has done some level of analysis of the benefit of test automation. Don’t blindly dismiss these potential benefits without first understanding the impact (if any) of jailbreaking a device in your specific situation. Similarly, this analysis will be equally beneficial if it tells you that there is real impact/risk to testing on a jail-broke device.

    Lee Barnes
    www.utopiasolutions.com

  • In my POV , testing an app n a jailbroken device s nt accurate/approximate.In some cases these jailbroken device doesn't support multitasking, also it supports 1ly wireless networks and when it comes for 3G networks it doesn't support.Even in cases the app original functionality gets stuck.So dont try any jailbroken options in the devices..

    Please cr8 me if any !!!!!!!!!!!!!!!!!!!!!!!! Cheers :) 

  • Greg,

    This conversation reminds me of a similar issue in the early 90's when the automated testing vendors often required that special code be compiled in to the application client to allow communication with their tool. Many organizations BLINDLY refused to test their applications this way because it wasn’t being tested in the exact same configuration as it was to be deployed. Many more chose to release their applications with the test tool code present. In the end, the presence of the code had no effect for MOST applications.

    For better or worse, jailbreaking/rooting is likely required to provide any meaningful functionality from an automated testing perspective – especially for iOS devices. You need to understand the impact of a jail-broke device on YOUR application in YOUR environment to determine if the risk level outweighs the advantages of automation. The impact should be examined from more than a technical perspective. For example, regulatory requirements would likely preclude testing on a jailbroke device for a healthcare app requiring validation under FDA regulations.

    The worst thing you could do is to dismiss these tools solely because they require jailbreaking. I’m assuming that since you’re looking into automated tools, the organization has done some level of analysis of the benefit of test automation. Don’t blindly dismiss these potential benefits without first understanding the impact (if any) of jailbreaking a device in your specific situation. Similarly, this analysis will be equally beneficial if it tells you that there is real impact/risk to testing on a jail-broke device.

    Lee Barnes
    www.utopiasolutions.com

  • As am I Tony. Trying to get that explanation myself. Perhaps the change that occurs from jailbreaking/rooting causes the device to be different from a standard consumers and thus runs the risk of behaving differently under test?

  • Simply, there is no way to provide automation framework without jailbreaking/rooting.

    Our point of view as a vendor is that Apple is more concerned about jailbreaking at the consumer level, and not in the enterprise space.

    We do see some concern about it with our customers/prospects, and some have not been able to overcome these concerns.

    I am curious to know what you mean by "invalidate the testing".


    Tony Cannizzo

    www.zap-fix.com

This reply was deleted.
Welcome to Mobile QA Zone, a Next Generation Software Testing Community.Invite your friends to join this community.Write to us to become a featured member.