Challenges faced in Compatibility Testing

Comment

Comment by Divya on February 29, 2012 at 2:17pm

Vishal, thanks for the valuable feedback :)

>> Last thing I want to share, when you are going
to test any app or Game for mobile/wireless device,
it should be tested thoroughly on real device instead
of any simulator or cloud device.

-- Agreed but as we wont be having that much devices on hand, we have to go for
simulator/cloud options

Comment by Vishal Kishor Sakharkar on February 29, 2012 at 1:51pm

Hi Divya,

  

Tips - You should be ready with one sheet which is having popular devices, screen size and os
version details , i/p methods (one sheet for one os). Check your apps supported os/display/ i/p method
Select the popular devices (hit device like in BB - 9900, 9700, 8520 ) in your list based on the criteria then do testing.

Yes Divya, your all four points are valid and every qa faced those while testing. But we need to make some pre-requisites before testing any App or Game. Those can be:

1. Confirm and make the list of Technology you are supporting.
BB
Android
iOS

2. Make the Resolution list according to technology you are supporting, like BB, iOS and Android.
e.g.
BB - 320x240
360x480
480x800

(The above are latest screen resolutions for BB)

3. Make your own checkist points for Keypad, Touch and Touch n' Type device and clasify it.

4. OS version may differs in BB, but we have to test the app in every OS version if requirement is there.

5. Run the valid Testing Criteria for each technology.

The above things can be help you to sort out the confusion in compatibility testing.

@ Vaibhav: - Great Sharing.

Creation of Compatibility Test Suite :- Compatibility testing should not involve more than 10% of your regression test suite. i.e. if you have 100 test cases in your regression suite then 10 or max 15 test cases should be there in compatibility test suite.

Please elaborate this point, becasue I havn't tested any app or game via Automation.

Last thing I want to share, when you are going to test any app or Game for mobile/wireless device, it should be tested thoroughly on real device instead of any simulator or cloud device.

Make correct if I went wrong.

Regards,
Vishal.

Comment by Divya on February 29, 2012 at 10:39am

Thanks Mani, Anurag,

Vabhiav,  valuable info u have provided , thanks a lot :)

Regards,

Divya

Comment by Anurag Khode on February 28, 2012 at 9:19am

Good Info!Thanks Vaibhav,Divya!

Comment by Vaibhavraj Jadhav on February 28, 2012 at 6:30am

Hi Divya,

Excellent write up. My 2 cents on this is as below -

Creation of Compatibility Test Suite :- Compatibility testing should not involve more than 10% of your regression test suite. i.e. if you have 100 test cases in your regression suite then 10 or max 15 test cases should be there in compatibility test suite.

 

Features to be tested in Compatibility Test Suite :- As we have already completed functional testing, in turn there should not be any transcational based testing i.e. if its Banking app then deposit/ withdrawal.

Testing should be carried out around specific to UI features, device specific feature (e.g. Back Button in BB and Back Button in Android and only one key on iOS devices).In UI feature, you can differantiates apps pages as Tier 1 pages, Tier 2 pages etc. Tier 1 pages (page which gets max hits by user ) should be excellent fit for all devices.  

Color code of the pages should be uniform through all the devices.

 

Getting the right set of Devices for Testing :- From your user base, you need to select the top devices

 

Biggest challange is to get the devices on desired carrier, we use to face such sceanrio in each product release. Only way to overcome this is Go for cloud based solutions. (where Cost matters ).

As per my knowledge, Perfecto is the only cloud based provider which gives devices on Indian carrier. DA doesnt.

 

Hope this helps. Please correct if something is wrong.

 

Thanks,

Vaibhav

Comment by nagendhran mani on February 27, 2012 at 5:47pm

good information......

Comment by Divya on February 27, 2012 at 4:47pm

For android, If you want to check the app is compabitiobile wid various devices, you can use 

testdroid cloud tool