[meego-releases] MeeGo v1.1 Handset UX/Applications ww33.1 Test Report (for 1.0.80.14.20100806.1 images)

Mon Aug 9 05:25:21 PDT 2010

Test Objective
--------------------------------------

This is QA weekly testing cycle for weekly images released by distribution team. Our testing focus would be new features exploration and regression testing, to ensure major issues for new features are exposed in timely fashion and changes introduced from the last build do not break working features. The test covers UX and applications (MeeGo Touch Framework, home screen, system UI, Dialer, SMS, fennec browser, and photo viewer, contacts, Settings, connectivity applets etc).

Build Location
---------------------------------------
http://repo.meego.com/MeeGo/builds/trunk/1.0.80.14.20100806.1/handset/images/meego-handset-ia32-aava-mtf/
http://repo.meego.com/MeeGo/builds/trunk/1.0.80.14.20100806.1/handset/images/meego-handset-ia32-shcdk-mtf/

Test Environment
---------------------------------------
Hardware:
Aava Koski DV1 (Firmware: IFWI 2.46.4? Kboot 014)
Moorestown QS CDK with NEC PMIC (Firmware: IFWI 2.54, Kboot 0.10)
Pinetrail Netbook (MSI U130, EeePC 1005PE)
Test equipment: bluetooth devices, wlan devices, network simulators, sim simulators, wireless AP

Test Coverage
---------------------------------------
Test types:
Sanity test
Regression test
Bug verification
Updated UX/Applications Components:
Fennec

Test Result
---------------------------------------
Result Overview

Ø  Aava
Total TCs

Passed

Failed

Not testable (Blocked)

Run Rate

Pass rate of total

Pass rate of executed

UX/Applications: 65

43

13

9

86.15%

66.15%

76.79%

Ø  MRST CDK
Total TCs

Passed

Failed

Not testable (Blocked)

Run Rate

Pass rate of total

Pass rate of executed

UX/Applications: 44

21

13

10

75%

47.72%

61.76%

Quality Summary

Common (apply for both Aava and CDK)
Highlights:

Ø  N/A
Lowlights:

Ø  Pulse audio still does NOT work on Aava and CDK with bug (150<http://bugs.meego.com/show_bug.cgi?id=150>) REOPENED.

Ø  Fennec is updated and several REGRESSION issues are detected for it. Fennec download failed (5073<http://bugs.meego.com/show_bug.cgi?id=5073>), No response when clicking application home button of Fennec (5016<http://bugs.meego.com/show_bug.cgi?id=5016>).

Aava specific highlights/lowlights:

Ø  Wifi driver works on Aava today.

Ø  Audio driver doesn't work on Aava DV1 althought the fw_sst.bin is updated to the latest on(4813<http://bugs.meego.com/show_bug.cgi?id=4813>)

CDK specific highlight/lowlights:

Ø  Audio driver works on MRST CDK.

Ø  Wifi driver still does NOT work on MRST CDK (3625<http://bugs.meego.com/show_bug.cgi?id=3625>).

Detailed Test Results
Sanity test results on Aava:
Total TCs

Passed

Failed

Not testable (Blocked)

Run Rate

Pass rate of total

Pass rate of executed

UX/Applications: 44

24

11

9

79.55%

54.55%

68.57%

Sanity test results on MRST CDK:
Total TCs

Passed

Failed

Not testable (Blocked)

Run Rate

Pass rate of total

Pass rate of executed

UX/Applications: 44

21

13

10

75%

47.72%

61.76%

Legend:

Low risk, most functions work well, minor bugs or limited normal bugs

Medium risk, main function work, normal bugs and limited major bugs

High risk, main functions loss, a few critical/major bugs

Not available/Not integrated

The grading and top issues apply for both Aava and CDK if they are not specified.

Components

Grading

Test Results

Comments (Top Issues)

Home/MeeGo Touch Framework/Theme

Sanity test to cover

3380<http://bugs.meego.com/show_bug.cgi?id=3380> - [REG] X button to close running application live thumbnail window is not always visible
3921<http://bugs.meego.com/show_bug.cgi?id=3921> - [REG] Some buttons (App-launcher, Home, Close) and app icons (Settings/Lock icons) are rendered as red blocks
3927<http://bugs.meego.com/show_bug.cgi?id=3927> - [REG] Black screen then white screen appearing for almost 2 mins before displaying home screen
4066<http://bugs.meego.com/show_bug.cgi?id=4066> - System Tray Icons are Half Displayed

Fennec browser

Total TC: 21
Run rate: 100%
Pass rate: 90.48%

3527<http://bugs.meego.com/show_bug.cgi?id=3527> - [fennec-meego-theme] Can't display thumbnail image of blank tab
5073<http://bugs.meego.com/show_bug.cgi?id=5073> - [REG] Fennec download failed
5076<http://bugs.meego.com/show_bug.cgi?id=5076> - HomePage button on right control-stripe acts as HomeScreen button

System UI

Sanity test to cover

2801<http://bugs.meego.com/show_bug.cgi?id=2801> - [Aava] Battery driver reporting incorrect/incomplete information
3921<http://bugs.meego.com/show_bug.cgi?id=3921> - [REG] Some buttons (App-launcher, Home, Close) and app icons (Settings/Lock icons) are rendered as red blocks

Dialer

Sanity test to cover

3474<http://bugs.meego.com/show_bug.cgi?id=3474> - [REG] No way to dial selected contact in recent, people, favorite views
4067<http://bugs.meego.com/show_bug.cgi?id=4067> - Dialer Tab icons display are half cut from Top
4816<http://bugs.meego.com/show_bug.cgi?id=4816> - The people list filter result is wrong

SMS

Sanity test to cover

2313<http://bugs.meego.com/show_bug.cgi?id=2313> - No sms notification information when a new message arriving

4293<http://bugs.meego.com/show_bug.cgi?id=4293> - The history SMS message widget shakes if there are only 1 to 2 message in history

Virtual Keyboard

Sanity test to cover

3917<http://bugs.meego.com/show_bug.cgi?id=3917>- Unable to launch VKB

Photo viewer

Sanity test to cover

3929<http://bugs.meego.com/show_bug.cgi?id=3929> - [Photo viewer]No album showed in the photo viewer
3960<http://bugs.meego.com/show_bug.cgi?id=3960> - [Photo viewer]The photo can't be full screen in the single photo view after showing the title bar

Music Player

Sanity test to cover

Video Player

Place holder application

Contacts

Sanity test to cover

4811<http://bugs.meego.com/show_bug.cgi?id=4811> - [REG] Can NOT always save edited content

Connectivity Applets

Sanity test to cover

811<http://bugs.meego.com/show_bug.cgi?id=811> - Click Failure network in WIFI network list can crash Settings
3397<http://bugs.meego.com/show_bug.cgi?id=3397> - Wifi applet always shows the same networks many times

3625<http://bugs.meego.com/show_bug.cgi?id=3625> - [MRST] Wifi driver is not enabled

Control Panel

Sanity test to cover

- Connectivity applet testing is blocked by 3625<http://bugs.meego.com/show_bug.cgi?id=3625>
- Look and Feel looks like only an example to show widgets
- For other items, they are all empty

Email

The pkg isn't installed into image

Instant Messanger

The pkg isn't installed into image

Sync client

The pkg isn't installed into image

Issue Summary
--------------------------------------
Key Issues:
Open UX/applications high bugs on Handset: Here<http://bugs.meego.com/buglist.cgi?cmdtype=dorem&remaction=run&namedcmd=handheld_UX_open_high&sharer_id=43>

New issues: 2 new issues are detected today.
5073<http://bugs.meego.com/show_bug.cgi?id=5073> - [REG] Fennec download failed
5076<http://bugs.meego.com/show_bug.cgi?id=5076> - HomePage button on right control-stripe acts as HomeScreen button

Verified fixed issues:  N/A

Reopened bugs: 1 bug is REOPENED today
4016<http://bugs.meego.com/show_bug.cgi?id=4016> - Default favicon image can't be displayed on bookmark list

Conclusions and Recommendations From QA
--------------------------------------

>From QA perspective, 0806 handset images quality is improved because Wifi driver can work on Aava and Audio driver works on CDK. But Wifi driver doesn't work on CDK (3625<http://bugs.meego.com/show_bug.cgi?id=3625>), and audio driver doesn't work on Aava DV1 (4813<http://bugs.meego.com/show_bug.cgi?id=4813>), these issues blocked application testing to some extent and need to be fixed ASAP.

Detailed Test Report
--------------------------------------
http://wiki.meego.com/Quality/HandsetTestReport/UXWeekly20100806

Thanks
--Cathy
MeeGo Handset QA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/private/meego-releases/attachments/20100809/9f54b238/attachment-0001.html>