[meego-releases] MeeGo Netbook v1.1 IA Netbook/Nettop/Notebook System Functional Test Report - 20100914

Tao, Daniel daniel.tao at intel.com
Fri Sep 17 01:15:20 PDT 2010 

MeeGo Netbook v1.1 IA Netbook/Nettop/Notebook System Functional Test Report - 20100914

Test Objective
-----------------------

System Functional Test focuses on testing the entire system and evaluates the typical system usages from end user’s perspective. The test case is a set of combined user operations to cover different usage scenarios and hardware conditions, including peripherals, network environment and etc. The testing will give each test case a mark in 5 levels between 0 and 1 according to score criteria.

Image Location
-----------------------

meego-netbook-ia32-1.0.90.2.20100914.1.img<http://repo.meego.com/MeeGo/builds/trunk/1.0.90.2.20100914.1/netbook/images/meego-netbook-ia32/meego-netbook-ia32-1.0.90.2.20100914.1.img>


Test Environment
-----------------------
Hardware: (Pinetrail based Netbook) HP Mini 210 MSI U130, EeePC 1005PE, (Nettop) MSI AE1900-B, (Notebook) Calpella AspireOne 5740-6025
Test equipment:  wireless AP, BT keyboard, BT earphone, BT mouse, USB disk, SD/MMC cards.

Quality Summary
-----------------------
Highlights:

l  Pass radio is 83.88%;

l  The pass rate is upgraded compared with 0907 image (80.80%->83.88%) with 4 major regression issues are fixed (6159<http://bugs.meego.com/show_bug.cgi?id=6159>, 6139<http://bugs.meego.com/show_bug.cgi?id=6139>, 6151<http://bugs.meego.com/show_bug.cgi?id=6151>, 6305<http://bugs.meego.com/show_bug.cgi?id=6305>) ;

l  QA executed 246 test cases, and submitted 4 new bugs in this test cycle;
Lowlights:

l  We still need to improve the quality of Media player, web browser and setting;

l  Banshee & media panel hang when play a same video twice (5591<http://bugs.meego.com/show_bug.cgi?id=5591>);

l  [Chromium]It will cause the Chromium window close when do Import Bookmarks and Settings (6570<http://bugs.meego.com/show_bug.cgi?id=6570>);

l  Empty list in language settings window (3880<http://bugs.meego.com/show_bug.cgi?id=3880>);

l  “date&time”setting application has nothing displayed (3881<http://bugs.meego.com/show_bug.cgi?id=3881>);

l  Failed to select some languages during the system installation because languages name are shown as unreadable codes (6816<http://bugs.meego.com/show_bug.cgi?id=6816>);
Test Result
-----------------------
Result Overview:
Netbook Results
Total Test Case

Passed

Failed

Not testable

Run rate

Pass ratio

Pass rate of executed

138

n/a

n/a

1

99.28%

83.88%

n/a


Nettop Results
Total Test Case

Passed

Failed

Not testable

Run rate

Pass ratio

Pass rate of executed

59

n/a

n/a

11

81.36%

61.86%

n/a


Notebook Results
Total Test Case

Passed

Failed

Not testable

Run rate

Pass ratio

Pass rate of executed

59

n/a

n/a

4

93.22%

78.81%

n/a

Detailed Test Results:
Pass Ratio Trend for Netbook
[cid:image003.png at 01CB567C.344F3250]
Usage Scenario

Grading

Test Case Number

Average Score

Comments/Top Bugs

Install the image

　

4

0.88

Bug 6816<http://bugs.meego.com/show_bug.cgi?id=6816> - Failed to select some languages during the system installation because languages name are shown as unreadable codes

View photos

　

14

0.98

n/a

Listen to music

　

19

0.82

n/a

Watch video

　

10

0.80

Bug 5591<http://bugs.meego.com/show_bug.cgi?id=5591> - Banshee & media panel hang when play a same video twice
Bug 6228<http://bugs.meego.com/show_bug.cgi?id=6228> - The Banshee would hang with Searching unknown information by Matching Artist or Album during the media playing

Play streaming

　

1

1.00

n/a

Sync media from/to USB/SD

　

2

1.00

n/a

* Capture image/video

　

2

0.63

Bug 1849<http://bugs.meego.com/show_bug.cgi?id=1849> - Cheese will stop recording video automatically when working on a resolution higher than 320x240

* Record voice

　

1

1.00

n/a

Play games

　

3

0.83

n/a

Surf web sites

　

23

0.86

Bug 6556<http://bugs.meego.com/show_bug.cgi?id=6556> - The "Exit" will lead to other Chromium window close
Bug 6570<http://bugs.meego.com/show_bug.cgi?id=6570> - [Chromium]It will cause the Chromium window close when do Import Bookmarks and Settings

View plugin on web

　

3

1.00

n/a

* Chat on IM

　

5

0.75

Bug 6167<http://bugs.meego.com/show_bug.cgi?id=6167> - failed to login MSN by IM client
Bug 3879<http://bugs.meego.com/show_bug.cgi?id=3879> - Notification for gmail failed to live update unless reboot system

Use social network features

　

8

0.84

n/a

* Update/sync personal information

　

6

0.88

n/a

* Use email client

　

8

0.69

Bug 3874<http://bugs.meego.com/show_bug.cgi?id=3874> - Email application auto quit after finishing configuring yahoo account
Bug 6498<http://bugs.meego.com/show_bug.cgi?id=6498> - Email rules don’t work
Bug 5859<http://bugs.meego.com/show_bug.cgi?id=5859> - Evolution require deleted previous account’s password when sending mail

* Print one page

　

1

1.00

n/a

Operate UI shell

　

11

0.77

Bug 3881<http://bugs.meego.com/show_bug.cgi?id=3881> - “date&time”setting application has nothing displayed

Update software

　

4

0.69

Bug 2252<http://bugs.meego.com/show_bug.cgi?id=2252> - cancel update confirmation will lead to abnormal

Use misc utilities and applications

　

13

0.79

Bug 3880<http://bugs.meego.com/show_bug.cgi?id=3880> - Empty list in language settings window


[cid:image002.png at 01CB5670.E9E9CA50]

Issue Summary (4):
Bug 6774<http://bugs.meego.com/show_bug.cgi?id=6774> - Some HotKeys are not working in "Sound Recorder" application
Bug 6776<http://bugs.meego.com/show_bug.cgi?id=6776> - “Changes the visibility of image collection pane in the current window” function is not work when Imageviewer work under "slideshow" mode
Bug 6868<http://bugs.meego.com/show_bug.cgi?id=6868> - sometime, fail to open recent web-pages from myzone thumbnail
Bug 6870<http://bugs.meego.com/show_bug.cgi?id=6870> - Unable to show the connected web-cam in device panel

Indicators:
See more system functional test bugs HERE<http://bugs.meego.com/buglist.cgi?bugidtype=include&bug_id=6305,6167,4994,1810,6173,1723,5591,428,2284,6560,3882,6190,5635,6554,6362,6084,6557,5550,6168,6167,6159,3879,6241,6189,5283,6251,3880,1227,5889,1793,3876,6084,145,1188,5416&query_format=advanced>

Thanks
MeeGo Netbook QA


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100917/1d2d24fe/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 11749 bytes
Desc: image002.png
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100917/1d2d24fe/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 26792 bytes
Desc: image003.png
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100917/1d2d24fe/attachment-0003.png>

More information about the Meego-releases mailing list