[Meego-qa] MeeGo Netbook 1.2.0 Cedartrail System Functional & Dataflow Test Report -- 20111118 image

[Pan, LiangX]

Test Objective
-----------------------

System Functional Test focuses on testing the entire system and valuates the typical system usages from end users perspective. The test case is a set of combined user operations to cover different usage scenarios and hardware conditions, including peripherals, network environment and etc. The testing will give each test case a mark in 5 levels between 0 and 1 according to score criteria.

Dataflow is the flow of data to, from and within a device. Inputs are derived from system use cases and each core dataflow test case presents the basic instance of the use case in MeeGo core stack. The result of core dataflow could report the maturity level of MeeGo core os software and hardware integration on specific device. It’s a weekly testing cycle against weekly build.

Image Location
-----------------------
meego-netbook-ia32-20111118.2.img

Test Environment
-----------------------
Hardware: Cedartrail (Dual core)
Test equipment: wireless AP, USB disk, SD/MMC cards

Quality Summary
-----------------------
System functional test pass ratio: 85.11%. whole image quality is basically stable (85.14% ->85.11%), compared with 1111 image. Two old issues(Bug24103<https://bugs.meego.com/show_bug.cgi?id=24103>, Bug23456<https://bugs.meego.com/show_bug.cgi?id=23456>) have been disappeared. Two new issues were found by QA.

•   Bug 24250<https://bugs.meego.com/show_bug.cgi?id=24250> - Click "Fullscreen" button, video stops at one picture

•   Bug 24258<https://bugs.meego.com/show_bug.cgi?id=24258> - Manage apps can't install new packages
Dataflow Test pass ratio: 96.6 %. whole image quality is stable (94.8%->96.6%), compared with 1111 image. One old issue(Bug23129<https://bugs.meego.com/show_bug.cgi?id=23129>) has been disappeared. No new issue was found by QA. But some critical issues still exist, Such as:

•   Bug 22862<https://bugs.meego.com/show_bug.cgi?id=22862> - audio capture(mono) doesn't work well with internal MIC
Test Result
-----------------------
Result Overview:

System Functional Test
Total Test Case

Passed

Failed

Not testable

Run rate

Pass ratio

Pass rate of executed

141

120

21

0

100%

85.11 %

85.11 %


Dataflow Test
Total Test Case

Passed

Failed

Not testable

Run rate

Pass ratio

Pass rate of executed

58

56

2

0

100%

96.6 %

96.6%


Issue Summary:

•         Cedartrail New bugs:

•   [New]Bug 24250 <https://bugs.meego.com/show_bug.cgi?id=24250> - Click "Fullscreen" button, video stops at one picture

•   [New]Bug 24258 <https://bugs.meego.com/show_bug.cgi?id=24258> - Manage apps can't install new packages

•   Bug 23750 <https://bugs.meego.com/show_bug.cgi?id=23750> - The coins of the "Neverball" game shows incorrect

•   Bug 24259 <https://bugs.meego.com/show_bug.cgi?id=24259> - System tray bar covers action buttons of date&time setting Interface.

•   Bug 24257 <https://bugs.meego.com/show_bug.cgi?id=24257> - Gadgets app can't connect to server

•   Bug 23956 <https://bugs.meego.com/show_bug.cgi?id=23956> - Select different themes there is no change

•   Bug 23957 <https://bugs.meego.com/show_bug.cgi?id=23957> - Automatically switch to another application when the second open one tab.

•   Bug 23959 <https://bugs.meego.com/show_bug.cgi?id=23959> - Turn on System tray, Take Screenshot doesn't work well

•   Bug 23960 <https://bugs.meego.com/show_bug.cgi?id=23960> - System tray covers action buttons of Categories dialog

•   Bug 23797 <https://bugs.meego.com/show_bug.cgi?id=23797> - Get through the slideshow of photos is not displayed on myzone panel

•   Bug 23654 <https://bugs.meego.com/show_bug.cgi?id=23654> - [REG]In application panel there are two Internet Browser.

•   Bug 22854 <https://bugs.meego.com/show_bug.cgi?id=22854> - No battery exposed by kernel.

•   Bug 23650 <https://bugs.meego.com/show_bug.cgi?id=23650> - The focus can't scroll down with press down(↓) key and disappeared.

•   Bug 23422 <https://bugs.meego.com/show_bug.cgi?id=23422> - The dropdown menu of setting device type is covered on bluetooth panel.

•   Bug 23753 <https://bugs.meego.com/show_bug.cgi?id=23753> - No audio and vedio's history report display on myzone panel.

•   Bug 23406 <https://bugs.meego.com/show_bug.cgi?id=23406> - Lid switch doesn't work.

•   Bug 23755 <https://bugs.meego.com/show_bug.cgi?id=23755> - Unable to show photo thumbnail on ‘select a file’ panel.


•         Netbook v1.2.0 Old bugs:

 *   Bug 22828<https://bugs.meego.com/show_bug.cgi?id=22828> - Unable to send file by GTalk (Sending/Receiving progress stays at 0%).
 *   Bug 10584 <https://bugs.meego.com/show_bug.cgi?id=10584> - Fail to switch priority between wifi and wired connection.
 *   Bug 22862<https://bugs.meego.com/show_bug.cgi?id=22862>  - audio capture(mono) doesn't work well with internal MIC
 *   Bug 23754<https://bugs.meego.com/show_bug.cgi?id=23754> - Cannot display previous screen after running xrandr command.

Detail Test Result Reference
System founctional Test
Please refer to  http://qa-reports.meego.com/1.2/Netbook/System%20Functional/Cedartrail/3389 for detailed test report

Dataflow Test
Please refer to  http://qa-reports.meego.com/1.2/Core/Dataflow/Cedartrail/3388 for detailed test report


Thanks
Liang



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-qa/attachments/20111123/1d037d7e/attachment-0001.html>