[meego-releases] MeeGo Netbook v1.1(trunk) Sanity Test Report - 20100921
Tao, Daniel
daniel.tao at intel.com
Sat Sep 25 03:41:51 PDT 2010
MeeGo Netbook v1.1 (trunk) Sanity Test Report - 20100921
Test Objective
Sanity test is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in sanity test reflects the most important and basic functionality of the distribution. Basic performance indicators are also collected in sanity test, including: boot time, shutdown time, suspend time, resume time, and logs will also be collected to help analyzing performance issues if there are any.
Image Location/Kickstart file
meego-netbook-ia32-1.0.90.3.20100921<http://repo.meego.com/MeeGo/builds/trunk/1.0.90.3.20100921.1/netbook/images/meego-netbook-ia32/meego-netbook-ia32-1.0.90.3.20100921.1.img>.1.img
Test Environment
* Hardware: Pinetrail Netbook: Dell INSPIRON mini 1012/1G memory/1.66Ghz/160G HDD; Diamondville Netbook: Asus Eeepc901/1G memory/1.6Ghz/12G SSD
* Test equipment: Wireless AP
* Used Operator: N/A
Quality Summary
Highlights:
* Sanity checkpoints pass ratio: 93.1%;
* The image quality keeps stable (93.1%->93.1%) compared with 0917 image;
* Boot time is decreased 3 seconds (22.359s ->19.183s);
* Proxy settings in repository configuration files is fixed (5822<http://bugs.meego.com/show_bug.cgi?id=5822>);
Lowlights:
* software Update : A security signature is not present is re-opened (6189<http://bugs.meego.com/show_bug.cgi?id=6189>);
* [REG]unable to import data from usb-disk by click "import media" button in device panel (7394<http://bugs.meego.com/show_bug.cgi?id=7394>);
* Failed to resume from S3 sometimes on dellmini1012(40%) (3932<http://bugs.meego.com/show_bug.cgi?id=3932>);
* Played media history will not be shown in myzone(29<http://bugs.meego.com/show_bug.cgi?id=29>);
New Features
* N/A
Performance data:
* Boot time is 19.183s on HDD disk, 12.814s on SSD disk
Test Result
Result Overview
Total TCs
Passed
Failed
Not testable
Run rate
Pass rate of total
Pass rate of executed
29
27
2
0
100%
93.1%
93.1%
Detailed Test Results
Category
Check points
Result
Notes (bugs)
Create Image
Create livecd type image by MIC2 on Fedora10 host
Pass
n/a
Install image
Boot and install
Pass
System startup & shutdown
Boot from live USB stick
Pass
n/a
Boot from local disk
pass
Stopwatch:
17.909s on Dell mini;
12.596s on Eeepc
Bootchart:
19.183s on Dell mini
12.814s on Eeepc
Shutdown
Pass
4.735s on DELL mini
Suspend & resume
Suspend via closing lid and cmd
Pass
2.175s on DELL mini
Resume
Fail
3.359s on DELL mini
3932<http://bugs.meego.com/show_bug.cgi?id=3932> - Failed to resume from S3 sometimes on dellmini1012(40%)
Powertop Log
Cn state avg residency and Wakeup times
Pass
C4 mwait 4.2ms (36.3%)
Wakeups/60Sec :113.1
On DELL mini
Stability
Shutdown and Reboot system 10 times without failure or crash
Pass
n/a
Use multiple applications (>8, media player, browser, IM, email, and so on) based on scopes defined in other checkpoints
Pass
n/a
Network/connectivity - ethernet, wifi connection basic check
Check Ethernet connection
Pass
n/a
Wifi scan & associate (check from carrick)
Pass
n/a
UI shell - dropdown panels basic check , zones switch, application launcher to launch key applications
Dropdown panel: click icons on drop down panel to check corresponding panels could launched (Will not test the functionality of panels to avoid to many tests)
Pass
Zones: switch in zone panel; drag application to another zone; switch by alt+tab
Pass
n/a
App-launcher: check if the UI is correct; check if key applications (browser, media, MeeGo Help, settings) could be launched;
Pass
n/a
Myzone - web service, event, favorite apps basic check
Check if UI is correct;
Pass
n/a
Check if web services could be fetched;
Pass
n/a
Check if new dates&tasks could be displayed
Pass
n/a
Check media content, browser history and recent docs
Fail
29<http://bugs.meego.com/show_bug.cgi?id=29> - Played media history will not be shown in myzone
Internet Applications -- Multi-tabs websites surfing and flash plugin check
Open 2 tabs to surf 2 websites
Pass
n/a
Check if flash plugin could basically work
Pass
n/a
Multimedia -- audio/video/photo playback, and play views switch
Check if the UI is correct (top bar, side bar, play queue)
Pass
n/a
Check if sample audio/video/photo could playback successfully.
Pass
n/a
Check if thumbnail/index of sample picture/video works
Pass
n/a
Device Panel
Check if the device panel could basically work
Pass
Peripheral devices (touchpad, USB, SD, LCD)
Touchpad
Pass
n/a
USB keyboard/mouse
Pass
n/a
External monitor
Pass
6173<http://bugs.meego.com/show_bug.cgi?id=6173> - [REG]Unable to auto-switch the monitor
USB stick/SD/LCD
Pass
n/a
Basic linux command such as yum install/zypper install/debuginfo-install
zypper install/default repo configuration
Pass
n/a
Issue Summary
* Top bugs
* 3932<http://bugs.meego.com/show_bug.cgi?id=3932> - Failed to resume from S3 sometimes on dellmini1012(40%)
* 29<http://bugs.meego.com/show_bug.cgi?id=29> - Played media history will not be shown in myzone
* 5591<http://bugs.meego.com/show_bug.cgi?id=5591> - Banshee & media panel hang when play a same video twice
* 6570<http://bugs.meego.com/show_bug.cgi?id=6570> - [Chromium]It will cause the Chromium window close when do Import Bookmarks and Settings
* 3880<http://bugs.meego.com/show_bug.cgi?id=3880> - Empty list in language settings window
* 3881<http://bugs.meego.com/show_bug.cgi?id=3881> - "date&time"setting application has nothing displayed
* 6816<http://bugs.meego.com/show_bug.cgi?id=6816> - Failed to select some languages during the system installation because languages name are shown as unreadable codes
* New bugs
* 7394<http://bugs.meego.com/show_bug.cgi?id=7394> - [REG]unable to import data from usb-disk by click "import media" button in device panel
* Re-opened bugs
* 6189<http://bugs.meego.com/show_bug.cgi?id=6189> - software Update : A security signature is not present
* Verified bugs
* 5822<http://bugs.meego.com/show_bug.cgi?id=5822> - Proxy settings in repository configuration files
Detailed Acceptance Test Report
http://wiki.meego.com/Quality/NetbookTestReport/NetbookSanity20100921
Thanks
MeeGo Netbook QA team
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100925/a03be99e/attachment-0001.html>
More information about the Meego-releases
mailing list