[Meego-qa] Netbook-IA-Pinetrail-Sanity Test Report for 1.2 Trunk:Netbook -1.1.80.4.20101102.1

Tao, Daniel daniel.tao at intel.com
Wed Nov 3 01:41:59 PDT 2010 

Quality Summary

Sanity Pass ratio is 86.2% and the whole quality of image is downgraded (89.7%->86.2%) compared with 1026 image because of  a regression issue that fail to mount USB/SD cards(9291<http://bugs.meego.com/show_bug.cgi?id=9291>) . 3 new normal issues are reported in today's image. Also several 1.1 release block bug fixes are not integrated into trunk image. The boot time is increased 12 seconds compared with 1.1 final image.

Improvement:

 *   Banshee's version is updated from 1.7.3-2.55 to 1.7.3-3.3, and several 1.1 release block bug fix is integrated;

Key Issues:

 *   The boot time is increased 7 seconds (9527<http://bugs.meego.com/show_bug.cgi?id=9527>);
 *   Fail to mount USB/SD cards (9291<http://bugs.meego.com/show_bug.cgi?id=9291>);
 *   Show white snapshot in favorite pages(9626<http://bugs.meego.com/show_bug.cgi?id=9626>);

Testing Summary
Total TCs

Passed

Failed

Not testable

Run rate

Pass rate of total

Pass rate of executed


29


25


4


0


100%


86.2%


86.2%


Performance data:

 *   Boot time is 30.596s on HDD disk, 16.251s on SSD disk

Testing Detail
Test Objective

Sanity test is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in sanity test reflects the most important and basic functionality of the distribution. Basic performance indicators are also collected in sanity test, including: boot time, shutdown time, suspend time, resume time, and logs will also be collected to help analyzing performance issues if there are any.

Image Location/Kickstart file

meego-netbook-ia32-1.1.80.4.20101102.1.img<http://repo.meego.com/MeeGo/builds/1.1.80/1.1.80.4.20101102.1/netbook/images/meego-netbook-ia32/meego-netbook-ia32-1.1.80.4.20101102.1.img>

Test Environment

 *   Hardware: Pinetrail Netbook: Dell INSPIRON mini 1012/1G memory/1.66Ghz/160G HDD; Diamondville Netbook: Asus Eeepc901/1G memory/1.6Ghz/12G SSD
 *   Test equipment: Wireless AP
 *   Used Operator: N/A

Detail Test Result
Category

Check points

Previous Trunk Result

Current Trunk Result

Notes (bugs)

Create Image

Create livecd type image by MIC2 on Fedora10 host

Pass

Pass

n/a

Install image

Boot and install

Pass

Pass

n/a

System startup & shutdown

Boot from live USB stick

Pass

Pass

n/a

Boot from local disk

Pass

Pass


Stopwatch:
28.842s on Dell mini;

15.449s on Eeepc
Bootchart:
30.596s on Dell mini

16.251s on Eeepc

Shutdown

Pass

Pass


5.685s on Dell Mini

Suspend & resume


Suspend via closing lid and cmd

Pass

Pass


4.232s on Dell Mini

Resume

Fail

Fail

4.512s on Dell Mini
9189<http://bugs.meego.com/show_bug.cgi?id=9189> - fail to resume from S3 on Dell Mini1012

Powertop Log

Cn state avg residency and Wakeup times

Pass

Pass


C4 mwait  4.7ms (47.7%)
Wakeups/60Sec :183.4

on DELL mini

Stability

Shutdown and Reboot system 10 times without failure or crash

Pass

Pass

n/a

Use multiple applications (>8, media player, browser, IM, email, and so on) based on scopes defined in other checkpoints

Pass

Pass

Network/connectivity - ethernet, wifi connection basic check

Check Ethernet connection

Pass

Pass

n/a

Wifi scan & associate (check from carrick)

Pass

Pass

n/a

UI shell - dropdown panels basic check , zones switch, application launcher to launch key applications

Dropdown panel: click icons on drop down panel to check corresponding panels could launched (Will not test the functionality of panels to avoid to many tests)

Pass

Pass

n/a

Zones: switch in zone panel; drag application to another zone; switch by alt+tab

Pass

Pass

n/a

App-launcher: check if the UI is correct; check if key applications (browser, media, MeeGo Help, settings) could be launched;

Fail

Fail


9101<http://bugs.meego.com/show_bug.cgi?id=9101> - Language and Time&Date pages in settings menu: missing content

Myzone - web service, event, favorite apps basic check

Check if UI is correct;

Fail

Fail

9102<http://bugs.meego.com/show_bug.cgi?id=9102> - 3 icons missing in setting UI

Check if web services could be fetched;

Pass

Pass

n/a

Check if new dates&tasks could be displayed

Pass

Pass

9650<http://bugs.meego.com/show_bug.cgi?id=9650> - Unable show Chinese task in myzone panel

Check media content, browser history and recent docs

Pass

Pass

n/a

Internet Applications -- Multi-tabs websites surfing and flash plugin check

Open 2 tabs to surf 2 websites

Pass

Pass

n/a

Check if flash plugin could basically work

Pass

Pass

n/a

Multimedia -- audio/video/photo playback, and play views switch

Check if the UI is correct (top bar, side bar, play queue)

Pass

Pass

n/a

Check if sample audio/video/photo could playback successfully.

Pass

Pass

n/a

Check if thumbnail/index of sample picture/video works

Pass

Pass

n/a

Device Panel

Check if the device panel could basically work

Pass

Pass

n/a

Peripheral devices (touchpad, USB, SD, LCD)

Touchpad

Pass

Pass

6176<http://bugs.meego.com/show_bug.cgi?id=6176> - occasionally touch pad unable to work in some netbooks (Eeepc 1005pe, 901, HP mini etc.)

USB keyboard/mouse

Pass

Pass

n/a

External monitor

Pass

Pass

n/a

USB stick/SD/LCD

Pass

Fail

9291<http://bugs.meego.com/show_bug.cgi?id=9291> - unable to mount new volume on MeeGo netbook

Basic linux command such as yum install/zypper install/debuginfo-install

zypper install/default repo configuration

Pass

Pass

n/a

Issue Summary

 *   Top bugs (4):

o    9291<http://bugs.meego.com/show_bug.cgi?id=9291> - [REG]Fail to mount USB/SD cards

o    9102<http://bugs.meego.com/show_bug.cgi?id=9102> - 3 icons missing in setting UI

o    9101<http://bugs.meego.com/show_bug.cgi?id=9101> - Language and Time&Date pages in settings menu: missing content

o    6176<http://bugs.meego.com/show_bug.cgi?id=6176> - occasionally touch pad unable to work in some netbooks (Eeepc 1005pe, 901, HP mini etc.)

 *   New bugs (3):

o    9650<http://bugs.meego.com/show_bug.cgi?id=9650> - Unable show Chinese task in myzone panel

o    9626<http://bugs.meego.com/show_bug.cgi?id=9626> - White snapshot in favorite pages

o    9615<http://bugs.meego.com/show_bug.cgi?id=9615> - Fail to chart between two or more persons with IM (log in with jabber account)

 *   Verified bugs (0):

o    N/A
Thanks,

Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-qa/attachments/20101103/5a4a61cb/attachment-0001.html>

More information about the MeeGo-qa mailing list