[meego-releases] MeeGo-handset-ARM-N900-Acceptance Test Report for Trunk:Testing-20100906
srikanth.4.yarlagadda at nokia.com
srikanth.4.yarlagadda at nokia.com
Mon Sep 6 02:30:59 PDT 2010
Test Objective
-----------------------
Acceptance test is a very brief run-through of the functionality of the entire MeeGo distribution, to assure that the basic health of the distribution and report major regressions at the earliest time. All the checkpoints in acceptance test reflects the most important and basic functionalities of the distribution. We do the acceptance test against Trunk: Test repo.
Build (Image)
-----------------------
Internal Image:- meego-handset-armv7l-n900acceptance-1.0.90.2.20100906.1-raw.tar
Trunk: Testing repo (http://download.meego.com/testing ) to create the sanity test image against Trunk: Testing
Test Environment
-----------------------
*
Hardware: NOKIA N900
* Test equipment (bluetooth devices, wlan devices, network simulators, sim simulators etc): wireless AP
Quality Summary
---------------------------
Highlights:
* Bluetooth settings can be found.
* Me application is implemented in People application.
* The issue some buttons are rendered as red blocks in Video player is fixed.
* Contextual menu of sharing photos appeared in Photos application.
* Telephony can be found in connectivity of settings.
Lowlights:
* Bug 4021<http://bugs.meego.com/show_bug.cgi?id=4021>- N900: Unable launch Browser
* Bug 5419<http://bugs.meego.com/show_bug.cgi?id=5419>- Video clip playback failed.
* Bug 5292<http://bugs.meego.com/show_bug.cgi?id=5292>- N900: Music player missing
* Bug 3551<http://bugs.meego.com/show_bug.cgi?id=3551>- xterm doesn't launch correctly and doesn't work
* Bug 4048<http://bugs.meego.com/show_bug.cgi?id=4048>- Not able to shut down the Device (N900) via power button
* Bug 4049<http://bugs.meego.com/show_bug.cgi?id=4049>- Unable to detect USB device after plugging it in computer
* Bug 3670<http://bugs.meego.com/show_bug.cgi?id=3670>- [SMS] The SMS history fail to show.
* Bug 3921<http://bugs.meego.com/show_bug.cgi?id=3921>- [REG] Some buttons (App-launcher, Home, Close) and app icons (Settings/Lock icons) are rendered as red blocks
* Bug 4985<http://bugs.meego.com/show_bug.cgi?id=4985>- [REG]touchscreen calibration issue
* Bug 5651<http://bugs.meego.com/show_bug.cgi?id=5651>- VKB exited automatically after tapping some characters in it
* Bug 5790<http://bugs.meego.com/show_bug.cgi?id=5790>- Bluetoth settings does not display properly after tapping Bluetooth from Settings
* Bug 5031<http://bugs.meego.com/show_bug.cgi?id=5031>- ssh authentication fails over usb network
* Bug 5850<http://bugs.meego.com/show_bug.cgi?id=5850>- It takes long time to launch core applications at the first time.
* Bug 5995<http://bugs.meego.com/show_bug.cgi?id=5995>- Settings and Mail applications are missing in application menu in landscape mode
* Bug 5997<http://bugs.meego.com/show_bug.cgi?id=5997>- The bottom row applications' icons are not fully displayed in portrait mode
* Bug 5301<http://bugs.meego.com/show_bug.cgi?id=5301>- N900: Runtime status not shown on toolbar
* Bug 5650<http://bugs.meego.com/show_bug.cgi?id=5650>- Pictures cannot be seen in Photos viewer
* Bug 6085<http://bugs.meego.com/show_bug.cgi?id=6085>- The character "0" and another two symbols are not fully displayed in virtual keyboard in landscape mode
* Bug 6083<http://bugs.meego.com/show_bug.cgi?id=6083>- VKB does not hide automatically after opening hardware keyboard
* Bug 3291<http://bugs.meego.com/show_bug.cgi?id=3291>- [REG] Wifi applet, connect to network is unstable, 1/2 failure rate
* Bug 6324<http://bugs.meego.com/show_bug.cgi?id=6324>- Bluetooth cannot find nearby devices sometimes.
* Bug 6329<http://bugs.meego.com/show_bug.cgi?id=6329>- Settings application crashes when Bluetooth pairing with other devices.
* Bug 3380<http://bugs.meego.com/show_bug.cgi?id=3380>- [REG] X button to close running application live thumbnail window is not always visible
* Bug 5988<http://bugs.meego.com/show_bug.cgi?id=5988>- Close button on full screened social networking window doesn't work
* Bug 6026<http://bugs.meego.com/show_bug.cgi?id=6026>- The email launched from quick lanch bar and app-lancher are not consistent
* Bug 5953<http://bugs.meego.com/show_bug.cgi?id=5953>- [QMF] account creation does not work
* Bug 6320<http://bugs.meego.com/show_bug.cgi?id=6320>- Close button does not work when email application window is in full screen
Performance data:
* Bootup time: 63s
* Setting: 8s
* photo start time: 6s
* people start time: 12s
* Video player start time: 13s
* SMS start time: 12s
* Dialer start time: 15s
* Email start time: 7s
* Social Network start time: 7s
* Chat start time: 13s
Test Result
-----------------------
Result Overview
-----------------------
Total TCs Passed Failed Not testable Executable rate Pass rate of total Pass rate of executed
36 12 20 4 88.89% 33.33% 37.50%
Detailed Test Results
---------------------------
Category Check points Result Notes (bugs)
Create Image Create image with MIC2 Pass
System startup Image boot from SD card Pass
Home screen Check if core applications* (Dialer, SMS, fennec browser, photo viewer, audio player, video player, contacts, email, Terminal) can be launched from app-launcher Fail Bug 5850 - It takes long time to launch core applications at the first time.
Bug 3551 - xterm doesn't launch correctly and doesn't work
Bug 4021 - N900: Unable launch Browser
Bug 5292 - N900: Music player missing<http://bugs.meego.com/buglist.cgi?quicksearch=5850%2C3551%2C4021%2C5292>
Switch between different applications Pass
Check home screen theme and layout Fail Bug 5995 - Settings and Mail applications are missing in application menu in landscape mode.
Bug 5997 - The bottom row applications' icons are not fully displayed in portrait mode.
Bug 3921 - [REG] Some buttons (App-launcher, Home, Close) and app icons (Settings/Lock icons) are rendered as red blocks<http://bugs.meego.com/buglist.cgi?quicksearch=5995%2C5997%2C3921>
Display orientation detection Pass
Fennec browser Browsing website over wifi (WEP secured by default, WPA is WEP is not testable) Fail Bug 4021 - N900: Unable launch Browser <http://bugs.meego.com/show_bug.cgi?id=4021>
Browsing website over 3G (WCDMA) N/A Not support in meego 1.1 release.
Browser gesture support Fail Bug 4021 - N900: Unable launch Browser <http://bugs.meego.com/show_bug.cgi?id=4021>
Dialer Dial one outgoing call, accept by other party, terminate this call (phonesim, GSM & WCDMA) (GSM and WCDMA cannot be covered until real modem supported,Will use phonesim to test before real modem support) Pass Only Dialer application launching . SIM function is not implemented yet.
Display PhoneBook Pass
Check call history Pass Only can check call out history.
SMS Send SMS ( phonesim, 2G & 3G ) (2G and 3G cannot be covered until real modem supported,Will use phonesim to test before real modem support) Pass SMS application can be launched and SMS can be composed, SIM function is not implemented to test send/receive message function.
Check message history (received, sent, draft) Fail Bug 3670 - [SMS] The SMS history fail to show.<http://bugs.meego.com/show_bug.cgi?id=3670>
Connectivity Scan wifi, check wifi connection (open/wep/wpa) and disconnection Fail Bug 3291 - [REG] Wifi applet, connect to network is unstable, 1/2 failure rate<http://bugs.meego.com/show_bug.cgi?id=3291>
Browse website and download some files through wifi Fail Bug 4021 - N900: Unable launch Browser <http://bugs.meego.com/show_bug.cgi?id=4021>
Check bluetooth connection (pairing and connect to bluetooth handset) Fail Bug 6324 - Bluetooth cannot find nearby devices sometimes.
Bug 6329 - Settings application crashes when Bluetooth pairing with other devices.<http://bugs.meego.com/buglist.cgi?quicksearch=6324%2C6329>
Connect device to computer with usb cable, check the usb mass storage device is detected by computer Fail Bug 4049 - Unable to detect USB device after plugging it in computer <http://bugs.meego.com/show_bug.cgi?id=4049>
Check sshd connection: connect board using usb gadget Fail SSH connection only can be established if usb cable is connected before device boot up.
Bug 5031 - ssh authentication fails over usb network<http://bugs.meego.com/show_bug.cgi?id=5031>
Run time status/Settings/Control Panel Check if runtime status are correct on tiny toolbar/runtime status panel Fail Bug 5301 - N900: Runtime status not shown on toolbar <http://bugs.meego.com/show_bug.cgi?id=5301>
Virtual keyboard Check if virtual keyboard can be launched automatically with apps which need input text field (fennec, SMS) Pass
Support text input Fail Bug 5651 - VKB exited automatically after tapping some characters in it.
Bug 6085 - The character "0" and another two symbols are not fully displayed in virtual keyboard in landscape mode.
Bug 6083 - VKB does not hide automatically after opening hardware keyboard.<http://bugs.meego.com/buglist.cgi?quicksearch=5651%2C6085%2C6083>
Photo viewer Check if the UI is correct (top bar, menu, button, etc.) Pass
Check if photo thumbnail index works Fail Bug 5650 - Pictures cannot be seen in Photos viewer. <http://bugs.meego.com/show_bug.cgi?id=5650>
Check if photo could be played Fail Bug 5650 - Pictures cannot be seen in Photos viewer. <http://bugs.meego.com/show_bug.cgi?id=5650>
Audio Check if the UI is correct (top bar, menu, button, play queue etc.) Fail Bug 5292- N900: Music player missing<http://bugs.meego.com/show_bug.cgi?id=5292>
Check if music could be played (WAV, AAC & MP3 format ) on loudspeaker (play music by command line if audio player is not ready. aplay to play music file) Fail Bug 5292- N900: Music player missing<http://bugs.meego.com/show_bug.cgi?id=5292>
Video Check if the UI is correct (top bar, menu, button, play queue etc.) Pass
Check if video thumbnail index works Fail Bug 5419- Video clip playback failed.<http://bugs.meego.com/show_bug.cgi?id=5419>
Check if video could be played (MP4, WMV & 3GP-H264) Fail Bug 5419- Video clip playback failed.<http://bugs.meego.com/show_bug.cgi?id=5419>
Contacts Check if it works with dialer, SMS Pass
Security & Power Check PIN activation/deactivation (Need GUI support) N/A
PIN requested at boot N/A
Power Off / Power On Fail Bug 4048- Not able to shut down the Device(N900) via power button.<http://bugs.meego.com/show_bug.cgi?id=4048>
Touchscreen Use touch screen to operate the system Fail Bug 4985 - [REG]touchscreen calibration issue<http://bugs.meego.com/show_bug.cgi?id=4985>
Installation Use zypper to install a package N/A Blocked for wifi applet connect to network is unstable.
Issue Summary
---------------------------
Top bugs:
http://bugs.meego.com/buglist.cgi?quicksearch=4021%2C5419%2C5292%2C3551%2C4048%2C4049%2C3670%2C3921%2C4985%2C5651%2C5031%2C5850%2C5995%2C5997%2C5301%2C5650%2C6085%2C6083%2C3291%2C6324%2C6329
New bugs:
http://bugs.meego.com/buglist.cgi?quicksearch=6324%2C6329
Acceptance test trends
---------------------------
Detailed Acceptance Test Report
---------------------------
http://wiki.meego.com/Quality/HandsetTestReport/N900Acceptance20100906
Thanks and regards
Srikanth Yarlagadda
MeeGo Testing RST
Cell:-+358 ( 0)50 487 3519
"Try and fail but don't fail to try"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100906/cbf2ee92/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Picture (Device Independent Bitmap) 1.jpg
Type: image/jpeg
Size: 42543 bytes
Desc: Picture (Device Independent Bitmap) 1.jpg
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100906/cbf2ee92/attachment-0001.jpg>
More information about the Meego-releases
mailing list