[meego-releases] MeeGo v1.1 Core 2010-09-27 Test Report (for 1.0.90.4.20100924.1 images)

Mon Sep 27 01:50:30 PDT 2010

Test Objective

--------------------------------------

It's a weekly testing cycle for preview images released by distribution team to ensure MeeGo core middlewares provides stable HW and usage model independent application services and APIs for building the vertical specific user experience. Besides testing for OS middleware (Ofono, Connection Manager, Bluetooth, Package Manager, Tracker, Pulseaudio, Libsocialweb, Buteo, Geoclue, Gupnp and Gfx), we also did basic testing for OS Base (Kernel and device drivers) on top of Aava and CDK. Our testing focus would be basic function testing and regression testing, regression testing set is identified based on change and bug fixing to verify if there are any regressions introduced by changes.

Build Location
---------------------------------------

Aava: http://repo.meego.com/MeeGo/builds/trunk/1.0.90.4.20100924.1/handset/images/meego-handset-ia32-aava-mtf/

Test Environment
---------------------------------------
Hardware:
Aava
Pinetrail Netbook
Test equipment: bluetooth devices, wireless AP

Test Coverage
---------------------------------------
Test types:
Regression test
Bug verification
Regression testing set is identified based on packages changed:

1.0.90.3.20100921.1

1.0.90.4.20100924.1

PackageKit

0.6.7-3.11

0.6.7-4.8

Connman

0.59-1.9

0.60.1-1.2

contextkit-meego-providers

0.0.8-1.6

0.0.10-1.3

Libsocialweb

0.25.4-1.5

0.25.5-1.1

Kernel

2.6.35.3-7.1

2.6.35.3-8.1

Libdrm

2.4.21-1.4

2.4.21~git7ec9a1effa-1.2

Pvr-bin

5.3.0.0016-3.1

5.3.0.0016-4.1

X11-server

1.9.0-2.5

1.9.0-3.5

ofono

0.26-1.23

0.26-2.2

paulseaudio

0.9.21-16.19

0.9.21-17.3

tracker

0.9.16-1.17

0.9.16-2.2

Test Result
---------------------------------------
Total TCs

Passed

Failed

Not testable (Blocked)

Run Rate

Pass rate of total

Pass rate of executed

444

406

36

2

99.5%

91.4%

91.8%

Quality Summary

Highlights:

Ø  X boots up successfully in 0924 image on iCDK with bug 7050<http://bugs.meego.com/show_bug.cgi?id=7050> - [MFLD CDK] X fails to startup on MFLD CDK was verified.

Ø  The playback quality issue on CDK is not reproducible on AAVA(6353<http://bugs.meego.com/show_bug.cgi?id=6353> - [CDK] Audio playback via pulseaudio is not smooth).

Ø  Some Gfx's major issues were verified, such as 5804<http://bugs.meego.com/show_bug.cgi?id=5804>, 7165<http://bugs.meego.com/show_bug.cgi?id=7165>.
Lowlights:

Ø  One regression issue from camera driver is that the v4l2 interface by ffmpeg doesn't work (7437<http://bugs.meego.com/show_bug.cgi?id=7437>).

Ø  Audio recording not working issue is reproduced on AAVA( 6352<http://bugs.meego.com/show_bug.cgi?id=6352> - Pulseaudio recording functionality works incorrectly).

Issue Summary
--------------------------------------
New issues:

7426<http://bugs.meego.com/show_bug.cgi?id=7426> - [AAVA] unable to playback encoded audio since firmware is not integrated
7437<http://bugs.meego.com/show_bug.cgi?id=7437> - [REG][MRST] ffmpeg will hungup or killed when capture video with V4L2 interface
7334<http://bugs.meego.com/show_bug.cgi?id=7334> - [libsocialweb-qt]The method 'isOnline' is unimplemented in the header file 'swclient.h'
7335<http://bugs.meego.com/show_bug.cgi?id=7335> - [libsocialweb-qt] the method 'hideItem' is unimplemented in the header file 'swclient.h'
7405<http://bugs.meego.com/show_bug.cgi?id=7405> - [libsocialweb-qt]A pointer of a invalid 'SwClientService' instance will be returned even if the service is not available
7464<http://bugs.meego.com/show_bug.cgi?id=7464> - [es1] Many OpenGL ES1 cases of mesa-demo rendering corrupt (i.e. gears_x11)
7457<http://bugs.meego.com/show_bug.cgi?id=7457> - [EGL] MESA_screen_surface functions are not supported by pvr driver
7338<http://bugs.meego.com/show_bug.cgi?id=7338> - [OpenVG] Mesa VG demos are failed duto "symbol lookup error: /usr/lib/egl/egl_dri2.so: undefined symbol: _glapi_get_proc_address"
6994<http://bugs.meego.com/show_bug.cgi?id=6994> - Failed to send out Damage event after GL rendering happens
7474<http://bugs.meego.com/show_bug.cgi?id=7474> - "pkcon install" will not install package without performing "pkcon refresh"

Verified fixed issues:

6880<http://bugs.meego.com/show_bug.cgi?id=6880> - USB send/Receive does not work

5951<http://bugs.meego.com/show_bug.cgi?id=5951> - kernel build error - cy8ctmg110_ts: commit 1b0857fcd88 breaks build

2769<http://bugs.meego.com/show_bug.cgi?id=2769> - [Aava] Need to add Aava South Canyon DV2 patches to the AC tree

6514<http://bugs.meego.com/show_bug.cgi?id=6514> - The loadfw patch didn't get integrated into MeeGo trunk even if it is accepted by AC tree already

7050<http://bugs.meego.com/show_bug.cgi?id=7050> - [MFLD CDK] X fails to startup on MFLD CDK
6622<http://bugs.meego.com/show_bug.cgi?id=6622> - The package 'rest-0.6', required by 'libsocialweb-keyfob', is not found and deprecated
5671<http://bugs.meego.com/show_bug.cgi?id=5671> - The "ItemChanged" signals were emitted even if the cached items are same with the "last" ones
5804<http://bugs.meego.com/show_bug.cgi?id=5804> - X failed to start in QEMU with "-vga std" after xserver upgraded to 1.8.99
7165<http://bugs.meego.com/show_bug.cgi?id=7165> - pvr-bin and psb-video in non-OSS should be ExclusiveArch for X86
7063<http://bugs.meego.com/show_bug.cgi?id=7063> - touchpad is very sensitive on asus pinetrail netbook
6303<http://bugs.meego.com/show_bug.cgi?id=6303> - Screen becomes dark after X start up
375<http://bugs.meego.com/show_bug.cgi?id=375> - "Refresh package lists" doesn't work even if the repos change
746<http://bugs.meego.com/show_bug.cgi?id=746> - refresh/install/update functions in pkcon and gpk applications do not if proxy set
6614<http://bugs.meego.com/show_bug.cgi?id=6614> - pkcon or zypper can't work with external repo and internal repo at the same time
2434<http://bugs.meego.com/show_bug.cgi?id=2434> - pkcon will not get the current status of installed/available packages before some operations
7268<http://bugs.meego.com/show_bug.cgi?id=7268> - X for Handsets contains architecture specific packages
7263<http://bugs.meego.com/show_bug.cgi?id=7263> - MeeGo Handset Base Support contains architecture specific packages

Conclusions and Recommendations From QA
--------------------------------------
>From QA observation, 0924 images quality is improved because X boots up successfully on iCDK and major issue 7050<http://bugs.meego.com/show_bug.cgi?id=7050> - [MFLD CDK] X fails to startup on MFLD CDK was verified, some major issues from Gfx and Package management were also verified(5804<http://bugs.meego.com/show_bug.cgi?id=5804>, 7165<http://bugs.meego.com/show_bug.cgi?id=7165>, 375<http://bugs.meego.com/show_bug.cgi?id=375>). However, one regression issue from camera driver is that the v4l2 interface by ffmpeg doesn't work (7437<http://bugs.meego.com/show_bug.cgi?id=7437>), and Audio recording not working issue still exists on Aava( 6352<http://bugs.meego.com/show_bug.cgi?id=6352>) , these issues need to be fixed ASAP.

MeeGo Core Qualtiy Overview
--------------------------------------
Note:
Below dashboard is try to give you an overall quality view of MeeGo Core. It's based on the latest testing result on these platforms, not just the testing result in this report.

Legend:

Low risk, most functions work well, minor bugs or limited normal bugs

Medium risk, main function work, normal bugs and limited major bugs

High risk, main functions loss, a few critical/major bugs

Components

Grading

Comments (Top Issues)

OS base(kernel and device driver)

Aava Koski

2801<http://bugs.meego.com/show_bug.cgi?id=2801> - [Aava] Battery driver reporting incorrect/incomplete information
2090<http://bugs.meego.com/show_bug.cgi?id=2090> - [aava] Power button on Aava Koski (South Canyon) DV1 device does not shut down system
4062<http://bugs.meego.com/show_bug.cgi?id=4062> - [AAVA] compass driver for YAS529 is not ready(not integrated)
5417<http://bugs.meego.com/show_bug.cgi?id=5417> - [Aava] when booting, user may not find bluetooth device by about 2/3 possibility
6548<http://bugs.meego.com/show_bug.cgi?id=6548> - Accelerometer Driver on MEEGO Build causes more than 240 interrupts/sec

MRST CDK

 236<http://bugs.meego.com/show_bug.cgi?id=236> - [CDK]sound record not work
3861<http://bugs.meego.com/show_bug.cgi?id=3861> - [MRST] unable to find pmic/max7315 gpio, also missing gpio51-63
3983<http://bugs.meego.com/show_bug.cgi?id=3983> - [CDK] isl29020's lux_output value always shows 0
4985<http://bugs.meego.com/show_bug.cgi?id=4985> - [REG][CDK] touchscreen calibration issue
5669<http://bugs.meego.com/show_bug.cgi?id=5669> - [CDK] fail to use gst-launch mrstcamsrc and X crash at the first startup
6548<http://bugs.meego.com/show_bug.cgi?id=6548> - Accelerometer Driver on MEEGO Build causes more than 240 interrupts/sec

MFLD iCDK

Can boot to homescreen successfully, but not run all cases of core on iCDK.

Netbook

3928<http://bugs.meego.com/show_bug.cgi?id=3928> - system suspend automatically after idle(no keystroke or touch) 5 minutes
6205<http://bugs.meego.com/show_bug.cgi?id=6205> - UB6250 SD card unable to be recognized in aspire one
6176<http://bugs.meego.com/show_bug.cgi?id=6176> - occasionally touch pad unable to work in eeepc1005pe

IVI (Russellville and Crown Bay)

3776<http://bugs.meego.com/show_bug.cgi?id=3776> - [REG][ivi/embedded] ethernet interface NOT found
3944<http://bugs.meego.com/show_bug.cgi?id=3944> - [Russellville] Integrate Timberdale driver in Kernel-2.6.35
3896<http://bugs.meego.com/show_bug.cgi?id=3896> - [Crown Bay] Integrate driver for TopCliff IOH in Kernel-2.6.35
5412<http://bugs.meego.com/show_bug.cgi?id=5412> - Russellville: IVI home is unusable through input devices when touchscreen connected

OS middleware

 Gfx

major issues are on handset platform, netbook gfx subsystem works well.
1334<http://bugs.meego.com/show_bug.cgi?id=1334>:Significant visible tearing in GL applications
3959<http://bugs.meego.com/show_bug.cgi?id=3959> - some graphics application can't run
5887<http://bugs.meego.com/show_bug.cgi?id=5887> - mcompositor seg faults on Aava when built against GLESv2 enabled Qt
5074<http://bugs.meego.com/show_bug.cgi?id=5074> - Enable non-root X for handset

 Ofono

Basic functionality work in good state

 Connman

Some issues wait for upstream
6958<http://bugs.meego.com/show_bug.cgi?id=6958> - Fail to reconnect to Palm pre phone after Bt disconnection

 Bluetooth

Basic commands (hciconfig, sdptool, hcitool and l2ping) and basic profiles work.
7390<http://bugs.meego.com/show_bug.cgi?id=7390> - Bluetooth SYNC profile failed to work at GetPhonebook()

 Tracker

1503<http://bugzilla.meego.com/show_bug.cgi?id=1503> - [REG]Can't view bmp&gif photos using meegophotos
4865<http://bugs.meego.com/show_bug.cgi?id=4865> - Tracker-store will occupy more than 90% CPU usage when indexing large numbers of files

 Package manager

Basic functionality works well

 Sensor framework

5932<http://bugs.meego.com/show_bug.cgi?id=5932> - [CDK] Orientation on CDK has wrong direction

PulseAudio

6352<http://bugs.meego.com/show_bug.cgi?id=6352> - Pulseaudio recording functionality works incorrectly
6353<http://bugs.meego.com/show_bug.cgi?id=6353> - [CDK]Audio playback via pulseaudio is not smooth

Contextkit

Framework works, can set properties and fetch from application.

Buteo

3869<http://bugs.meego.com/show_bug.cgi?id=3869> - buteo-sync-plugins: deploy supported services
4945<http://bugs.meego.com/show_bug.cgi?id=4945> - add ScheduleWorld service profile with calendar sync in iCalendar 2.0
6023<http://bugs.meego.com/show_bug.cgi?id=6023> - fix and complete CtCaps for hcontacts and hcalendar
6136<http://bugs.meego.com/show_bug.cgi?id=6136> - /etc/sync/nokia-syncml-conf.xml is missed

Libsocialweb

6158<http://bugs.meego.com/show_bug.cgi?id=6158> - 'Flickr' item will be missed if the photo is smaller than 250 pixels

Geoclue

6669<http://bugs.meego.com/show_bug.cgi?id=6669> - [REG] geoclue network-based providers failed to get position/address

Gupnp

Most functionality works well

Test report is also archived at:
http://wiki.meego.com/Quality/CoreTestReports/weekly20100924_mrst

Thanks,
Xiaolei
MeeGo Core QA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.meego.com/pipermail/meego-releases/attachments/20100927/0e46f603/attachment-0001.html>