[Meego-community] Overview of Open Test Service (was: Re: MeeGo Core Test Suite Development Guideline)

[Ville Ilvonen]

Hi,

There was a lot of good discussion around our proposed approach (1) 
while I was busy with other things. Great! I try to open up blind spots 
in the documentation I put in the wiki the last week. Here goes:

XML+tool support has been designed to help us automate testing with 
following scenario:

As a developer and tester
I can write tests with any test tool
and wrap my tests inside test packages
so that I can have test results from open testing system

So you put code (or releases) and tests in and get test results out.

In between (to be open) magic takes place so that:
- software and test packages are build (automated)
- software is updated to target device (automated)
- test cases are executed              (automated)
- test results are made visible for us (automated)

We agreed to call this Open Test Service with Intel and have already 
tool support for it - open sourcing work to publish is ongoing as 
defined at
http://wiki.meego.com/Test_Packaging#Tools_release_schedule

Even when we're working to make this available I'm very careful not to 
promote any particular test tool for people to use but support all of 
them. (I'm not promoting vi for people who want to use emacs).

In practice we have test packages containing tests written with C, C++, 
Python, Perl, javascript, Ruby and shell scripts using tools like 
QTestlib, Check, MIN, cppunit, jsunit, Cucumber and so on. XML test 
definitions are designed to wrap these tests and automated boring 
routines so that there's more time for more productive work like user 
experience testing, development, documentation and so on.

I didn't cover manual testing intentionally - testrunner tool supports 
execution of manual steps but it's mainly targeted for automation.

Open sourcing those tools takes time, please bear with us. I like to be 
realistic and I'm not expecting everyone to welcome these with bells and 
whistles. In the meanwhile join us at #meego-qa-tools (2) and/or 
continue discussion here or at forum.

Regards,

-Ville

(1) http://wiki.meego.com/Test_Packaging
(2) http://wiki.meego.com/Quality/QA-tools