[Meego-qa] New test management tool proposal and workshop in Dublin

Mon Nov 8 04:54:59 PST 2010

On 11/08/2010 10:20 AM, ext Yu, Ling L wrote:
> These existing QA tools have already helped a lot on QA activities, but from MeeGo testers' perspective, there are still some gaps on QA tools:
> - Lack of management tool for feature test traceability, e.g.
> 	Provide mapping between tests and features: from test cases to features and from features to test cases;
> 	Be able to select test set according to features;
> 	Provide meaningful test coverage summary;
>    

Thanks a lot for your input.

This is well covered with our tools (testplanner <-> testrunner).
Newly deployed qa-reports.meego.com doesn't handle this mapping so I created
*Bug 9767* <http://bugs.meego.com/show_bug.cgi?id=9767> - [FEA] 
Feature-mapping between test plan and tests visible in reports

Please see attachment screencaptures at 
http://bugs.meego.com/show_bug.cgi?id=9767
> - Lack of management tool facilitates test execution coordination for all partners involving in testing
> 	e.g. publish test execution plan for a build, assign testers for test execution, merge the results from different testers, and so on
>    
I believe we need open test management meetings to get this started. 
Anyway it's test team leads and managers who coordinate this activity.
Tool can help only when we have testers and test management in open 
meetings, volunteers can participate and tell their interest, i.e. test 
management in place.

Merging several results files into one report is already supported with 
http://qa-reports.meego.com - would you please give it a try?

> - Lack of tool to automate Quality Metrics publishing, e.g
> 	Automatically accumulate test results, show trend for different test category, test platform and etc.
> 	Publish feature test status and result summary, linking to latest results of feature test
>    
Automated publishing to qa-reports.meego.com is not there yet but it's 
very user-friendly - bliss to use. It also shows trend in respect to 
previous round.

Would you please elaborate your feature-wishlist to 
http://bugs.meego.com/enter_bug.cgi?product=Development%20Tools -> 
meego-qa-reports ?

> I propose to discuss these gaps on Dublin workshop.
>    
Sure.

Regards,

-Ville

> Thanks
> -Ling
>
> -----Original Message-----
> From: meego-qa-bounces at lists.meego.com [mailto:meego-qa-bounces at lists.meego.com] On Behalf Of Ville Ilvonen
> Sent: Friday, November 05, 2010 6:38 PM
> To: meego-qa at lists.meego.com
> Subject: Re: [Meego-qa] New test management tool proposal and workshop in Dublin
>
> Hi,
>
> People have requested and more open test management for MeeGo.com so
> here goes my proposal.
>
> Basic principles of MeeGo test management:
>
> 1. Tests are open and available for contributions
> 2. Tests cases are stored in version control
> 3. Test management meetings are open for participation
>
> In practice this could happen so that we store all of our tests in
> version control, like in MeeGo development projects.
> After that MeeGo Quality can follow MeeGo contribution policy and people
> interested in Quality can gain merit with contribution.
> Currently we have test suites available [1,2,3] at gitorious project
> QA-tools, some tests in Testlink which is not open for community :(
> I propose we start using version control for all tests, open for all.
>
> So we end up in tool support for my proposal.
>
> We have open test plan format, test-definition. Idea behind this test
> definition is that it allows testers and developers to run easily any
> tests written with any test tool or language. You can also easily wrap
> existing test suites and automate them with this format.
>
> This format is defined because we use it extensively in fully automated
> testing (test control computer executes all tests automatically). Format
> also supports semi-automated testing (some test steps executed by
> computer, some steps executed by tester) and manual testing (all steps
> executed by tester). Format definition is available [4], documented [5]
> with demo [6]. Format is XML but we have user friendly tool support to
> edit test plans with Testplanner [7]. You can install those tools for
> Ubuntu 10.04 from Tools:Testing repository [8] and try them yourself.
>
> If you use these tools for test management they provide result format
> which can be used with MeeGo QA reporting system [9]. It's being taken
> into use officially the next week, November 8th. Intent is that test
> reports would NOT be sent to meego-qa with large attachements [10].
>
> Finally, making MeeGo Quality, Test Management and Bug Triaging meetings
> open allows people to follow activities and contribute.
>
> So there was my proposal, all comments welcome. Practices and processes
> are being ramped up so many things can be changed based on your feedback.
>
> Please also check what kind of features are being developed to MeeGo QA
> tools [11].
>
> MeeGo Quality wiki [12] is still bit of a mess but you can find
> information there. Dublin workshop referred below is arranged during
> unconference days.
>
> Regards,
>
> -Ville
>    Meego QA Tools Project
>
>
> [1] MeeGo Core Test Suite - MCTS - http://gitorious.org/qa-tools/mcts
> [2] MeeGo Netbook UX Test Suite - MNTS - http://gitorious.org/qa-tools/mnts
> [3] MeeGo Handset UX tests - http://gitorious.org/qa-tools/handset-ux-tests
> [4] Test Plan definition - http://gitorious.org/qa-tools/test-definition
> [5] Test Plan documentation examples -
> http://wiki.meego.com/Quality/QA-tools/Test_plan
> [6] Testrunner UI v0.2.5 demo -
> http://www.youtube.com/user/meegoqatools#p/u/8/nileVzw52xM
> [7] Testplanner demos -
> http://www.youtube.com/user/meegoqatools#p/u/6/IQMPp5Nk4f4 (this is bit
> old, Kyosti has developed quite much further)
> [8] Test tools repository @ autotest guide -
> http://wiki.meego.com/Quality/QA-tools/Autotest-guide#Setting_up_test_automation_environment
> [9] MeeGo QA reporting - http://qa-reports.meego.com/
> [10] MeeGo QA mailing list - November archive -
> http://lists.meego.com/pipermail/meego-qa/2010-November/thread.html
> <http://meegoqa.leonidasoy.fi/>
> [11] List of open features to MeeGo QA tools - http://bit.ly/9G4EOl
> [12] MeeGo QA Wiki - http://wiki.meego.com/Quality
>
>
>
> On 11/03/2010 09:56 AM, ville.p.ilvonen at nokia.com wrote:
>    
>> Sure, I'll share my proposal for comments at meego-qa at meego.com
>> <mailto:meego-qa at meego.com>
>>
>> I'm traveling currently so Friday latest.
>>
>> Thanks,
>> -Ville
>> -- 
>> Sent from my Nokia N900
>>
>> ----- Alkuperäinen viesti -----
>>      
>>> Hi Ville,
>>>
>>>
>>>
>>> I just discussed with Fan related new test management tool proposal and
>>> workshop related to that in Dublin.
>>>
>>>
>>>
>>> Ville: How you feel could you share any info/material related to this
>>> tool for us before having workshop in Dublin. We would like to get
>>> familiar with tool before actual workshop to ensure that we are on top
>>> of it and can really be active in the actual workshop.
>>>
>>>
>>>
>>> Br, Petri
>>>
>>>
>>>        
>>      
> _______________________________________________
> MeeGo-qa mailing list
> MeeGo-qa at lists.meego.com
> http://lists.meego.com/listinfo/meego-qa
>    