[Meego-qa] Questions about Meego general test report template

[Qingqing.Tu at elektrobit.com]

Hi Tu,

I have not seen any responses to this thread. Here are my terminology
opinions.

Definition of "Not testable", "N/A" and "Blocked" - I put the following
definitions on http://wiki.meego.com/Quality/Glossary#Test_case_verdict
- please contribute there by correcting my wrong sayings ;-)
- Blocked: This is special value required for the cases where verdict
cannot be given due to failure in preconditions. 
- N/A - Not Applicable: can be seen as a initial value which is left as
a verdict for a test case when no other verdict cannot be given due to
some reason (e.g. test case could not be executed due to failure in test
infra, etc.). 
For all Blocked and N/A verdicts it would be preferable that reason why
pass/fail verdict could not be given is documented either by comment
"Test case X verdict not given due to reason Y" or bug ID.

I suppose we need the ones for Pass and Fail too ... But I suppose those
are more straightforward.
- Fail: A test is deemed to fail if its actual result does not match its
expected result.
- Pass: A test is deemed to pass if its actual result matches its
expected result.

For tables having different things and detailed results no added value
on this.

Jari, thanks for the response. 
For the reason why pass verdict is given, I suppose in certain case like
there is improvement etc,it is necessary but I don't see the need for
all the pass verdict. For others I agree with you. :)

Br,
-Tu

-----Original Message-----
From: meego-qa-bounces at lists.meego.com
[mailto:meego-qa-bounces at lists.meego.com] On Behalf Of ext
Qingqing.Tu at elektrobit.com
Sent: 22 November 2010 10:25
To: meego-qa at lists.meego.com
Subject: [Meego-qa] Questions about Meego general test report template

Hi,

I have been looking at the general test report template. The template
structure is in good shape and clear for me, however, while looking into
detail of the template, I have some questions, these questions may be
met by someone else who will check the template, so I raise one topic
here to discuss to make things 100% clear and get common understanding.
http://wiki.meego.com/TestReportTemplate

1. In section "Testing Summary - definition": 
1.1 Definition of "Not testable", "N/A", "Blocked" is missing from this
section but they are in use in the two tables under same section.
What does "Not testable", "N/A", "Blocked" stand for? 

My understanding: 
- "Blocked" means test cases cannot be tested since one gating/entry
test case Fails and blocks related test cases; like there is two test
cases, first test case is targeted to test "add new contact to
phonebook", second test case is targeted to test "make phone call to new
added contact from phonebook", if the first test case fails e.g current
release doesn't support add new contact, thus the second test case can
not be tested as be blocked by the same error.
- "N/A" means test cases which are not testable due to feature is not
ready for test, e.g not integrated yet. I suppose "Not testable" covers
"N/A",  or is "Not testable" = "N/A" + "Blocked"? 

1.2 Different terminology and test result are using in the two tables in
section "Testing Summary - definition":

IMO, the 1st table is test result metrics of one test cycle, 2nd table
is breakdown of component test result metrics, both data of the two
tables are coming from same test cycle, the two tables have common test
result definition like passed and failed, but also have some difference,
like 1st table has "Not testable" and "Measured" but no "N/A",
"Blocked", 2nd table is reverse. I am wondering why. 
Besides, I suppose "Run rate" and "Executable rate" is same thing, I
prefer to use same term to avoid confusion. Pls comments or correct me
if I am wrong.

2. In section "Detailed test result": 
Is each test case result required to add? E.g in one existing weekly
report
http://qa-reports.meego.com/1.2/Core/Weekly/N900/25#test_objective,
detailed test cases result were attached. But this is not mentioned in
the test report template.

Br,
-TuQQ





----------------------------------------------------------------
Please note: This e-mail may contain confidential information intended
solely for the addressee. If you have received this e-mail in error,
please do not disclose it to anyone, notify the sender promptly, and
delete the message from your system.
Thank you.

_______________________________________________
MeeGo-qa mailing list
MeeGo-qa at lists.meego.com
http://lists.meego.com/listinfo/meego-qa


----------------------------------------------------------------
Please note: This e-mail may contain confidential information
intended solely for the addressee. If you have received this
e-mail in error, please do not disclose it to anyone, notify
the sender promptly, and delete the message from your system.
Thank you.