Here is a small summary (not finished yet)
I've tried to collapse all our ideas.Thanks for your brainstorming.
Comments awaited (it's still a quick summary draft, I have to change some wording and make it clearer.
marco
1/ Developer must perform internal "test run" before announcing anything
2/ Testing Team
- a few people like local support team members (1)
- choosen well known debug experts (2)
(1) To ensure language issues also deal with too - sometimes easy to overlook things that may be "rendered" differently in other languages
(2) include users of a capable ability... but not newcomers who have not had chance to "get their hands dirty" with Xoops...
Could be the same team as module testing...
They received betas
After corrections were made, an RC is released.
3/ Test are made per developments phases
This at each step of development process if it is possible
Core devs document/explain changes and awaited impacts/side effects (what we should test first because part was heavy changed",please, so we can test with more efficiency.
4/ This RC is posted officialy at xoops.org. If needed RC2/RC3/ect. are released.
Devs ask community to get involved in release protocol "Are you ok with this version to be a final. Be responsible for. please make final tests and notice bugs devs and testing team have not detected"
5/ have some time to test.
We should set a minimum time duration between last RC and final, and between RC (time frame for RC release to stable)
6/ No public release until team was totally happy with it
A certain number of positive test reports should be required prior to deciding that the release is ok for everyone to use, regardless of how long that takes.
+
no public release until an RC (minimum) was published @xoops.org so that end users (community) make final tests
"unstable" releases be clearly labeled as such, so people are aware that they shouldn't blindly upgrade their production sites.
A dedicated test team who know what they are doing would enable the core team to release an RC in the confidence that it *does* install properly and function properly - provided the system requirements are met and the HTTP/PHP server and database server are correctly configured.
Users can then debug the RC, but at least the core team will know that an 'I get a blank screen' post is a system config problem, not a XOOPS problem.
7/ Message to end users
In each news announcement, we should remember to users
- install/upgrade process : this one should be standardized (i think it already is).
- make personnal tests before use in production environment :" anything new is unstable". test it locally
- process for feedbacks
8/ Time for announcment
Devs should be there the day after announcments to help people if new bugs are detected and make patch.
7/ Test on several platform
IIS/Win XP
Apache/WinXP
Unix/Linux
Most of the error reporting and problems seem to comme from upgrading. Since the core devs probably don't test it with the many different setups we all have, it is near impossible for them to find all problems.
It would be important that each can list up the particulars of their installs, server platforms of course, but modules also and any other special hacks they use
Tools
1/ discussion should be private until a mature status is agreed on - as public discussion too easily disintegrates into off-topic chat.
2/ Release could be posted on a separate page, with a big red notice at the top stating that they haven't been fully tested yet.
The announcements could still be posted here, but the extra step of having to go somewhere else to download might be adequate.
4/ Rough aims and methods for such a team... (they can be finalised at a later stage)
5/ tool for tester :I will (marco) make such a list, dedicated to core, inspired from
Checklist4ModDev (QA tool for module developers)
6/ Consice definition of what would be considered a good bug report
example --> "cookie cutter" template is targeted to the inexperienced XOOPS user
Quote:
Polite Opening,
I'm experiencing a bug after [installing | upgrading to] <xoops version> and would like to report a bug.
Bug Description:
----------------
Provide as much information regarding the bug as possible. Include any appropriate information such as, what you did prior to noticing the bug, what are the affects of the bug, does the bug affect all users, is the bug viewable in all browsers, etc.
Steps for reproduce the bug:
----------------------------
In clear, concise, numbered steps, walk the Devs through the proces of reproducting the error, if possible.
Example:
1. Add "Site Info" to the list of used blocks
2. Edit block and enable to show admins
3. Configure other settings as you want
4. Check block on frontend page and detect missing part of information
Debug Output:
-------------
Put your PHP, MySQL, and Smarty Debug output here. Seperate with headings. If you are getting a White Screen of Death (WSOD), paste whatever php output is on the screen.
Example:
PHP Debug Output:
<output goes here>
MySQL Debug Output:
<output goes here>
Additional information:
-----------------------
The bug in question can be viewed at <link> (provide indicators of where to look on the page). Admin rights granted to responsible person on request (optional, but may be helpful).
Posible solution:
-----------------
If possible, provide a detailed description of how to fix the problem. As above, write in clear, conscise, numbered steps that the Devs can easily follow.
Server Information:
------------------
<Operating System>
<Web Server/version>
<PHP Version>
<MySQL Version>
Polite Closing,
Your Name
+theme and modules.
Volonteers (to be checked)
jdseymour
kaotik
Lance_
SoccerDad
rowdie
Chappy
Teibaz
JMorris
dap997
We need testers on each area : core and core themes
