many thanks mith for having opened our forum.

hum, let's validate now collectively process.

Quote:


Proposal (taken from here https://xoops.org/modules/newbb/viewtopic.php?topic_id=40697&viewmode=flat&order=ASC&start=30#forumpost178558)

TIMEFRAME/PROCESS

1/ Core Team must perform internal "test run" before releasing an RC or beta
In case of large modifications (like 2.2 has), core team gives to Testing Team a RC per development phase.
Core devs document/explain changes and awaited impacts/side effects (what we should test first because part was heavy changed),please, so we can test with more efficiency.
Core Team has is own testing environment, and transmits to beta testing team XOOPS beta file in a whole package (no separate files,if yes, only in rare occasion-->should not be an habit)

2/ Core Team transmit to Testing Team Beta or RC in a whole .zip or .tar file
Test are made per developments phases in case of large modifications
No file has to be corrected manually by the beta testing team: beta testing team should act as if they where end users, installing for the first time xoops.
I think --> Beta testing team should not be on an "On the fly debugging team" (see point 1), to garantee quality of our work and efficiency.
Testing Team transmits bugs through a bug tracker tool (to be defined, see below)
We should set a minimum time duration between last RC and final, and between RC (time frame for RC release to stable)--> to be defined
Core Devs give which area has been strongly modified.
I will (marco) make such a draft list, dedicated to core, inspired from Checklist4ModDev (QA tool for module developers), that list points to be tested and in which environment.
As said before, we should improve these points :
- Test on several platform (IIS/Win XP, Apache/WinXP, Unix/Linux, etc.)
- test on several browsers


3/ A few exchange between Core and Testers are made, improvments are provided (using same process like 1/ and 2/)
After corrections were made, a new RC is released if everyone is happy with this beta/RC

4/ This RC is posted officialy at xoops.org.
Devs and testing team ask together community to get involved in release protocol "Are you ok with this version to be a final ? Be responsible for ! please make final tests and notice bugs we, devs and testing team, have not detected"
No public release until team was totally happy with it

5/ Final is announced
Message: in each news announcement, we should remember to users
- install/upgrade process : this one should be standardized (<<"enter the administration interface and update the system module". Perhaps it would be easier to have a standard script to run each time -for every even the most simple of upgrades- that will do that for the user.>> --> I think mith has made some improvments in 2.2.3 in this area, no ?
).
- Warning : "make personnal tests before use in production environment" (:" anything new is unstable"). "test it locally!"
- process for feedbacks
changelog and new features
A list of changes and new features, and a link to new classes/functions incoporated into core
Some lines for Modules Devs if necessary

Time
- Devs should be there the day after announcments to help people if new bugs are detected and make patch.

6/ Upgrade Xoops.org and Subsidiaries
Xoops.org and subsidiaries should be updated with last XOOPS version very early after a version is announced as stable (no more than 2 days after):
- to make our CMS reliable
- to promote it with efficiency (Promotion & Marketing team should be agree with that)

I post this one to discuss.
please feel free to add you comments and your feedback.
please comment that one !
marco

nb we definitively need the whole core team feedback to start our works, i think. When are they all back ?

Quote:
Definitely agree, although my two other threads in this forum could say the opposite.
One thing, though, how do we do about errors that we cannot reproduce? When we are debugging and troubleshooting, but cannot test? Who do we talk to?

Quote:
I generally agree, although the upgrade to 2.2.x will probably be a part of a bigger reorganisation and can take some time.

Quote:

i don't think there is much you can do about that, other than if possible to contact the person who reported it and get as much info from them as possible. phpinfo() etc, what they did to get the error.

in the end it could just be a corrupted file during transfer that maybe the problem. but i would concentrate more on the errors that can be reproduced as they are more general to affect everyone.

Quote:

Hence the reason we have setup a couple of testing sites for the team. A bug, unless very specifically caused by server setup, should be reproduced on the test servers which run common webserver specs.

About timetable, do we organize things on weekly basis? so that all releases are done the same day of the week (Tuesday or Wednesday for example). This way the team knows that all testing needs to be done by the Sunday for bug reports to be, hopefully, applied to the next RC release of that week.

Do the Core devs wish to do the upgrade/install on the beta server? For each release I will probably have one test site for an upgrade and another for clean install, just to check both aspects.

Cheers.

I think what we would like the best (but will have to get this cleared with the rest) is a fixed weekday, where betas are available (we can perhaps use a cron job like with the CVS Nightly releases) and then the beta testers will have x days to test it and report problems.

I think that a release on Wednesdays and reports due by Sunday is a good scheme.
With automated releases, we can also focus on writing some information for you instead of spending the time exporting and packaging stuff from XOOPS CVS.
For example, every Wednesday at 10.00 AM GMT, I will have written a report on what has changed in the core files since last week, so beta testers know what to test.

I could probably also let bugs "carry over" from one week to another - like "We know this one exists, but haven't fixed it/cannot reproduce it yet"

Since available time is just as unstable for us as it is for most people here, it is important to have some room for slow pace from time to time.

Forgot to answer this:
Quote:
No, I don't think that is a good idea.

When I update or install, I rarely make mistakes or put in invalid data... when a beta tester updates or installs, he/she SHOULD make mistakes and put in invalid data on purpose.

So it would be best if the beta testing team had a couple of people with credentials to the beta sites so they can update them.

I got ahead of myself on this one.

I am thankful for the work done so far, with the number of people willing to do organised testing and I'm sure that we will benefit greatly from it.

but just now we'll slow things down just a notch. Internally in the core developer team, we need to work out some rules and procedures to go by, before we can load things on your shoulders.

So disregard what I said about the weekly betas etc. but stay ready for beta testing, when we get there.