10
Looking better
What will be next step, once this specification is finished? Will you (with the Docs team, perhaps?) make a "Guide to Approval", explaining how to accomplish each task? I think that would be a tremendous guide. You say yourselves in "S4 - Textsanitizer usage" that instructions on correct usage should be available and I think other tests could use the same (e.g. what constitues "Smarty compliant"?), so might as well make a section for each test, explaining
a) its purpose
b) how to make a module pass this test and
c) how the test is performed
I could also suggest a new category - C (for Code Standards) - where you take the P2 test about object orientation as well as general coding standards and core usage. I suggest that this new category has the following tests:
P2, A5 (Note that there are TWO A5's, I mean the one about permissions), A6, A7 and A11. An added use of core Notification (where applicable) could perhaps also be a new test?
Only one piece of feedback left now: B1 - it doesn't really matter to me, when I install a module, whether it has been in Beta and/or RC before being released. It is a nice reassurance for me to know that it will probably have fewer bugs than a module that was released without it, but since you ARE testing the module for bugs etc. I think it will not be fair for a module that passes ALL tests apart from this one to be noted "Did not pass this test"... when everything is working, the need for a Beta/RC is limited, I think. It may serve to be an
explanation for why the module fails other tests, and you should be able to conclude after performing the tests that it should have been released in beta/RC versions prior to final launch, but it should not be a test in itself.
Use what you can of this and I am of course available for working with you on the coding standard and core usage tests.