The Test Program release model includes periods of active development followed by evaluation and finally a release "lock down" step to prevent changes. Each object of the test program (Test Plan, Fixture Def, DIB Def, Test Exec, etc.) can be independently locked by changing its status to "released". Once an object is released, saving changes requires the Title to also change. The status of an object changes from "alpha" to "beta" and finally "released" or "ready" to identify to the developer what step that piece exists of the release process. Use the version field to quickly identify revisions for locking or for recovery. Update major version number for each major "feature" added to the testplan and minor revision numbers to identify small changes. (Major.Minor or 00.01, 00.02, 01.00, 01.01, etc). If any revisions or changes are needed after the item has been released or identified during the evaluation phase, the version number scheme allows the developer to quickly restore a previous revision.
The first step uses various Guru apps, including the main Cassini application, Device Configuration Editor, Device Control Editor, Cassini Test Plan Editor, and other Guru apps to save all the test program objects with "alpha" status. The next step includes preparing the testplan(s) for production using Cassini app's Test Exec Editor and User App Manager and changing the status to "beta". The final product is a "released" User App that combines the Test Exec (with locked Test Plan) and Short Cut (locked patches) to prevent changes in production. Log in with "Operator" user name to only see Short Cuts, Execs and User Apps with status of "released".
These guidelines can be modified to fit a corporate policy for releasing and sign-off if needed by limiting visibility with permissions or user roles. Additionally, test floor automation platforms can trigger test execs or User Apps to activate via a network interface for bi-directional control and real-time system status. Since the versions are sorted alphabetically, at least one proceeding zero is recommended so the lists are sorted appropriately.
Step 1 - Development, Alpha Status
1. Use the latest "Dev" Short Cut (highest version number) to take advantage of the latest development features. Typically a "Production" short cut is released after careful validation process.
2. The Fixture and DIB definitions should all be generated first would display custom buttons to be used by the Test plan editor. Start with version "00.01" and status "alpha" for all objects. The Status should remain "alpha" for all definitions and components until verified by the next evaluation stage.
3. Include continuity tests that abort on failure to protect equipment from dangerous shorts.
4. Run from the Cassini Testplan editor to establish initial limits and data names. Use the Worksheet Details to prototype CSV and STDF output.
5. Every significant revision is always stored in the revision history, be sure to increment the minor version number at this time. (00.01, 00.02, 00.03... )
6. Add a one line description for the revision in the comment field. Use the Description or Notes in the Global for other changes. Be sure to annotate your decisions in specific test panels to allow for greater understanding when reusing panels and reviewing changes.
7. Once test specs are covered, continue onto the next Evaluation stage and prepare the operator user interface.
Step 2 - Evaluation, Beta Status
1. Create a new Test Exec to manage the testplan implementation with specified Handler/Prober and define operator workflow. Start with version "0.1" and status "alpha".
2. Leave the "Do not update testplan revision automatically" option disabled to get the latest revision of the testplan.
3. Use Handler "re-plunge" BIN for continuity failures by mapping soft bin to hard bin used by the Handler.
4. Evaluate Limits and data generation (STDF) processing via Guru Agents. (note: Guru Agents are universal for all program and are not version controlled)
5. Evaluate "Production" fast loading Short Cut to see if any new features implemented with the "Dev" Short Cut are required. Contact [email protected] to update the fast loading Short Cut.
6. Evaluate each component and change it's status to "beta" once it is validated. The version number should not change.
7. Create a new User App based on a previous one using a Title that the operator uses to launch the correct program. User Apps should all point to a single "Production" fast loading Short Cut application.
8. If any problems are encountered, change the status back to "alpha" and return to the Development stage. If no problems are found, continue on to the next stage to prevent revisions.
Step 3 - Prevent Revisions, Released Status
1. Starting with the Testplan, increment the major version (01.00) and change the status to "released". This forces a Title or Version change on any future changes.
2. Optionally, change the version to "01.00" and status to "released" for all definitions including: Test Exec, Fixture, DIB, Device Control, etc.
3. From the Test Exec, enable the "Do not update testplan revision automatically" option. This prevents any revision from being loaded automatically.
4. User App can initially be created to use the latest Test Exec, and once the Test Exec status changes to released, the User App can be locked to that version. Using Test Exec status of "released" or locking to a specific version of Test Exec is functionally equivalent. Increment the Major version number of the Exec, for example "01.00" and optionally align with the Testplan version.
5. Evaluate user App using "Engineer" logon. Once verified, change the status of the User App to "released" to expose it to the "Operator" logon. This must be the last step because version increment is required for any additional changes.
6. "Operator" users launch User App and/or Short Cuts > Production, then System > Test Execs. Other features are disabled to streamline the UI and prevent unauthorized changes.
7. Patches are locked within the Shortcut, Exec is locked with the "User App". If problems arise, continue to next stage for revision.
Step 4 - Modify, Re-Title and Obsolete
1. Depending on the type of change, the process to handle post release production situations or address science projects or diagnose production issues starts by modifying either the Testplan, Test Exec, or User App. In all cases, the Version should increment the minor number, i.e. 01.01 and Status to changed to "alpha".
2. To modify Tests or Limits, open the current release Testplan and save with a new Title, increment the minor version number and status to "alpha" and then repeat from Step 1 - Development.
3. To update the testplan or modify bins or select a different limit table, edit the TestExec, increment the major version number and status to "alpha" and then repeat from Step 2 - Evaluation. After Step 3 is completed, a new User App can also be generated.
4. Old User Apps and Test Execs can be obsoleted with Guru Explorer to be removed completely from view. (note: Obsoleted objects can be recovered if needed)