Policy and procedure for updating DAQ Linux systems: ==================================================== Author : Ron Fox Last Modified : February 10, 2004 Modification History: Date Modification February 13, 2004 Original version ------------------------------------------------------- This document covers all systems named spdaq*, Linux systems named u[2-6]pc{2,3}, Linux systems named: u1pc3, u1pc4, u1pc5, useepc2 ------------------------------------------------------- Guiding Principles ------------------------------------------------------- 1 Data Acquisition system software should be stable so that experiments are repeatable and the software configuration is understandable to the users. 2 Experiment special requirements must be met that create a design tension with 1. 3 Quality management will require that pre-release versions of software be available on production systems so that new software can be tried under realistic conditions prior to general release. --------------------------------------------------------- What software are we talking about? --------------------------------------------------------- Three specific packages are discussed in this document. - SpecTcl - the data analysis software that lives in (from the user's point of view) /usr/opt/spectcl - spectrodaq - The data distribution software. This is a persistent server that runs in all spdaq and data u systems. - nscldaq - Spectrodaq client software. From the users point of view this lives in /usr/opt/daq --------------------------------------------------------- Directory organization --------------------------------------------------------- NOTE: This is currently in flux for the daq software. This section describes the end goals. Version naming: Spectrodaq has versioning for both the server and client libraries. Spectrodaq versions are of the form: servermajor.serverminor.serveredit/clientmajor.clientminor.clientedit We are currently running spectrodaq-0.8.1/0.8.1 major version 0, minor version 8 and edit level 1 for both client and server. SpecTcl and nscldaq have a version name of the form: major.minor-editlevel (edit level is a three digit number). Spectrodaq: This is a persistent server. There cannot be more than one of these running at a time. Therefore it makes no sense to talk about multiple versions of spectrodaq installed on a system. Client software must be compiled and linked against the appropriate client libraries. Unfortunately this means that we can only do internal (alpha) testing before unleashing spectrodaq on the public. spectrodaq lives in /usr/opt/spectrodaq The same version must be installed on all systems. SpecTcl: This has a stable directory scheme that supports multiple versions of the software. All versions are installed in directories that live below /usr/opt/spectcl. The directories are named after the major and minor version of the software. For example, at this time, the directories 1.0 1.1 2.0 and 2.1 exist in the SpecTcl installation. A symbolic link named current points to the 'current stable version' of the software. At this time current -> 2.1 nscldaq: This directory scheme is in flux from a single version install in /usr/opt/daq to a multiversion install below /usr/opt/daq. At this time, /usr/opt/daq contains the current distribution. (7.3), Underneath it, you wil find, on many systems, a directory named 7.4 which is the pre-release of version 7.4 with features available for beta test. The phasing of /usr/opt/daq's organization to match SpecTcl's is: - When 7.4 becomes the stable release, it will be placed in /usr/opt/daq and a current link will be created to point to it. Existing makefiles will work without modification but will link to 7.3 from then on. By editing one line in existing makefiles, it will be possible to link to the 7.4 distribution. - When 7.5, and 7.6 become stable, current will just be pointed to them. Users will have to choose to upgrade by modifying a line in their makefile. - When 8.0 become stable, the 7.3 installation will be removed, completing the transition to a SpecTcl like directory structure. ----------------------------------------------------------------- Versioning and the release/update policy ----------------------------------------------------------------- The guiding principles dfeine our release policy. Goals of the release policy: Updates come in three flavors: From least to most intrusive: edit-levels - usually these are maintenance releases that fix problems in existing release or back port features from later versions. Edit levels should as much as possible require no action on the part of the user to adopt (new shared libraries that's it). minor - A release with new functionality and bugfixes that are purely evolutionary with no structural implications. minor releases within the same major version should only require recompilation to use, with at most a oneline change to the makefile to indicate which version is used. major - Substantial new functionality that has structural implications Major releases may require user source code and makefile modifications to work properly. In all cases we define a stable release of the software. The stable release is the software that users will use if they don't do anything special. We also define 'old' releases of the software. Users may have to make environment and path variable settings to use old software releases. (e.g. set LD_LIBRARY_PATH). They will under no circumstances need to make source or Makefile changes. Old releases only get minimal bugfix maintenance. Finally we define 'pre' releases of the software. Pre-release software are minor and major releases of software that have a higher version number (taken as a decimal number of the form major.minor) than the current release. This section will describe the update policies for all three of these release types. Updating Stable Releases: ------------------------- - The only updates allowed to stable releases are edit-levels that fix software defects. These edit-levels are installed in three phases where possible: 1. The problem is reproduced, fixed and tested on the 'reference test system'. At present the reference test system is spdaq14. The reference test system is a computer group 'owned' system that runs the current release of software on current operating system software. 2. A source distribution of the edit level is produced and used to do an installation on systems on which the problem was initially reported. Users of that system test the fix. 3. When the fix is tested, a tarball is made of /usr/opt and placed in /soft/conf/cfengine2/share/debian/daq. As needed or at a short break in the experimental schedule, software updates are triggered via planting the file UPDATE_ME in the /usr/opt on the target systems. NOTES: - In the case of SpecTcl and Spice systems, where possible, user testing should be done on a spice system. - In the case of SpecTcl and Spice systems, full deployment need not wait for a break in the experimental schedule but can occur anytime during off hours. - All updates should be announced to daqsystemupgrades@nscl.msu.edu. - All updates should be logged at http://internal.nscl.msu.edu/hardware in system description entries. Updating old releases: ---------------------- Only edit level updates can be applied to old releases. The procedure for applying these updates is identical to updating stable releases. Not all defects fixed in current and later releases need to be fixed in old releases. Where possible, users are urged to migrate towards the current release. fixes will only be backported to old releases if there is a compelling reason the user cannot upgrade (e.g. time crunch or in the middle of a running experiment or series of experiments that is using the old release for stability reasons). Creating and Updating Pre releases: ------------------------------------ Major and minor pre-releases are created and installed for only the following reasons: - To deploy functionality required by an experiment that is not supported by the stable release. - To fix defects in the current release that cannot be fixed without requiring recompilation of user code. - To provide features or fix defects that require user code or makefile changes other than version selection. The procedure for udpating is identical to the procedure for updating old and current releases, however it is allowed to deploy pre-releases more widely without as much test/verification as for edit-level releases for those releases. --------------------------------------------------------------------- CVS usage --------------------------------------------------------------------- This section describes the CVS discipline required to maintain all these release levels. We basically maintain branches for each of the releases. Towards the end of each release cycle, a major/minor release is merged with its prior revision just to ensure that no fixes have fallen through the cracks. CvS repositories are stored on sourceforge for SpecTcl and nscldaq. spectrodaq is in: /user/proj2/projects/spectrodaq/cvs/repository/DAQ/spectrodaq SpecTcl: :ext:ron-fox@cvs.nsclspectcl.sourceforge.net:/cvsroot/nsclspectcl nscldaq: :ext:ron-fox@cvs.nscldaq.sourceforge.net:/cvsroot/nscldaq Sourceforge CVS access requires: - ssh (e.g. export CVS_RSH=ssh) transport. - A sourceforge account for checkout and export. - To do commits and other repository modifications, you will need to be added to the project by me. ---------------------------------------------------------------------- Sourceforge releases ---------------------------------------------------------------------- As software becomes moderately stable (defect report rate is better than 1/month), a source distribution is created and uploaded to sourcerforge. To upload this you will need to be a project member.