This talk was first given by Curt
Sampson at the November 16th, 1998 meeting of the
Vancouver Linux Users' Group.
CVS An Aid to
Software Configuration Management
Under Unix
- SCM Software Configuration Management is a system or framework for reliably building programs.
- CVS Concurrent Versions System is a program to do revision control on a large base of code, and permit multiple developers to work on multiple versions at once.
- During this talk we will discuss many of the functions needed in an SCM system, with particular attention to revision control with CVS.
I. Revision Control with RCS
RCS Revision Control System
- Developed by Walter Tichy at Purdue around 1982.
- Similar in concept to SCCS, from Bell Labs, circa 1975
- RCS is faster than SCCS and also has slightly better functionality.
- Now available from the FSF under the GPL.
What does RCS do?
- Stores many different revisions of a file.
- Enables you to retrieve arbitrary revisions.
- Enables you to see what has changed between revisions.
- Marks files with their revision code.
RCS Check-in
- Taking a file and doing a `check-in' of it creates a new revision group. This is a file, ending in
,v, which contains all of the revisions of that file that have ever existed.
$ ls
file1
$ ci file1
file1,v <-- file1
enter description, terminated with single '.' or end of file:
NOTE: This is NOT the log message!
>> The first file.
>> .
initial revision: 1.1
done
$ ls
file1,v
RCS Check-out
- Once we have a revision group set up for a file, we can use the
co command to check out a copy in order to examine it.
$ ls
file1,v
$ co file1
file1,v --> file1
revision 1.1
done
$ ls l
total 2
-r--r--r-- 1 cjs staff 85 Nov 14 19:14 file1
-r--r--r-- 1 cjs staff 275 Nov 14 19:14 file1,v
- Note, however, that this file is read-only. If we overrode that read-only designation, changed it, and tried to check it in, we can't.
$ chmod +w file1; vi file1
...
$ ci file1
file1,v <-- file1
ci: file1,v: no lock set by cjs
- We need to check out a locked copy with
co l. Since we didn't do that, we can lock our current unlocked check-out with rcs l.
$ rcs l file1
RCS file: file1,v
1.1 locked
done
- Now we can check in our change. Last time we did a check-in, the checked-out copy was deleted. As a shortcut, we can ask
ci to check out a locked or unlocked copy of the file after the check-in by using the l or u options, respectively.
$ ci l file1
file1,v <-- file1
new revision: 1.2; previous revision: 1.1
enter log message, terminated with single '.' or end of file:
>> Added another line.
>> .
done
$ ls l
total 2
-rw-r--r-- 1 cjs staff 127 Nov 14 19:15 file1
-r--r--r-- 1 cjs staff 435 Nov 14 19:29 file1,v
- This is called strict locking. RCS allows you to turn off strict locking, so that you can check in an update to a revision you do not have a lock on. However, this can lead to problems if multiple users have the same revision checked out.
RCS Logs
- The
rlog command shows information on the revision set.
$ rlog file1
RCS file: file1,v
Working file: file1
head: 1.2
branch:
locks: strict
cjs: 1.2
access list:
symbolic names:
keyword substitution: kv
total revisions: 2; selected revisions: 2
description:
The first file.
----------------------------
revision 1.2 locked by: cjs;
date: 1998/11/15 03:29:14; author: cjs; state: Exp; lines: +1 0
Added another line.
----------------------------
revision 1.1
date: 1998/11/15 03:14:43; author: cjs; state: Exp;
Initial revision
===================================================================
RCS Check-out of Other Revisions
- You can check out a particular revision, rather than the most recent, by using the
r option with co.
$ co r1.1 file1
file1,v --> file1
revision 1.1
done
- Revisions can be selected based on any one of several criteria
- Revision Number
- Date
- State
- Login of person who did the check-in
- Symbolic name assigned to a revision
- Symbolic names are assigned with
rcs n.
$ rcs -nfoo:1.2 file1
RCS file: file1,v
done
$ co -l -rfoo file1
file1,v --> file1
revision 1.2
done
$ rlog file1
...
symbolic names:
foo: 1.2
...
- Note that symbolic names can refer to different revisions in different files.
$ rcs -nfoo:1.3 file1
RCS file: file1,v
done
$ rcs -nfoo:1.1 file2
RCS file: file2,v
done
$ co -rfoo file1 file2
file1,v --> file1
revision 1.3
done
file2,v --> file2
revision 1.1
done
RCS Comparing Revisions
- You can see the differences between two revisions using the
rcsdiff command. This is very similar to the standard Unix diff command, and takes many of the same arguments.
$ rcsdiff -r1.1 -r1.2 file1
===================================================================
RCS file: file1,v
retrieving revision 1.1
retrieving revision 1.2
diff -r1.1 -r1.2
2a3
> This is the third line of the first file.
cynic $ rcsdiff -u -r1.1 -r1.2 file1
===================================================================
RCS file: file1,v
retrieving revision 1.1
retrieving revision 1.2
diff -u -r1.1 -r1.2
--- file1 1998/11/15 03:14:43 1.1
+++ file1 1998/11/15 03:29:14 1.2
@@ -1,2 +1,3 @@
This is the first line of the first file.
This is the second line of the first file.
+This is the third line of the first file.
RCS Identification
- Identification information can be added to the file with
$Keyword$ keywords. This information can include
- File name and path
- Revision number
- Symbolic name used on checkout
- Date
- Author
- Who has the lock on this revision
- State
$ cat file1
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
This is the first line of the first file.
This is the second line of the first file.
This is the third line of the first file.
$ ci -u file1
file1,v <-- file1
new revision: 1.3; previous revision: 1.2
enter log message, terminated with single '.' or end of file:
>> Added ID string.
>> .
done
$ cat file1
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
This is the first line of the first file.
This is the second line of the first file.
This is the third line of the first file.
- This change of the
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $ string is called keyword substitution. It can be turned off for binary files.
- Typically you uses language constructs to embed these strings in the binary files that are the result of compilation. In C, you would put
char *version = "$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $";
in the source files used to build a program. You can later use the ident command examine the binary files to see which versions of which source files are in it.
$ ident /usr/bin/dig
/usr/bin/dig:
$NetBSD: crt0.c,v 1.28 1997/10/20 13:49:45 mycroft Exp $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
[etc.]
- This information is useful for determining if a bug or security hole is present in a program; you can see if it has the buggy version or the updated, fixed version of a file compiled in.
RCS Branches
- To this point we have had revisions incrementing from 1.1 in a linear fashion. Each revision has one predecessor, and one successor.
- It is possible for a revision to have several successors; this is called a branch.
- We have an existing 1.3 revision derived from 1.2 Let's edit the 1.2 revision and check in a different set of changes to it.
$ co -l -r1.2 file1
file1,v --> file1
revision 1.2 (locked)
done
$ vi file1
...
$ ci -u file1
file1,v <-- file1
new revision: 1.2.1.1; previous revision: 1.2
enter log message, terminated with single '.' or end of file:
>> Add ID string at end.
>> .
done
- Since a 1.3 revision already exists, RCS automatically creates a branch at this point. The number 1.2.1.1 indicates that this is revision 1.1 on a branch that starts at 1.2. The revision tree now looks like this:
1.3 1.2.1.1
| |
1.2 ------+
|
1.1
- You can have branches of branches to any degree, if necessary. After more branches and check-ins, our tree might look like this:
1.6 1.2.1.4 1.2.1.3.1.1
| | |
1.5 1.2.1.3 --------+ 1.2.1.1.1.2
| | |
1.3.1.1 1.4 1.2.1.2 1.2.1.1.1.1
| | | |
+------1.3 1.2.1.1 ---------------------+
| |
1.2 ------+
|
1.1
- A branch name has an odd number of dot-separated digits in it, and the most recent revision on each branch is called the head. A check-out of a branch brings back the head revision
$ co r1.2.1 file1
file1,v --> file1
revision 1.2.1.4
done
- When branch and revision names become complex, symbolic names become quite useful.
$ rcs ncjstest:1.2.1
$ rcs nrcctest:1.2.1.1.1
RCS Merging Revisions
- Sometimes, having done some work on a branch, we want to put those changes on the trunk. However, changes may also have been made to the trunk. You cannot just check in your changed version, since that would wip out changes made since you branched.
- RCS allows us to merge the set of changes made since the initial revision on the branch, and the set of changes made since that branch on the trunk, to come up with a new copy of the file.
- Say we want to merge the following sets of changes:
- 1.2 to 1.3: adding the ID string at the beginning of the file
- 1.2 to 1.2.1.1: adding the ID string at the end of the file
$ co -l -r1.3 file1
file1,v --> file1
revision 1.3 (locked)
done
$ cat file1
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
This is the first line of the first file.
This is the second line of the first file.
This is the third line of the first file.
$ rcsdiff -u -r1.2 -r1.2.1.1 file1
===================================================================
RCS file: file1,v
retrieving revision 1.2
retrieving revision 1.2.1.1
diff -u -r1.2 -r1.2.1.1
--- file1 1998/11/15 03:29:14 1.2
+++ file1 1998/11/16 08:23:53 1.2.1.1
@@ -1,3 +1,4 @@
This is the first line of the first file.
This is the second line of the first file.
This is the third line of the first file.
+$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
$ rcsmerge -r1.2 -r1.2.1.1 file1
RCS file: file1,v
retrieving revision 1.2
retrieving revision 1.2.1.1
Merging differences between 1.2 and 1.2.1.1 into file1
$ cat file1
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
This is the first line of the first file.
This is the second line of the first file.
This is the third line of the first file.
$Id: CVSpres.html 319 2001-11-07 06:23:15Z cjs $
- Sometimes there will be conflicts in a merge: a line was changed one way in one branch, and the same line was changed a different way in another branch. The computer cannot decide which is right, so it puts in conflict markers to show the conflicting changes.
- For example, if the main branch changed `Line two' to read `Line 2', and the 1.1.1 branch changed it to read `Line the second', we would see this as the conflict marker:
<<<<<<< file1
Line 2
=======
Line the second
>>>>>>> 1.1.1.1
RCS Multiple Users and Workspaces
- Files can be checked out from revision groups elsewhere by giving the full path.
$ cd ~rcc/work
$ co -u ~cjs/source/thisprog/file1,v ~cjs/source/thisprog/file2,v
../a/file1,v --> file1
revision 1.3 (unlocked)
done
../a/file2,v --> file2
revision 1.1 (unlocked)
done
- The full path has to be given for all operations, not just the checkout.
$ rcsdiff -r1.1 -r1.2 file1
rcsdiff: RCS/file1,v: No such file or directory
$ rcsdiff -r1.1 -r1.2 ~cjs/source/thisprog/file1,v
=========================================================
RCS file: /home/cjs/source/thisprog/file1,v
retrieving revision 1.1
retrieving revision 1.2
diff -r1.1 -r1.2
2a3
> This is the third line of the first file.
RCS More than Just Source Code
- Programs in C and other languages.
- Text files: documentation.
- Text files: HTML. I use it for my personal web pages.
- Configuration files: Internet Portal uses RCS for maintaining files in
/etc and elsewhere when multiple sysadmins are maintaining systems.
- It works on binary files too.
RCS Problems
- Awkward when multiple users are using it:
- Must specify full path to each RCS file for each operation when not in current directory
- Must lock a file before changing it; users cannot edit files locked by other users.
- Awkward with large numbers of files; each file has to be updated separately.
II. Revision Control with CVS
CVS Concurrent Versions System
- Originally written as shell scripts by Dick Grune at Vrije Universiteit in Amsterdam around 1986.
- Translated into C and significantly extended in 1990 by Brian Berliner at Prisma. (There it was used for the SunOS 4 kernel, which comprised over a thousand files at the time.)
- Originally used for maintaining local changes to vendor source code, while permitting (relatively) easy local updates as vendor code is updated.
CVS Based on RCS
- On an individual file basis, CVS is very similar to RCS. It provides the same operations.
- CVS differs from RCS in three major ways:
- It extends operation from files to directory trees.
- It removes the necessity of locking files before modifying them.
- It changes branch semantics somewhat.
CVS Initialising the Repository
- CVS stores the directory trees under revision control in a repository. This must be initialised before use.
$ export CVSROOT=/home/cjs/cvsroot
$ cvs init
- After the repository has been initialised, we can import the initial set of files we'd like to start with.
$ cvs import -m "Initial import." mytree vendor release_1
N mytree/file1
N mytree/file2
cvs import: Importing /home/cjs/work/cvs-pres/cvsroot/mytree/one
N mytree/one/file3
N mytree/one/file4
cvs import: Importing /home/cjs/work/cvs-pres/cvsroot/mytree/two
N mytree/two/file5
No conflicts created by this import
CVS Checking Out and Updating Modules
- The initial operation a developer will perform is a check-out. This can be from a local repository (such as
/cvsroot) or a remote one (cvs.netbsd.org:/cvsroot). If remote, it will use rsh or the program specified by CVS_RSH.
$ cvs co mytree
cvs checkout: Updating mytree
U mytree/file1
U mytree/file2
cvs checkout: Updating mytree/one
U mytree/one/file3
U mytree/one/file4
cvs checkout: Updating mytree/two
U mytree/two/file5
- Once checked out, you will regularly need to update your copy of the source to bring in changes from other developers.
$ cd mytree
$ ls CF
CVS/ file1 file2 one/ two/
$ cvs update
cvs update: Updating .
U file1
P file2
cvs update: Updating one
cvs update: Updating two
- With CVS, each user has his own `workspace' or `sandbox.' He can do whatever he likes in his copy of the sources with affecting other users.
- CVS does not have offer any facilities for `backing trees'; when you do a build, you check out the whole thing. You burn disk space in trade for simplicity.
CVS Making and Checking In Changes
- Files do not need to be locked to make changes. Just make your changes and then commit them.
$ cvs update
cvs update: Updating .
M file2
$ cvs commit -m "my changes"
cvs commit: Examining .
Checking in file2;
/home/cjs/work/cvs-pres/cvsroot/mytree/file2,v <-- file2
new revision: 1.2; previous revision: 1.1
done
CVS Commands
- Some commands are similar to what RCS does, extended to directory hierarchies:
cvs log: show the RCS logs of a file.cvs diff: show differences between revisions, or between current copy and repository copycvs tag: assign a symbolic name to a set of revisions.cvs co rrev or cvs update rrev: check out specific versions of filescvs commit: commit changes (check-in)cvs update jX.X jX.X: merge revisions.cvs admin: execute RCS commands.
- There are also some new commands:
cvs import: Import files to vendor branch.cvs update: bring checked-out files up to date.cvs add: Add a file to the repository.cvs remove: `remove' a file from the repositorycvs status: show current checked out revision and status of it.cvs annotate: show who wrote each line.cvs history: show repository access history
CVS Branches
- CVS actually has three branches it looks at when you check-out files:
- The current branch you're working with.
- The `mainline' branch.
- The vendor branch.
- CVS will look through these in order to check out the appropriate file.
- New branches can be created for experimental work; these branches would later be merged back into the mainline.
- New branches would also be created for releases; these would never be merged. The steps for a release are:
- Tag branch:
cvs rtag b netbsd-1-3 src
- Update or checkout branch:
cvs update rnetbsd-1-3
- Work on branch.
- Tag for releases:
cvs rtag netbsd-1-3-BETA src
cvs rtag netbsd-1-3-RELEASE src
cvs rtag netbsd-1-3-PATCH01 src
CVS Importing Vendor Sources
- CVS was designed for this, though it's not perfect for it.
- Extremely useful for any Open Source software where you use non-release versions or where you make modifications.
- Import steps:
- Delete files not in new import that you're not using.
- Do import.
- Watch merge.
- Resolve conflicts.
CVS Checks and Logging
- CVS can run a program to verify that changes are OK before commit.
- CVS can execute arbitrary programs after commit, for example, to send notice of commit to mailing list.
CVS Driving Open Source
- Networked: easy to have repository on one machine and checkout on another.
- Supports a large number of developers: hundreds of developers can be working on code without conflicts, assuming other communication mechanisms (such as mailing lists) are also in place.
- Anonymous check-out: lets anybody check out a copy of the source, see history of changes, and keep up to date.
CVS More Than Just Source Code
- Web sites with multiple developers.
- Documentation Repositories: used for this by sysadmins at Internet Portal.
- Easy updates for Open Source programs: supports `bazaar' development method.
III. Configuration Management
- Source code is not a program. Source code compiled with particular tools in a particular way is a program.
- What functions are wanted for SCM?
- Revision control.
- Build systems: how do you turn the source code into an executable?
- Testing environment: how do you run the program? How do you ensure you can test all functionality?
- Automated testing.
- Change tracking.
- Build systems: these are not trivial. There is much more than just the source code involved in creating a binary. Other variables include:
- Compiler version
- Compiler flags.
- Linker version
- Linker flags.
- Libraries used.
- Shared libraries.
- Environment in which program is run.
- Even small changes to any of the above can reveal previously hidden bugs. E.g., a different optimisation level on the compiler may shift a structure, which may put it out of alignment.
- Testing
- Building a test environment
- Keeping track of outside influences
- Regression testing
- Automated testing
- Change tracking: there are many reasons to make changes.
- What changes have been requested, and by whom?
- What is the intended purpose of the change?
- Who approved or rejected this change, and why?
- Who made the change, and what files were affected?
- Did it get on all the branches it needs to be on?
- Who reviewed and tested the change?
- Has this affected other program functionality? (Has it `spilled over' into another area of the program?)
- How do we back out the change, if necessary? Do other changes rely on this one?
IV. Further Information
Other Revision Control Systems
- Front ends for CVS: tkCVS
- Perforce
- ClearCase
- Intersolv PCVS
- Microsoft Visual SourceSafe
Other SCM Tools
- make
- Gnats: GNU bug tracking system.
- Tinderbox: runs builds and shows when the tree's gone up in flames. See
http://www.mozilla.org/tools.html.
Web Sites and Books
- Configuration Management Bubbles:
http://www.loria.fr/~molli/
- Information and FAQs on configuration management
- Annotated Bibliography and URL list
- Free and commercial CM tools
- Tools related to CM (such as new variants on
make)
- Problem management tools (bug tracking)
- CVS Bubbles:
http://www.loria.fr/~molli/cvs-index.html
- Perforce website:
http://www.perforce.com/
- O'Reilly Books:
- Bolinger and Bronson, Applying RCS and SCCS
- Jameson, Kevin, Multi-Platform Code Management
- Oram and Talbott, Managing Projects with make