You are currently browsing the Sticky Bits blog archives for May, 2011.

Configuration Items

May 30th, 2011

A Configuration Item (sometimes referred to as a Computer System Configuration Item, or CSCI) is an artefact (hardware and/or software) that has an end-user purpose – that is, it contributes, in some way, towards the attributes of a system or product, or the development processes followed to produce the system or product. A Configuration Item (CI) is an artefact that is treated as a self-contained unit for the purposes of identification and change control. For example:

  • Specifications
  • Design models
  • Source code
  • Make files
  • etc.

Configuration Item identification

Configuration identification is the process of identifying the attributes and procedures that define every aspect of a configuration item.

Configuration identification performs three distinct roles:

Uniquely identifying each Configuration Item.

CIs are the artefacts to be controlled under Configuration Management. Each CI must be uniquely identified by name, version (revision) and location (on the file system).

Establishing a change policy

The CI’s change policy defines how the artefact is revision controlled; and whether that control changes during the lifetime of the project.

Establishing a review policy

Ideally, every artefact should be reviewed. Configuration identification establishes how the artefact will be reviewed and when it will be reviewed.

Configuration Identification must be performed prior to establishing any revision control system. All CI attributes and policies must be recorded in the project’s CM plan.

Configuration lifecycles

Configuration lifecycles define how an artifact is checked into and out of the revision control system. The configuration lifecycle defines the ‘life’ of one particular change, as it applies to that artifact (c.f. Change Management, below). The lifecycle is typically shown as a state transition diagram.

For any real project of any complexity a number of lifecycles will be required. Also note, an artefact’s lifecycle may change through the lifetime of the project. This is defined by the artefact’s change policy.

Listed below are a number of common configuration lifecycles.

‘Archive’ lifecycle

Archive artefacts are configuration items that are simply stored in the CM system. Archive artefacts are never modified once they exist. For example, the binary (download) file for a particular release.

 

image

Figure 8 – The Archive lifecycle
 
‘Basic CI’ lifecycle

The ‘Basic’ CI lifecycle (Figure 9) is for artefacts that must be archive and may be modified over time. For example, the Compiler used on the project may be updated over the course of the project’s lifetime.

image

Figure 9 – the Basic lifecycle

In addition, this lifecycle is used for artefacts that have not yet been released (see Release Management, below). Such items should be stored in the project’s RCS repository (for completeness) and should have their modification history recorded but these items do not need to be subject to stringent review or quality checks before being checked in.

‘Reviewed CI’ lifecycle

A ‘Reviewed’ Configuration Item must pass a review (peer or formal) before it can be checked back in (Figure 10). For example, a Requirements Specification.

This lifecycle is sometimes known as a Document lifecycle.

image

Figure 10 – The Reviewed CI (or Document) lifecycle

Any artefact that has been released from development, or originates from outside the development department should be subject to this lifecycle.

‘Software CI’ lifecycle

The Software CI lifecycle (Figure 11) is an extension to the ‘Reviewed CI’ lifecycle (above). In addition to being reviewed, a software CI should also pass Static Analysis testing; and pass Unit (component) tests; before being checked in.

 

image

Figure 11 – The software item lifecycle.

The Software CI lifecycle can be implemented in environments that demand very high software quality, for example safety-critical applications.

 

To get the most out of your Revision Control System each project artefact must be defined, with its usage, review policy and, perhaps most importantly, its change policy. This information gives the developer the information needed when an artefact has to change – what is required and who must be involved. Having effective configuration identification in place also ensures that the project’s artefacts retain their quality (at whatever level that quality is set)

Revision Control

May 23rd, 2011

What is Revision Control?

Revision control is the management of multiple revisions of the same unit of information. The focus is on controlling access to the artefact; and recording the history of changes.

Revision Control is variously known as version control, source control, source code management and several other titles.

Revision control has its roots in the management of engineering blueprints and paper documents. Today, any practical application of Revision Control requires the use of specialist software tools.

Artefact Identification

The core to revision control is the artefact. An artefact is a unique unit of information that may change over time (or not – some artefacts are deliberately unchanging – see below). Every revision-controlled artefact is identified by a unique name made up of:

Artefact Name

The artefact’s name represents a human-readable identifier. This is typically how the artefact is referred to during the development process. The Artefact’s name is constant – that is, it does not change.

Revision identifier

The revision identifier is used to track the history of the information. It is usually a number (e.g. 1.1); the highest number is the most recent version (revision)

image

Figure 1 – The artefact identifier reflects the name of the unit of information and its history

As the artefact is modified (revised) the revision identifier is incremented and a new entity is created (see Figure 1). Thus, artefact MyDoc v1.0 is a different entity to MyDoc v1.1 since it contains different contents. One of the principles of revision control is that this process is hidden from the user, and they only see the most recent version, unless they explicity choose to access the artefact’s history (by accessing a previous revision).

Problems with ‘file system’ Revision Control

The simplest form of revision control is to use the directory system on your computer. Each revision of a product is kept as a separate directory on the disk (Figure 2). This is a simple technique but is rarely very effective.

image

Figure 2 – Storing file revisions using a file system is rarely effective

Since few file systems have a history facility built in, each revision must result in the storage of multiple (complete) copies of each artefact.

Normally, using a file system as a repository implies using a networked file server. Most modern file systems will recognise that a file is open for edited and restrict access (see File Locking, below). However, to improve responsiveness most users will make a local copy of their ‘working files’. This can lead to a problem known as ‘Latest-write wins’. This means, if multiple developers copy the same file to modify it (usually in different ways) then each copy their modified file back to the server this can lead to the loss of any previous changes; and only the last developer to save has their revisions included. Clearly, to control access to each file requires careful management to ensure consistency. Such systems, requiring process and procedures to retain consistency are easy to abuse, particularly in the heat of the rush to delivery.

Another problem comes when artefacts are shared between systems; especially if each product will make different demands, and require different modifications, to the artefact.

Using your file system for source control is, at best, adequate for one-man projects.

Revision Control Systems

For anything more than trivial revision control (for example, with more than one developer) then a Revision Control System (RCS) is required.

A Revision Control System (also known as a Version Control System – VCS) is a piece of software that acts as an archive for artefacts. The RCS stores the latest version of the artefact and all revisions. The developer can take latest revision (often called the ‘tip’) or any named revision.

Typically, the RCS stores first version of artefact, plus changes between one revision and the next (known as ‘deltas’). Artefact version numbering is usually automatically performed by the tool.

Access control

To ensure integrity of artefact revisions access control is required. Typically this involves locking the file against changes.

Acquiring the lock is referred to as Checking Out.

Committing the change (releasing the lock) is referred to as Checking In.

There are two common methods for achieving access control: File locking and Revision Merging

File locking

In a file locking system only one developer has write access to the artefact (Figure 4). Other developers will have read-only access to the current (stored) version. The file is only available again once it is checked back in.

image

Figure 4 – File locking allows only one developer at a time to modify an artefact.

File locking avoids complex merges due to large-scale changes since only one developer has access to the file and the ‘merges’ are essentially the new changes made by the developer.

Because a file may be checked out for a significant time developers may be tempted to simply circumvent the system

Revision Merging

Most systems allow multiple check-outs of the same file (see Figure 5).

If the file is checked-out multiple times the first developer to check-in always succeeds.

image

Figure 5 – Revision merging allows multiple check-outs on an object

Subsequent check-ins must merge their changes into the current revision. For simple merges this may be performed automatically by the RCS software. More complex merges may (and typically do) require human intervention.


Revision Control System Configurations

Revision control systems are designed to allow multiple users to access and modify artefacts from any location, either locally or across a network. There are two basic configurations of RCS – centralised and distributed. Each has its own merits; and the choice of RCS depends on the type of project being developed.

Centralised database systems

In a centralised system there is one master reference copy of all artefacts (Figure 6). Clients access the artefacts by making copies of a subset of central archive. This subset is a ‘view’ on the repository known as a Workspace. The workspace acts as an additional form of access control. The client is free to modify any artefact within their workspace, but may not modify anything not in their workspace (in fact, it should not be visible to them). Workspaces allow CM Managers to restrict access of staff to only the artefacts that are relevant to them.

image

Figure 6 – In a centralised system clients view a subset of the central repository

Centralised systems are best suited to geographically-close, commercial development projects, keeping all the company’s artefacts (which comprise their intellectual property) in a central location (for ease of back-up, etc.).

Distributed database systems

In a distributed system each client’s workspace is a bona fide repository. Each client has a full copy of the archive, complete with all revisions (Figure 7). The client is free to modify the database as they see fit.

image

Figure 7 – Multiple versions of the archive exist in a distributed system

Copies of the archives are kept synchronised with periodic updates (known as patches) sent between each of the RCSs.

Distributed database systems are widely used for open source software development. They are well suited to developments that may be geographically distant and being modify according to widely differing requirements.

What is Configuration Management? And why is it important to me?

May 16th, 2011

What is Configuration Management?

Configuration Management has its roots in the US Department of Defence in the 1950’s. It started as a technical management discipline and has been widely adopted by many other engineering disciplines, including Systems Engineering and Software Engineering. Configuration Management focuses on establishing, and maintaining, the consistency of a system or product throughout its lifetime. CM is a collection of competencies, techniques and tools whose purpose is to ensure the consistency of the system’s requirements, functional attributes and physical properties.

For the purposes of this paper we shall consider four fundamental, inter-related activities:

image

Revision Control

Revision control is concerned with controlling access to project artefacts and maintaining a history of changes to each artefact.

Configuration Identification

Before you can control any artefact you must identify what it is, what information it is going to contain and how it will be controlled.

Change Management

When an artefact has to change, Change Management controls when, or even if, the changes may be performed.

Release Management

Release Management focuses on the delivery of software outside the development department.

These activities are inter-related and highly dependent on each other.

 

Why perform CM?

Good Configuration Management ensures that the current design and build state of the system is known and recorded; and doesn’t rely on the tacit knowledge of the development team.

Being about to access an accurate historical record of system development is very useful – not only for project management and audit purposes, but for development activities such as debugging (for example, knowing what has changed between one set of tests and the next to help identify what could possibly be causing a fault).

Effective use of CM tools enables engineers to formally (and safely) record all change details. In particular, not only what was changed in the product but why it was changed

CM ensures that documentation can be produced which truly describes delivered systems.

Sometimes, it is necessary to rebuild old versions of products to reproduces problems for customers, such as the military, haven’t upgraded their tools to the latest versions.

Finally, and not to be overlooked (since these things happen!) good CM practices enable a system to be rebuilt correctly in the event of mistakes, failures and catastrophes.

 

Getting CM wrong

Configuration management can seem an awful lot of effort, much of which appears to be an overhead on the development of software. However, getting CM wrong can lead to the following:

– The wrong requirements being accepted

– The wrong design being implemented

– The wrong tools being used for development

– The wrong software being tested

– The wrong test suite being used

– The wrong version of software being released

Any of these could lead to huge amounts of wasted effort, wasted money, late deliveries and seriously dissatisfied customers. It doesn’t matter how good the software is, delivering the wrong product is never going to win you any plaudits.

Fundamentals of Configuration Management

May 16th, 2011

Configuration Management (CM) is a core process in any development activity. Software engineers realise this more than any other discipline, but for many software teams using Subversion as a source code repository is as far as they get. Configuration Management is a lot more than this.

These articles introduce the fundamental concepts of Configuration Management and what they mean to the development team. Revision control is the most well known and understood CM activity but few extend this to develop artefact change policies; and even fewer develop comprehensive review policies.

Change Management is the key to controlling your software development. Without Change Management projects rely on heroic efforts and (often!) blind luck to deliver.

I’ll introduce a new set of patterns to control branching within a source control system. Coupled with effective Change Management these patterns can save many hours of effort in maintaining the source control repository (a common problem in large repositories)

What is Configuration Management?

Revision Control basics

Defining your Configuration Items

Baselines and Branching

Change Management

Releasing Code

%d bloggers like this: