Series details: CRDA/56

The Bovine TB dataset is a record of the incidence of tuberculosis among cattle in Great Britain from 1977 to 2002; and record of incidence of tuberculosis among badgers, as potential carriers of disease, from 1972 to 1998. The purpose of this data was to assist Veterinary Officers working in the field. The work was part of the GB-wide tuberculosis management programme. The Tuberculosis (England and Wales) Order 1984 (made under Sections 32 and 34 of The Animal Health Act 1981), covers this programme of work; as does the State Veterinary Service Chapter 23.

The Bovine TB dataset was designed by Dr Roger Sainsbury as part of his work as a Veterinary Officer for the State Veterinary Service (SVS) in Truro. The State Veterinary Service operates throughout Great Britain on behalf of the Department for Environment, Food and Rural Affairs (DEFRA), the Scottish Executive Environment and Rural Affairs Department and the National Assembly for Wales.

The tuberculosis management programme is managed by DEFRA. Work on the central VETNET TbiC system was carried out by local veterinary surgeons and DEFRA vets.

DEFRA have issued the following disclaimer regarding this dataset: "This data was collated as part of an individual research programme by a member of the Ministry of Agriculture, Fisheries and Food (MAFF), now DEFRA. As such it must not be regarded as the official record of MAFF but as a representation of the early work in this field which is still ongoing." 1

The official method of recording Bovine TB information for the State Veterinary Service is by using the Great Britain-wide VETNET TbiC screens. The VETNET database was created in 1986. The system contains Herd and 'not negative' animal information, dates when tests were carried out, and dates when the next test is due. This system is centrally controlled. It is common practice for local offices of the SVS to keep 'unofficial' spreadsheets where the same information is held, but in a more flexible way for the purposes of manipulation and cross-referencing. Central VETNET administrators would probably consider these local versions as duplications of VETNET.

The dataset transferred to NDAD is not VETNET, but a local database developed by Dr Roger Sainsbury in Truro. Dr Sainsbury was a MAFF employee at the time. The system links data showing incidence of TB to computer-generated maps; the original name of the database was "TB Maps & Stats". The Bovine TB dataset was different from VETNET in the following ways:2

• It made cattle data available in the OUTPUT mapping programme
• It displayed data in a graphic format printout which was useful for field investigations
• It included information about badgers which had been examined
• It could combine badger and cattle data and give an overall disease picture

The history of the development of the system (including a written description of the early OUTPUT program) can be found in the Dataset Documentation catalogue.

The original system was a program system constructed from a number of program modules. There were originally four databases: (1) TB Herd database; (2) Cattle database; (3) Badger database; (4) Parish database. Data from the four separate databases were combined by the different programs to produce the program output.

In the end, only three of these databases were transferred to NDAD. The National Archives, at time of transfer, took the decision to exclude the Cattle database.3

The reason for this exclusion is as follows. In the original system, the Cattle database was used to input details of individual animals from the TB60 form; records were then aggregated to the herd level and transferred to the TB Herd database. There was not sufficient computer memory available to include the full Cattle data linked to TB Herd as a relational database. Because the program suite was intended to assist TB investigations, the Cattle data was not considered by TNA to be an essential component in recreating the application in NDAD's preservation copy of the dataset.4

Since the system was effectively defunct from the end of 2002, no further accruals to the Bovine TB dataset are expected.

The Bovine TB datasets and dataset documentation are public records under the Public Records Acts 1958 and 1967. The National Archives has assigned these datasets and documents the series reference MAF 430.

The Bovine TB datasets are open without restriction, except for certain location, name, address and grid reference fields. These fields remain closed for 50 years under Section 31a (Law Enforcement) and Section 38 (Health and Safety) of the Freedom of Information Act. These closed fields will open at end of the agreed 50 year period on an annual basis, for example the fields in the 1977 table will open in 2028, and those in the 2002 table will open in 2053. The earliest opening will be the fields in the 1972 table in Badger dataset, which will open in 2023. For further details on field closures, see the dataset catalogues (Links to dataset catalogues).

Two items of dataset documentation have been redacted in accordance with the same FOI requirements as above. The remainder of the documentation is open without restriction.

The Bovine TB datasets and related dataset documentation are Crown Copyright. Copies may be made for private study and research purposes only.

The Bovine TB datasets are subject to registration under the Data Protection Acts.

The language of the materials is English.

Documents relating to the Bovine TB datasets have been transferred to NDAD and are listed in the Dataset Documentation Catalogue.

Although no reports specifically connected to this dataset have been published, users may wish to study MAFF reports on the subject of Bovine TB and badgers published during the period covered by the dataset. These include Badgers, Cattle and Tuberculosis: Report to Rt Hon Peter Walker MP by Lord Zuckerman, London, HMSO 1980; Badgers and Bovine Tuberculosis: Report to Rt Hon Michael Jopling MP and Rt Hon Nicholas Edwards MP by Professor G M Dunnet et al, London, HMSO 1986; and Bovine tuberculosis in cattle and badgers: Report to the Rt Hon Dr Jack Cunningham MP by the Independent Scientific Review Group (Chairman Professor John R Krebs) London, MAFF 1997.

At the time of transfer to NDAD, the system was running on networked PCs in the Truro Animal Health Office; it had also been in use in other West Country offices. It was originally developed and run on a Commodore 8096 microcomputer, with additional development work carried out on a BBC Model B microcomputer.

The latest version of the system was run under the MS-DOS operating system; although not confirmed, it is likely that prior versions would have run under CBM DOS (Commodore 8096) and Acorn MOS (BBC Model B).

The Bovine TB application, known as "TB Maps & Stats", was developed using the BASIC programming language; initially BBC BASIC and Commodore BASIC, and since 1994, Microsoft QBasic.

The original application comprised several discrete program modules which were accessed by a compiled QBasic menu front end. The modules handled the data management and report generation for four databases, three of which have been accessioned by NDAD. The original modules included: badger data input; IR TB60 form generation; cattle data entry and updating; TB mapping; TB statistics; parish data updating; and a search module. Renditions of screen images illustrating the original system in use can be found in the Dataset Documentation Catalogue, reference CRDA/56/DD/1/10.

The series includes three entities: the TB Herd database, holding data on the incidence of TB in cattle herds in Devon and Cornwall; the Badger database, holding data on West Country badgers and their post-mortem results; and a Parish database holding data on test intervals in Cornish parishes.

The original system was a program system constructed from a number of program modules. There were originally four databases: (1) TB Herd database; (2) Cattle database; (3) Badger database; (4) Parish database. Data from the four separate databases were combined by the different programs to produce the program output. Only three of these databases were transferred to NDAD. (See Scheduling information above for further detail on this.)

Further details regarding the structure and schema of these individual databases are available in the Dataset Catalogues; see Links to dataset catalogues.

The TB Maps & Stats system began with information collected about badgers; Roger Sainsbury wrote a program in 1982 to produce an output, in tabular form, of badgers and their post-mortem results which had been examined by Veterinary Officers from around tuberculosis breakdown farms.

In 1983, mapping capability was added to the basic badger data. Dr Sainsbury wrote a further program to plot the badger data onto schematic maps covering an area around the breakdown farms. The need for accurate and relevant maps in the field is discussed in a brief report to the DRVO (see the Dataset Documentation Catalogue, reference CRDA/56/DD/1/1). Maps used for investigations by the Animal Health Division at that time were either OS Maps with additional information drawn onto them, or hand-drawn maps which were potentially inaccurate. TB maps were needed in the field, at progress review meetings, and for head office and regional use.

The resulting OUTPUT program was used with success, and maps covering TB farms were produced for vets in Cornwall and Devon for nearly 20 years. From here, a further series of programs were written which constructed a cattle database, in response to the need for TB cattle herds to be included on the maps. The program which generated the maps was modified to access the data to find relevant herds, which were in turn added to the maps.

Data was entered using DOS screen forms created using QBasic. Online instructions were provided, including indicators of valid data for some fields; some validation of data took place during data entry.

Cattle data was input by the TB section at the Animal Health Office, when a TB60 PM sheet was being prepared. The data captured at this point was used to produce the TB Herd database and the cattle database. The information about parishes was input at a very early stage, and was rarely modified. The badger data was input separately, although there were discussions to find a way to transfer badger data electronically from the MAFF badger system.

The system enabled the printout of maps or data lists, on paper, to be used by Veterinary Officers in the field. Among the possible outputs were data about a series of reactors, schematic maps, the TB history of a location or particular farm, parish lists, TB statistics, and some data analysis.

For more detail on outputs, see the written descriptions of the system in the Dataset Documentation Catalogue.

Details of the content and transformation validation checks performed by NDAD staff on the Bovine TB dataset are contained in the dataset catalogues: see Links to dataset catalogues.

Dataset catalogues provide more detailed information about individual datasets, and are currently available for the following dataset(s):

1. Internal DEFRA email from the Departmental Record Officer dated 19 March 2007, copied to TNA and NDAD. 'Ongoing' refers to the research work into tuberculosis, not to the Bovine TB dataset.

2. Email between DEFRA and The National Archives, 12 December 2001.

3. Email messages between NDAD and The National Archives, February-March 2004.

4. "As I understand it: the process starts with the slaughter of the cattle, so because it begins with the cow and not the herd, cattle are input on to the cattle table and then those that are TB relevant are transferred to the Herd table where the data is aggregated up to herd level. If Roger had had room to feed the whole cattle table into the memory he would have had a proper relational database, with one table for cattle, which had a herd key, relating to a table with herd details, but as there wasn't room, he aggregated the cattle and transferred them to the herd table. As this is a program suite designed to compare and map incidences of Badger post mortems around TB outbreak farms, the data units we are dealing with are badgers and herds, and their associated parishes. Therefore in order to recreate the program outputs this is all that would be needed...I am not sure how much extra use the cattle table would be for this - we know Roger himself has said it would be 'potentially useful for epidemiology'. I feel we can either take the view that we have enough data to recreate the Cornish TB database program, which is of historical interest and importance, given the subject matter, and as an early example of a GIS, and also of a government program suite that by-passed the IT department and was written by a lone scientist, and leave it at that." (Email from The National Archives to NDAD, 23 February 2004).