Series details: CRDA/1

The datasets derived from the Metropolitan Police's Crime Statistics System (known within the Metropolitan Police by the code "ME") contain data relating to crimes reported within the Metropolitan Police District which were input to the Crime Statistics System between 1990 and 1997. 1 They include details of offences, clear-ups, arrests, victims of crime, property stolen, and reports classified as "No Crime" (defined as "an allegation where the evidence is insufficient to establish that a crime has been committed"). 2

When it was implemented in 1989, the ME system included the following components:

• An ICL IDMSX database designed to hold 3 years' worth of data (about 3 million records).
• Monthly files and year-to-date files extracted from the database for use in ad hoc enquiries and for tabulations.
• Historical files of annual data.
• Various report writing programs.
• A local data inputting system used by the Metropolitan Police's G10 (Statistics) Branch.
• Local means of accessing the system used by G10 Branch.

Data was input by G10 Branch and its successor, the Performance Information Bureau (PIB) using copies of paper crime reports submitted by police stations. The data was sent to the Department of Computing Services (DCS) and its successor, the Department of Technology (DoT) for loading onto the database. The database and associated files were used by G10 and PIB for ad hoc enquiries and to produce monthly, quarterly and annual tables of crime statistics (e.g. types of crimes, numbers of offences recorded and cleared up, and persons arrested, by police division, sub-division and area, and by London borough). In 1989 185, 175 and 175 tables were produced for G10 each month, quarter and year, respectively (in 1991 this was reduced to 169, 164 and 167 tables each month, quarter and year, respectively). These tables were in turn forwarded to the Home Office and circulated within the Metropolitan Police for use in publications such as the Commissioner's annual report, press releases, and answers to parliamentary questions. Further details on the gathering and inputting of the data are given in How data was originally captured and validated.

The ME System was designed to replace an earlier version of the Crime Statistics System known as MC. This system also consisted of a database and associated files maintained by DCS and, as with the ME System, inputting and analysis of data from paper crime reports was carried out by G10. The replacement of MC by ME was prompted by DCS's desire in the mid-1980s to move its applications from a DME to a VME operating system, and to decommission an ICL 2966 mainframe computer at Jubilee House, Putney which was used for running the MC System. DCS and G10 took the opportunity of the move from DME to VME to revise the structure of the Crime Statistics database. The new database was intended to use ICL's Quickbuild products (ICL ReportMaster, ApplicationMaster, etc), and to reduce maintenance overheads by providing a means for quicker enhancements and amendments to software and inbuilt security through standardised utilities. At the same time, steps were taken to improve G10's ability to access and manipulate crime statistics data through the development of a transaction processing service (for further details, see How data was originally captured and validated).

Work on the conversion of DCS's DME applications to VME started in 1985 with two pilot projects involving data on traffic accidents and juvenile bureaux systems. In the following year DCS began work on designing a new Crime Statistics database. At that time the new database was scheduled to be operational by mid-1986. In the event, a series of delays and postponements meant that final implementation of ME does not appear to have occurred until late 1989. A service level agreement relating to the System was signed between G10 and DCS in March 1990 (see the Dataset Documentation Catalogue, reference CRDA/1/DD/1/5/1). Delays were caused by the following factors: the need to develop new statistical tables as a result of the reorganisation of the Metropolitan Police into eight areas in 1986; changes in G10's requirements regarding the amount of data to be stored in the database; the need to test the new report writing and tabulation packages; and apparent communication difficulties between G10 and DCS. A working party of G10 and DCS officers met for the first time in May 1986 to co-ordinate the conversion of the Crime Statistics System.

It should be noted that the ME System was originally designed as an interim system which would meet G10's needs prior to the implementation of the Metropolitan Police's Crime Report Information System (CRIS). The aim of CRIS was to replace paper crime reports and the central inputting of crime data with a network of dispersed terminals, which would allow police officers to input details of crimes directly to a database and to conduct on-line searches of the data. Development of CRIS started in 1981 and continued throughout the 1980s (£22.3 million were spent on the development of CRIS between 1987/88 and 1992/93). 3 A number of slippages occurred in the system's implementation date. By mid-1988 it was believed within DCS that CRIS would replace the ME System for the production of all new crime statistics in 1990 or 1991, after which ME would be used for about five years solely for access to historical data. In the event, field trials of CRIS did not start until 1992. The system began to be rolled out on an Area by Area basis within the Metropolitan Police beginning in 1994, and finally went live throughout the Metropolitan Police District on 1 October 1996. This meant that the ME System continued to be used for the recording of crime data and the production of crime statistics well beyond its intended lifespan. Indeed, the datasets transferred to NDAD suggest that even after CRIS was fully operational, the ME System continued in use for a limited period (until the end of the 1996-1997 financial year).

As the development of CRIS progressed, G10 and later PIB became increasingly concerned to ensure that CRIS data and data generated on the ME System should be compatible during the transitional period when both systems would be operating simultaneously. During the initial stages of the introduction of CRIS, paper printouts of crime reports produced by divisions using CRIS were input to the ME System by PIB, though this proved to be labour intensive. By 1992 DCS had started work on developing software to allow for the electronic data interchange (EDI) of ME and CRIS data. This required certain modifications to the ME System: initially, a shift from a calendar year to a financial year basis for producing datsets; later, more substantial modifications to the ME database. These changes are reflected in the datasets transferred to NDAD (see Scope and content), and appear to have been completed in the 1994-1995 financial year. The upgraded database was referred to as "MEL" by its users in PIB, who also acquired a new system for inputting data (see How data was originally captured and validated). New software for tabulating data and running ad hoc queries of the redesigned database appears to have been developed by the contractors working on the CRIS system. Some retrospective conversion of data generated on the older version of the database may also have occurred.

The ME System was developed and maintained by the Department of Computing Services (DCS) of the Metropolitan Police and DCS's successor, the Department of Technology (DoT). The system was used by G10 (Statistics) Branch and its successor, the Performance Information Bureau (PIB). For information on the history of these divisions, see the Administrative History of the Metropolitan Police.

Introduction

The datasets derived from the ME Crime Statistics System fall into the following categories:

• Datasets for the years 1990-1992 (references: CRDA/1/DS/1/1-3), where the division between one year and the next is based on a calendar year (1 January-31 December). Each of these datasets comprises two flat files: a year end extract file (YTDEXTRACT), consisting of data relating to offences, clear-ups, arrests and victims input during that year; and a year end "No Crime" extract file (YTDNOCREXT) holding data on allegations classed as "No Crime", also input during the year in question.
• Datasets for 1992-1993, 1993-1994 and 1994 (CRDA/1/DS/1/4-6) where the division between one year and the next is based on a financial year: i.e. they consist of data input to the ME System between 1 April and 31 March. The structure of these datasets is identical to that of CRDA/1/DS/1/1-3, except for the 1994 dataset which does not include a "No Crime" file (this dataset also differs from the others in that it has been truncated: rather than covering the whole 1994-1995 financial year, it does not include any data added to the system after September 1994). It is unclear why the ME datasets moved from a calendar to a financial year in 1992, but it is likely that this was connected to the transition to the CRIS System (see Aim and purpose), which employed a financial year.
• Datasets for 1994-1995 and 1995-1997 (references: CRDA/1/DS/2/1-2), again based on a financial year, with the second including two years' worth of data. Each dataset comprises a single YTDEXTRACT file which reflects changes to the ME System to allow for compatibility with CRIS. Further details of these differences are given below.
• A dataset (reference: CRDA/1/DS/3/1) which is believed to correspond to the data in the ME database at the end of the System's lifespan. It includes data which was input to the system between February 1992 and March 1997.

With the exception of CRDA/1/DS/3/1, it is thought that the ME datasets originated in "year-to-date" files which were produced as a cumulative record of data input to the database each year. They were updated on a monthly basis with data extracted from the database, and were "archived" at the end of the calendar or financial year.

The time periods covered by the above datasets indicate that there is often a considerable overlap of data from one dataset to another. Equally, it should be kept in mind that the chronological division between datasets is based on the dates when the relevant records were added to the ME System, not the dates when incidents occured or were reported to the police. It is clear that offences and "no crimes" were often reported well before the date when the record was input. Records of arrests and clear-ups might relate to an offence which was reported many years before, where an arrest or clear-up was made in the period covered by the dataset. In other cases administrative delays were presumably responsible for the gap between the reporting of a crime and the input of data relating to it. It should also be noted that while the ME datasets contain data on individual crimes, they do not include the names of crime victims or offenders.

Additional information on the contents of the two main categories of datasets is given below.

Calendar year datasets for 1990-1992; financial year datasets for 1992-1994 (CRDA/1/DS/1/1-6)

These datasets predate the changes to the structure of the ME System which were made to accommodate CRIS. Each consists of a year end extract (YTDEXTRACT) file and a "No Crime" extract file (YTDNOCREXT), except for CRDA/1/DS/1/6 which only comprises a YTDEXTRACT file.

The YTDEXTRACT files contain the following types of information relating to offences, clear-ups, arrests and crime victims:

• The record type: i.e. whether the record contains details of an offence which has been reported but not cleared up, an offence which has been cleared up, a clear-up not connected with a previously recorded offence, an arrest, or a victim of crime.
• Classifications of the type of crime according to the classification schemes used by the Home Office (known as "HO classes") and the Metropolitan Police (known as "CO classes").
• The type of Metropolitan Police Crime Report Book in which the paper crime report was filed (this acts as a broad classification of the offence: e.g. burglary, cheque fraud, shoplifting); the serial number of the crime report within the Crime Book.
• When the crime took place: i.e. the date that the offence was reported or that the arrest was reported; the hour of the day and day of the week when the offence occurred; and the year and month when the details were input to the database.
• Where the crime took place: i.e. the London borough or other local authority area, Metropolitan Police Area, Metropolitan Police Division and Subdivision (i.e. police station) where the offence was reported or the arrest was reported.
• The manner in which the offence came to the attention of the police.
• Details of the commission of the offence: i.e. any means of attack, weapon and means of getaway used; the number of attackers; any relationship between the victim and the attacker; the type of any drug involved; the nature and value of stolen property; the value of recovered property; whether the vehicle was recovered (in the case of crimes involving vehicles); and the number of times that stolen cheques or credit cards were used (in the case of offences of obtaining by deception).
• The number of victims of the offence and of prisoners held.
• In the case of arrests: the manner in which the arrest occurred; the type of police officer who made the arrest; the nature of any proceedings against the prisoner; the age, sex, and ethnicity of the prisoner; whether the prisoner was bailed; and whether the prisoner had previously been convicted of an offence.
• In the case of victims of crime: the age, sex and ethnicity of the victim and the nature of any injuries sustained.

More limited details are entered in the year end "No Crime" extract files (YTDNOCREXT), which only hold data on allegations classed as "No Crime":

• The record type (i.e. the fact that it is a "No Crime" record).
• The type of Metropolitan Police Crime Report Book in which the paper crime report of the "No Crime" was filed (this acts as a broad classification of the allegation: e.g. burglary, cheque fraud, shoplifting); the serial number of the crime report within the Crime Book.
• Classifications of the type of "No Crime" allegation according to the classification schemes used by the Home Office (known as "HO classes") and the Metropolitan Police (known as "CO classes").
• The date that the "No Crime" allegation was reported, and the year and month that the data was input to the database.
• Where the "No Crime" allegation was reported: i.e. the London borough or other local authority area, Metropolitan Police Area, Metropolitan Police Division and Subdivision (i.e. police station), and the Metropolitan Beat Code for the location where the "No Crime" was reported.

Financial year datasets for 1994-1997 (CRDA/1/DS/2/1-2); final database (CRDA/1/DS/3/1)

These datasets reflect changes to the structure of the ME System which were apparently made in 1994-1995 to facilitate concurrent running with CRIS (see Aim and purpose). Most of the information which they record about offences, arrests, clear-ups, victims and "no crimes" is broadly similar to that in CRDA/1/DS/1/1-6. However, there are a number of fundamental differences:

• "No crime" data is recorded along with data relating to offences, arrests, clear-ups and crime victims in a single extract file (YTDEXTRACT in the case of 1994-1995 and 1995-1997, DBEXTRACT in the case of the final database extract). These files are slightly smaller than the YTDEXTRACT files in CRDA/1/DS/1/1-6, in terms of the number of fields (60 fields versus 65 in CRDA/1/DS/1/1-6). Different field names are employed than in the preceding datasets, though in the majority of cases there was no change in fields' functions.
• Crimes and "no crimes" are no longer classified according to the Metropolitan Police's crime classification system ("CO classes"); only the Home Office classification scheme is employed. The set of HO codes differs markedly from that employed in CRDA/1/DS/1/1-6, although many codes are effectively the same. In addition, the datasets contain fields for Home Office Classification Prefixes ("used by the Home Office to distinguish between Attempt, Conspire, Incite, aid and abet, and Substantive offences") and Home Office Classification Suffixes ("used . . . to indicate relationships between attackers and their victims in crimes of violence, and whether there was any violence in a robbery"), which do not appear in the pre-CRIS datasets.
• There are fields recording detailed classifications of the "venue" (location) where an offence occured (VENUE-CODE, represented in NDAD as VENUE-CODE-MAJOR and VENUE-CODE-MINOR), and any vehicle or plant involved in an offence (VEHICLE-CODE), which do not appear in the earlier datasets.
• Types of stolen property are classified in greater detail than in the earlier datasets, with four fields (PROPERTY-1, PROPERTY-2 etc) set aside for stolen property details. Each of these is represented in NDAD as two fields: PROPERTY-1-MAJOR, PROPERTY-1-MINOR, etc.

The Crime Statistics System has now been superseded by CRIS, and it is believed that all data generated on the ME System has been transferred to NDAD. No further transfers of ME datasets to NDAD are therefore anticipated.

The datasets derived from the ME Crime Statistics System and related documentation are public records under the Public Records Acts 1958 and 1967. The Public Record Office has assigned the datasets the class reference MEPO 36. The documentation has been assigned to classes MEPO 36 and MEPO 37 (see the Dataset Documentation Catalogue for further details).

The datasets derived from the ME Crime Statistics System are open without restriction. Data is available for browsing on demand by users of NDAD and does not require to be booked in advance. Some of the documentation relating to the datasets is closed for 30 years, although open extracts from CRDA/1/DD/1 and CRDA/1/DD/5 have been made available (see the Dataset Documentation Catalogue for further details.)

Copyright of the datasets derived from the ME Crime Statistics System and related documentation rests with the Commissioner of Police for the Metropolis. Copies may be made for private study and research purposes only.

The Metropolitan Police have indicated that the datasets derived from the ME System are subject to registration under the Data Protection Act. Subject access to data as defined by the Act is permitted. Authorisation for the provision of subject access to data rests with the Metropolitan Police.

The language of the materials is English.

Registered files and other documents relating to the ME Crime Statistics System have been transferred to NDAD from the Metropolitan Police, together with a large number of programs associated with the system. See the Dataset Documentation Catalogue for further details of these items.

Crime statistics derived from the ME Crime Statistics System were incorporated in the annual Report of the Commissioner of Police of the Metropolis for the years covered by the datasets.

ICL 3960 mainframe computer manufactured by International Computers Limited (ICL), installed in Metropolitan Police premises at Jubilee House, Putney. An ICL 2966 and an ICL 2988 were used for development work on the ME database.

For details of the hardware used by G10 and PIB to access the database and input data, see How data was originally captured and validated.

VME operating system manufactured by ICL, with an SCL-based interface to queue jobs on the mainframe.

IDMSX (Integrated Database Management System), manufactured by ICL, was used to create the ME database. According to ICL, "IDMSX provides a database management system for storing a central pool of information or data, which is held in the form of an IDMSX database. The data may be stored, retrieved and updated by application programs". IDMSX is normally used with ICL's Data Dictionary System (DDS), which provides the basis for the use of other ICL products such as Application Master and QueryMaster. Application programs for IDMSX can also be written in COBOL, "extended by IDMSX Data Manipulation Language (DML) statements".4

ICL's ReportMaster and the Office for Population Censuses and Surveys' Tabulation Utility (TAU) were used for the production of tables. TAU (which had previously been used with the MC system) was phased out in 1991 when it was replaced with an in-house tabulating program, due to changes in the type of tables produced for G10.

SPSS-X 3.0 (manufactured by SPSS Inc, Chicago, USA) was used for ad hoc enquiries of monthly and year-to-date files and back data. This package and ICL's QueryMaster (see below) replaced two tabulation packages called FIND and SDTAB which had been used with the MC System.

ICL's QueryMaster was used for ad hoc enquiries of the database, monthly, year-to-date and historical files. According to ICL, "QueryMaster (QM) provides comprehensive selection facilities to retrieve data and text from an IDMSX database or from conventional files. The retrieved data is displayed on the screen, printed, output to a file or transferred to a terminal. Data retrieval and selection facilities include: Automatic use of CAFS and keys where appropriate; Sorting of retrieved data; Automatic navigation between record types. The end-user does not need to know details of internal database structure and data storage in order to access the data. The data definitions are held in the Data Dictionary System and are accessed by the Query View Compiler to create a view of the data for the end-user".5

ICL's Transaction Processing Management System (TPMS(X)) was used to manage G10's access to the database via the transaction processing service (see How data was originally captured and validated), while ICL's Application Master was used to set up specially designed screens which G10 could use on their DRS300 system for functions such as correcting errors arising from the daily loading of data onto the database. According to ICL, TPMS(X) "provides a complete environment, within the VME and OpenVME operating systems, in which the user is able to manage all his transaction processing requirements in a controlled and reliable way. Major features include: incorporation of applications constructed using COBOL and Application Master (AM); generation of a consistent TPMS(X) service description, using the facilities of the Data Dictionary System (DDS). It also provides a full range of facilities for screen design, recovery, security, testing, message routing, output spooling, auditing, service control, management of terminal response times, training and distributed transaction systems".6

Application Master is described by ICL as "a 4th generation language for application definition. The language components are held in a Data Dictionary System. The Application Master compiler generates interactive and non-interactive programs from the data held in the dictionary. An interactive program will typically form part of a FORMS or non-FORMS TP service. Application Master can be used to generate application programs which control the sequence of the screens seen by the user of the service. An interactive Application Master program can: Format data output to the screen; Validate the data input; Access and update data held in an IDMSX database; Manipulate data obtained from screens or from variables. A non-interactive program will run as a MAC or Batch job and can produce a report or an extract file from either an IDMSX database or a conventional file. A non-interactive Application Master program can: Produce reports; Access and update data held in an IDMSX database; Access and update data held in a conventional file; Manipulate data obtained from variables; Reorder information extracted from an IDMSX database or file".7

By 1992 Microsoft Excel 3.0 was used by PIB to manipulate data downloaded from the ME database onto local PCs.

As previously stated (see Scope and content), the calendar year datasets for 1990-1992 (CRDA/1/DS/1/1-3) and the financial year datasets for 1992-1993 and 1993-1994 (CRDA/1/DS/1/4-5) consist of two files for each dataset (a YTDEXTRACT file and a YTDNOCREXT file). The "truncated" financial year dataset for 1994 (CRDA/1/DS/1/6) consists solely of a YTDEXTRACT file. The financial year datasets for 1994-1995 and 1995-1997 (CRDA/1/DS/2/1-2), by contrast, each consist of a single YTDFEXTRACT file holding all data on offences, arrests, victims and "No Crime" allegations for that year. A further dataset (CRDA/1/DS/3/1) consists of a file which holds the data in the ME database at the end of its operational life. The format of this file (DBEXTRACT) is essentially identical to the YTDEXTRACT files in 1994-1995 and 1995-1997. Additional information on the structure of individual datasets is provided in the dataset catalogues: see Links to dataset catalogues.

A distinction should be made been the structure of the datasets - most of which originated in cumulative extract files archived each year - and the structure of the original IDMSX database. This is believed to have been a hierarchical database centered around tables holding data for different types of records, consisting of:

• "A" records, recording details of arrests
• "B" records, recording details of offences not cleared up before being entered into the database
• "C" records, recording details of offences reported and cleared up before being entered into the database
• "D" records, recording clear-ups not matched to a previous offence record
• "V" records, recording details of crime victims
• "N" records, recording allegations classed as "No Crime"

In addition to tables for these types of records, the database also included files concerned with the validation of data loaded onto the database and files designed to assist the searching of the database.

The original ME database was dynamic in that it was continually updated as crime reports were received. However, the datasets corresponding to the files of data extracted in annual archival runs (CRDA/1/DS/1/1-6 and CRDA/1/DS/2/1-2) could be said to be closed, in that once the files comprising these datasets were extracted by DCS and the Department of Technology, they were archived and not subsequently overwritten by other data.

The ME Crime Statistics datasets are based on data extracted by G10 and PIB from paper crime reports completed by police officers and their supervisors. These reports (known as "Form 478") recorded details of individual incidents reported to the police, including the nature of the alleged offence, details of victims, suspects, informants and property stolen, details of the investigation, information about physical evidence, and information about arrests. Photocopies of crime report forms, authorised by supervising officers, were required to be sent to G10 "at the earliest opportunity".8 Copies of Form 478 were also sent to other branches of the Metropolitan Police: e.g. divisional crime analysis units and SO11 (Crime Pattern Analysis) Branch. Original forms were retained in "crime books" (i.e. ring binders) at police stations according to whether they related to major crime, burglary, "beat" crime, vehicle crime, or child protection teams. For a specimen of Form 478, see the Dataset Documentation Catalogue, reference CRDA/1/DD/2/2.

Information from crime reports was encoded and input by staff in G10's input section using a local input system. At the time that the ME System was implemented in 1989, this local system was a Microdata 8000 series consisting of two M8006 file processors, two inboard ports, one system resource manager, four 4-port application processors, four 8-port application extensions, one 2000 character terminal and serial printer, thirty-two 2000 character VDU terminals with standard keyboards, and one secondary dataway master. It was acquired from Microdata Information Systems, Hemel Hempstead, Hertfordshire and was installed at G10 in late 1984 and early 1985. Responsibility for supporting the system later passed to McDonnell Douglas. The system replaced an earlier Datapad input system, installed in 1977, which had consisted of three NOVA 2 processors (each with 32KB of storage), thirty Datapad stations (each with a 32 character display), three teletypes and three 8-track 150 cps paper tape punches. The Datapad system had produced paper tape which was sent to DCS for inputting to the MC Crime Statistics database, supplemented by manually produced punch cards at times of high workload or during processor failures. By contrast, under the Microdata system data was loaded onto magnetic tapes which were sent by G10 to DCS on a daily basis.

The batch input of data continued to be the normal method of adding data to the database after the ME System was introduced. However, one major change that accompanied the development of ME was the introduction of a transaction processing service which allowed users in G10 to input, update, view and amend records; correct errors arising from the daily loading of data onto the database; and conduct ad hoc enquiries of the database and the monthly, year-to-date and historical files. An ICL DRS300 system consisting of a processor stack, printer and four terminals was installed at G10's offices in Bessborough Street, Pimlico. This was connected to the ME System running on an ICL 3960 at Jubilee House by a MICROLAN communications network. Although G10 thus acquired the capacity to add data directly to the database, this "interactive update" facility was only supposed to be used in emergencies. The Microdata system was also able to store up to eight months' worth of reports in its own local database.

Validation checks on the data were performed using the Microdata software at the time of inputting by G10. Further checks were carried out at the time that data was loaded onto the mainframe at DCS. This continued to be the case after the introduction of the ME System. However, G10 were now able to correct errors thrown out by the daily loading of data, by using their DRS 300 system, MICROLAN connection and transaction processing service.

By 1994 the Microdata system was nearing the end of its operational life. G10's successor, PIB, acquired a new inputting system based on networked PCs and a server. This system was connected to the mainframe running the database at Jubilee House, and involved a UNIX operating system, a Windows-type environment and inputting software created for PIB by the Department of Technology. Batch inputting of data was now replaced by the direct inputting of data to the database. Acquisition of the new system coincided with the move of PIB from Bessborough Street to offices at Tintagel House, Albert Embankment. The Microdata equipment was left behind at PIB's former location.

The datasets derived from the ME Crime Statistics System record details of crimes and allegations of crimes which were reported to the Metropolitan Police. This means that statistics based on them will suffer from the defects common to all statistics of "reported" crime: i.e. they will not cover unreported crimes, or crimes and allegations reported to the police but not recorded by them, and will be vulnerable to changes in police recording practices and in the level of reporting to the police.9 Victimisation surveys such as the British Crime Survey (conducted by the Home Office) have been developed to measure the "true" extent of crime by asking samples of the population about their experiences as crime victims. See the Series Catalogue for the British Crime Survey for further details.

All queries intended to produce meaningful statistical results from the YTDEXTRACT files should include selection by record type: a count of all the records in the file without specifying record type will not give the total of all the offences input to the ME System in the year in question. In the YTDNOCREXT files, however, all records are of type "N", i.e. no crime.

The significance of certain values relating to Metropolitan Police divisions and subdivisions, Home Office and Metropolitan Police crime classifications, and several other indicator codes in the data remains unexplained by either the metadata or other documentation provided by the Metropolitan Police. These instances are clearly flagged in the field descriptions.

The ME coding system is extensive and exhaustive, and complex relationships exist between the Home Office (HO) and Metropolitan Police (CO) classifications of offences and other indicator codes in the records (e.g. Offence-Drug-Involved-Ind in CRDA/1/DS/1-6) which are largely undocumented. Anyone wishing to compare published figures with results achievable from analysing data in the NDAD system is strongly advised to study the COBOL and Reportmaster programs which produced the original figures, to see the exact nature of queries originally submitted to the database. See the Dataset Documentation Catalogue for details of these programs.

Care should also be taken when interpreting fields with zero or blank values. In conception, the records in the YTDEXTRACT files (in CRDA/1/DS/1/1-6 and CRDA/1/DS/2/1-2) and the DBEXTRACT file (in CRDA/1/DS/3/1) are effectively divided into four sections:

1. Data about the record (record type, dates reported and input)
2. Data about the offence reported
3. Data about an arrest
4. Data about a victim

In the event that an offence is a clear-up, with one prisoner and one victim, all this information can be contained in a single record. Additional arrest or victim records will contain significant data in the section for offence details, and in one other section, according to the record type; information in the remaining sections will be redundant or blank. "No crimes" in the YTDEXTRACT files in CRDA/1/DS/2/1-2 and the DBEXTRACT file in CRDA/1/DS/3/1 appear as single records, with the "no crime" type being indicated by the EXT-REC-TYPE and the NO-CRIME fields.

For the purposes of displaying fields on the NDAD system, where there are blank or missing values we have tried to distinguish between genuine zero values (which will be displayed as 0) and values either missing or not appropriate to the record type (displayed as *). Nevertheless, in some cases, particularly where zero is a valid value for a field, care should be taken in interpretation. For example: if a record has the value zero in Offence-No-Of-Attackers (in CRDA/1/DS/1/1-6), this may be due to the fact that there were no attackers, or it may simply reflect the fact that no information was entered. The solution is to cross-check with other codes in the record, particularly the HO and CO codes: not all crimes are crimes of violence.

Details of the content and transformation validation checks performed by NDAD staff on each ME System dataset are contained in the catalogues of individual datasets: see Links to dataset catalogues.

Dataset catalogues provide more detailed information about individual datasets, and are currently available for the following dataset(s):

1. This catalogue is based primarily on information in seven registered files from the Metropolitan Police's DP series which were transferred to NDAD: see the Dataset Documentation Catalogue, references CRDA/1/DD/1/1-CRDA/1/DD/1/7. Information about the changes which were made to the ME database and to PIB's inputting system in 1994 (see Aim and purpose and How data was originally captured and validated) is derived from conversations with staff in the Metropolitan Police's Performance Information Bureau in August 1998. Background information on the development of the CRIS system can be found in Her Majesty's Inspectorate of Constabulary, Information Technology Provision: Metropolitan Police Service (London: Home Office, 1996), pp. 53-56.

2. Service-Level Agreement between the Department of Computing Services and G10 Branch, 5 August 1991 (see the Dataset Documentation Catalogue, reference CRDA/1/DD/1/5).

3. Her Majesty's Inspectorate of Constabulary, Information Technology Provision: Metropolitan Police Service (London: Home Office, 1996), p. 53.

4. ICL, "The Year 2000 VME Facilities Guide", 4th edition: ICL web site (http://www.icl.com/year2000/y2k_flash.htm) consulted on 25 November 1998.

5. Kudos Partnership/ICL, "Open VME: Year 2000 Facilities Guide" (Wilmslow, Cheshire, 1997), ICL ref. no. BN510621: ICL web site (http://www.icl.com/year2000/vmefg.html) consulted on 29 July 1998.

6. ICL, "The Year 2000 VME Facilities Guide", 4th edition: ICL web site (http://www.icl.com/year2000/y2k_flash.htm) consulted on 25 November 1998.

7. ICL, "The Year 2000 VME Facilities Guide", 4th edition: ICL web site (http://www.icl.com/year2000/y2k_flash.htm) consulted on 25 November 1998.

8. Metropolitan Police Operational Reference Document 42/91, 27 November 1991, p.11 (see the Dataset Documentation Catalogue, reference CRDA/1/DD/2/3).

9. On the defects of recorded crime statistics, see Catriona Mirlees-Black, Tracey Budd, Sarah Partridge and Pat Mayhew, The 1998 British Crime Survey: England and Wales, Home Office Statistical Bulletin 21/98 (London: Home Office, 1998), p. 1-2: available via the Dataset Documentation Catalogue for the British Crime Survey, reference CRDA/2/DD/1/1998/5.