Dataset details: CRDA/61/DS/1

The Retail Prices Index (RPI) measures the average change from month to month in the prices of consumer goods and services purchased in the UK. The RPI is the most familiar general-purpose measure of inflation in the UK, and it is used for indexing pensions, state benefits and index-linked gilts. Until December 2003, the RPI was the principal UK domestic measure of inflation for macro-economic purposes. 1 It has since been replaced in that role by the Consumer Prices Index (CPI). (See links to related datasets).

The dataset was downloaded by NDAD from the Office for National Statistics website on 31 March 2004. The data was captured in text format and in Navidata format. The CPI and RPI data were downloaded as a single batch, held in 20 numbered indices. For further information about subsequent treatment of these indices by NDAD, see Digital processing and conversion.

The dataset as accessioned by NDAD comprises a snapshot of the Retail Prices Index for the years 1987-2003. The original data as transferred to NDAD (see Source of acquisition) contained a few tables with a very small amount of additional data, some of which pre-dates 1987 and some of which post-dates 2003. This has not been included as part of this accession. 2

The RPI is an index to virtually all types of household spending. It indicates what households would need to spend in order to keep purchasing the same things that it bought in an earlier period. Exceptions that fall outside the scope of the index include savings and investments, charges for credit, betting, and cash gifts. The index is compiled using prices of a large and varied sample of products, in selected locations. The goods and services for which prices are recorded are called 'representative items'.

The RPI is compiled using over 650 separate representative items. Their price fluctuations are assumed to represent the price changes for all goods and services covered by the index, including those for which prices are not specifically monitored.

The RPI covers the following categories of goods and services: Food, Catering, Alcoholic drink, Tobacco, Housing, Fuel and Light, Household goods, Household services, Clothing and footwear, Personal goods and services, Motoring expenditure, Fares and other travel costs, Leisure goods, and Leisure services. It does not cover Unit trusts and stockbroking charges, foreign students' university tuition fees and university accommodation fees, which are covered in the CPI.

The RPI measures changes in prices, not price levels; so it is expressed in terms of the comparison of prices relative to January 1987, when the index is given a value of 100. For example, the index for January 2004 was 183.1, which means that it would cost £183.10 to buy the same goods and services that cost £100 in January 1987. This represents an 83% rise in prices.

The reference date of January 1987 is simply a convenient benchmark for comparison, and has no other relevance. The choice of data has no effect on the measurement of price changes between one month and another, and has been changed several times since the index began in 1947.

The dataset is a time series. It covers the RPI-related indices published monthly in Focus On Consumer Price Indices – formerly known as Consumer Price Indices Business Monitor MM23. This includes the all-items index, detailed figures for groups and sections, annual inflation and monthly percentage changes, RPIX (ie the RPI excluding mortgage interest payments) and RPIY, breakdown of goods and services, pensioner indices, weights used to produce the indices, and contributions to the changes in the all-items index. The dataset also includes the RPI all-items 'long run' series, which has annual figures back to 1800, and monthly figures back to 1947. However, most of the RPI indices in this dataset start in 1987, when the index was re-based (see reference date, above).

The RPI is reviewed annually, to ensure that the calculations properly reflect UK shopping patterns. A wide range of information is used to determine the representative items, including ONS surveys of household spending, external market research and feedback from the price collectors. As markets and fashions change, and new products are introduced, the RPI is adjusted accordingly; internet purchases and DVD players, for example, were added in recent years. Once reviewed, the RPI is held fixed for a year at a time.

The dataset as downloaded from the ONS website originally comprised a mixture of both RPI and CPI data, held in 20 numbered indices.

One table (Number 3.5) was included in the original transfer, but after completion of the Navidata extraction process by NDAD, the table appeared to be empty. In fact, the data from 3.5 is identical to 3.6. The identifier had been put in a formula and was not being detected by the Navidata extraction process. The department has advised NDAD that the system has now been changed and for future accruals the 2 tables will have the same data.

This means that the datasets as accessioned by NDAD comprise a total of 19 tables and not 20.

In addition, there were a very small amount of additional data, some of which pre-dates 1987 and some of which post-dates 2003, which were filtered out by NDAD in the processing stage and not included as part of this accession. The reason for this is set out in a note with Scope and content.

The Retail Prices Index snapshot 1987-2003 dataset and related documentation are open without restriction.

The dataset as downloaded from the ONS website originally comprised a mixture of both RPI and CPI data, held in 20 numbered indices. These were treated by NDAD as 20 separate tables. Each table has several fields (called 'series' within the Navidata software).

The original numbering scheme which ONS used for their indices has been retained by NDAD, in the titles given to each table. Numbers 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3 and 3.7 all comprise RPI data and form the Retail Prices Index snapshot 1987-2003 (CRDA/61/DS/1). Numbers 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 3.4 and 3.6 are CPI data and form the Consumer Prices Index snapshot 1988-2003 (CRDA/61/DS/2). There is no data for Number 3.5; see Digital processing and conversion.

The dataset comprises the following table(s):

The two basic price collection methods are local and central. Price collectors are staff from Research International acting on behalf of the ONS. Around the middle of each month they record about 110,000 prices for over 550 representative items. They collect these from a variety of shops in around 150 places throughout the UK. This is normally done by visiting the outlet, although some prices can be gathered by phone.

The reliability of the RPI depends on the goodwill of retailers. The information collected is treated with confidence; no data relating to individual retailers or businesses is ever passed on to third parties.

Central collection methods are used for items where all the prices can be gathered centrally with no field work. Currently some 110 indicators are obtained from central sources. Large chain stores that have national pricing policies for their branches assist the process by sending information directly to the ONS.

For further information on price collection, see Chapter four of the RPI Technical Manual 1998 (Dataset Documentation Catalogue, reference CRDA/61/DD/1/1), or Chapter four of the CPI Technical Manual 2005 (Dataset Documentation Catalogue, reference CRDA/61/DD/1/4); for validation methods, see Chapter five of either publication.

Weighting

Some components of the RPI are weighted to ensure that the index reflects the varying importance of the goods and services, the amounts spent in different regions of the UK, and in different shops. The weighting system for the RPI is derived mainly from the ONS Expenditure and Food Survey. In this survey, a sample of several thousand households keeps a record of their spending over a fortnight.

When calculating the weights, RPI excludes the top 4% by income and pensioner households who derive at least three quarters of their income from state benefits, on the grounds that their spending is significantly different from the majority of people. This is designed to make the RPI more representative of a typical household.

The weighting system changes each year to keep pace with the changes in the public’s spending habits.

For further information on weighting, see Chapter six of the RPI Technical Manual (Dataset Documentation Catalogue, reference CRDA/61/DD/1/1); or Chapter six of the CPI Technical Manual 2005 (Dataset Documentation Catalogue, reference CRDA/61/DD/1/4).

How the Index was calculated

When all the price data was checked and processed, the price indicators were combined. Changes in prices of individual goods and services were measured by comparing them to their levels in the previous January. The figures were weighted together using the weights for the current year to produce an overall average price change.

The average price changes were then linked with figures for earlier years. Through this linkage of calculations, the RPI takes account of the changes in the range of goods and services purchased; and provides like-with-like comparisons between different years.

This procedure ensures that the index will not present distorted information when representative items are removed from, or added to, the index.

For further information on how the RPI is constructed and calculated, see Chapter two of the RPI Technical Manual (Dataset Documentation Catalogue, reference CRDA/61/DD/1/1); or Chapter two of the CPI Technical Manual 2005 (Dataset Documentation Catalogue, reference CRDA/61/DD/1/4).

Table 2.3: RPI monthly percentage changes 1947 to 2004 has no data in the original file transferred to NDAD between the years 1947 until 1976. See Scope and content for details in pre 1987 data.

In table 2.6: RPI contributions to changes in all items index, some field descriptions are truncated in the original metadata file. Most of the fields in this table (CRDA/61/DS/1/6) are blank.

1. For further detail, users may want to refer to the publication The New Inflation Target: The Statistical perspective, London ONS, December 2003. See the Dataset Documentation Catalogue, reference CRDA/61/DD/1/2.

2. A meeting was held on 10 February 2004, between The National Archives and the Department, with NDAD present. The purpose of this meeting was to facilitate the transfer of the data from the department to NDAD, and a record of the meeting is held as part of NDAD's administrative records. At this meeting, an agreement between The National Archives and the Department was reached, stating clearly that the transfer was to consist of RPI data with 1987 as the 100 base, ie the reference point for comparison. NDAD's accessioning actions have therefore been very precise in representing only the date range as agreed; in addition, the inclusion of this earlier data might tend to give a misleading impression to the user, particularly as the dataset is a time series.