UK NEQAS for Antibiotic Assays

MANUAL FOR PARTICIPANTS

2011 v9

Updated Sept 1998, March 1999, August 1999

Added information March 2000, v2 August 2000, v3 February 2004, v4 March 2006, v5 September 2007, v6 January 2008, v7 July 2009, v8 November 2009, v9 July 2011

Note: this on-line version supersedes all printed versions

UK NEQAS For Antibiotic Assays

MANUAL FOR PARTICIPANTS 2010

Service Provided

The United Kingdom National External Quality Assessment Scheme (UK NEQAS) for Antibiotic Assays provides independent external quality assessment (EQA) of the measurement of antibiotic concentrations in human serum and plasma for the purposes of routine therapeutic drug monitoring and dosage individualisation. The antibiotics circulated currently are:

· amikacin (an aminoglycoside)

· flucytosine (an antifungal agent)

· gentamicin (an aminoglycoside) *

· teicoplanin (a glycopeptide)

· tobramycin (an aminoglycoside)

· vancomycin (a glycopeptide) *

* combined sample.

Participation can take place following the payment of an annual subscription - beginning in April each year, but laboratories are welcome to discuss joining at any time. The Scheme is open to UK and non-UK clinical and non-clinical laboratories, public and private healthcare laboratories. Reagent manufacturers and distributors are also welcome to join.

History

UK NEQAS for Antibiotic Assays began in 1972 with the distribution, by Professor David Reeves, of some gentamicin-containing serum samples to nineteen interested clinical microbiologists. The returns indicated that there was an urgent need to improve the quality (accuracy) of gentamicin assays. In 1974 the distributions became regular and some 92 laboratories were participating. In 1977, the currently used method of performance analysis, based on mean bias (and the standard deviation about that mean) over a batch of six samples, was introduced. At that time there were 236 participating laboratories. In 1980 tobramycin samples were added and netilmicin and chloramphenicol were added in 1982. Subsequent new analytes were: vancomycin (1984); flucytosine (1989); amikacin (1991) and teicoplanin (2005).

Chloramphenicol was removed in 2004 from all distributions. Netilmicin was removed in April 2009. The Scheme was recognised as a UK NEQAS in 1985. New software (EQA Core System) written and maintained by the Wolfson Computer Laboratory, Birmingham, UK, was commissioned in 1990. Professor David Reeves was Scheme Organiser until his retirement in March 1998 and was succeeded by Dr Les White. Following Dr White’s sad and sudden death in 2002 Professor Alasdair MacGowan has undertaken the role of the scheme Organiser.

To bring the Antibiotic Assay Scheme in line with other UK NEQAS Schemes, the new web-based Wolfson software was implemented in April 2007. This software allows the online entry of results and downloading of reports.

Senior Staff

Organiser/Director

Professor Alasdair MacGowan

Consultant Microbiologist

Chair of the Management Group

Dr Andrew Lovering

Consultant Clinical Scientist

Scheme Managers

Alan Noel Dr Mervyn Darville

Pre-Registration Clinical Scientist Clinical Scientist

Quality Control Manager

Ms Nicola Childs

BMS3

Participants

In May 2011 the number and location of participants per antibiotic were as follows:

SCHEME	ANALYTE	COUNTRY	COUNT
SMH	AMIK	EIRE	8
SMH	AMIK	HK	2
SMH	AMIK	NI	1
SMH	AMIK	S	3
SMH	AMIK	SZ	1
SMH	AMIK	U.K.	33
SMH	FLU	F	1
SMH	FLU	S	1
SMH	FLU	U.K.	4
SMH	GENT	CI	2
SMH	GENT	D	1
SMH	GENT	EIRE	23
SMH	GENT	HK	2
SMH	GENT	N	1
SMH	GENT	NI	4
SMH	GENT	NO	1
SMH	GENT	S	2
SMH	GENT	SZ	1
SMH	GENT	U.K	137
SMH	TEIC	EIRE	3
SMH	TEIC	NI	3
SMH	TEIC	S	1
SMH	TEIC	U.K.	5
SMH	TOB	D	1
SMH	TOB	EIRE	6
SMH	TOB	N	1
SMH	TOB	NO	1
SMH	TOB	S	3
SMH	TOB	SZ	1
SMH	TOB	U.K.	55
SMH	VANC	CI	2
SMH	VANC	D	1
SMH	VANC	EIRE	19
SMH	VANC	F	1
SMH	VANC	HK	2
SMH	VANC	N	1
SMH	VANC	NI	2
SMH	VANC	NO	1
SMH	VANC	S	3
SMH	VANC	SZ	1
SMH	VANC	U.K.	121

Participants Unique Code (Laboratory Number)

All participants are identified by a unique code number (Lab. Number) issued by Birmingham Quality (UKNEQAS, Birmingham), which maintains the confidential database of participant details. This number is constant across all UK NEQASs and is only issued once.

· Participants should quote their Lab. Number in all communications.

· Reports to participants will bear their unique Lab. Number and no other means of identification.

· It is very important that participants are familiar with their Lab. Number.

Recently, UK NEQAS has rationalised its codes across disciplines and Schemes. This now allows laboratories to participate in multiple Schemes across disciplines and to consolidate their participation under a single Laboratory Code Number.

Points of Contact

The first point of contact should be the Scheme Managers or the Scheme Organiser. The Lab. Number should always be quoted.

Postal Address

UK NEQAS for Antibiotic Assays

BCARE

Department of Microbiology

Southmead Hospital

Westbury-on-Trym

Bristol BS10 5NB

Extended Address for Courier Delivery/Pick-up

UK NEQAS for Antibiotic Assays

BCARE

Department of Microbiology

Lime Walk Building

Southmead Way

Southmead Hospital

Westbury-on-Trym

Bristol BS10 5NB

UK: 0117 323 6214 (answerphone)

Non-UK: +44 117 323 6214 (answerphone)

Fax (May be used to make monthly returns if web-reporting failure)

UK: 0117 323 8332

Non-UK: +44 117 323 8332

Email (May be used to make monthly returns if web-reporting failure)

World-wide: ukneqas.antibiotics@nbt.nhs.uk

Internet WWW Home Page

http://www.ukneqasaa.win-uk.net/

Weighed-in concentrations are published here 3 days after each distribution closes. These results may be taken as an approximate guide to the target concentrations used for scoring.

The target concentrations and statistical data are added as soon as available.

Other Microbiology NEQAS Schemes

The UK NEQAS for General Microbiology

Quality Assurance Laboratory

HPA

61 Colindale Avenue

London NW9 5HT

Tel: 020-89059890

Fax: 020-82051488

Email: organiser@ukneqasmic.win-uk.net

The UK NEQAS for Parasitology

Department of Parasitology

Hospital for Tropical Diseases

Mortimer Market

Capper Street

London

WC1E 6AU

Tel: 020-73830482

Fax: 020-73888985

Head Office for UK NEQAS

The UK NEQAS Office

PO Box 401

Sheffield

S5 7YZ

Tel: 0114-261 1689

Fax: 0114 261 1049

The UK NEQAS Home Page http://www.ukneqas.org.uk

Infection Control

All samples are presumed sterile and no live cultures are present in the production area during distribution morning. In line with CPA requirements, the Scheme Organiser is responsible for infection control within the Antibiotic Assay scheme.

Steering Committee

Click HERE to access steering committee information including names, addresses and contact numbers.

Samples and Distributions

Distributions

There are currently 12 distributions per year at approximately monthly intervals. Participants receive one sample for each analyte to which they have subscribed in each distribution. The distribution numbers are unique identifiers. All distributions are transported by post. Overseas samples are sent by airmail; UK and Isle of Man samples are sent by first class letter mail.

Samples

Each sample, apart from gentamicin and vancomycin (distributed in combination), comprises a single antibiotic. All samples are prepared in pooled human serum or DLDP (0.5-1.0 ml volume) supplied by an accredited commercial source. This screened matrix is negative for HIV antibody, Hepatitis C antibody and Hepatitis B Surface Antigen. A large volume of matrix is spiked with a pre-determined concentration of the antibiotic, thoroughly mixed, filtered and dispensed.

Sample production and postage takes place on a single day – spiked serum is not frozen after the antibiotic has been added. Since samples are prepared, dispensed and distributed in a single operation, integrity is guaranteed. Therefore, it is impossible to be inadvertently sent a sample from a previous distribution. Repeat samples are, however, available on request to all participants. Furthermore, laboratories whose return results more than 30% from the trimmed mean will be sent repeat samples to test in parallel with the original.

Samples should be treated as if they were clinical samples by the participating laboratory. They should be assayed upon receipt, or if necessary, stored in the dark at 2-8^OC, or frozen and thawed once only, before assay.

Extra samples are prepared each month and stored at -20^OC. Any repeat samples sent out are taken from this frozen stock. These frozen samples are stored for one year and then discarded.

Completing and Returning the Result Form

A form for returning results is sent out with the samples. The form shows the Lab. Number, the analytes taken and the method and sub-method of assay for each analyte (in coded form) that is on file for the participant.

In addition, each sample is matched to a clinical history. Although, the histories are fictitious, they are intended for use as a guide to the likely concentration range of the result. For example, a post-dose sample from a patient on once-daily gentamicin is likely to be >10 mg/L. Laboratories should dilute such a sample before assay to bring it into the assay concentration range. From April 2009 penalties have been incurred for returning “>” values. As with clinical samples, negative results should be reported as less than the sensitivity of your assay, for example <0.5 mg/L.

Upon receipt the participating laboratory should check the following:

· That the correct form (check Lab. Number in top right) has been received

· Samples containing the correct analytes have been received

· The samples are intact and not damaged or leaking

· That you are clear as to the closing date. This is two weeks after the samples are posted, but remember that delivery to some parts of the world may take several days.

**The Scheme Managers should be contacted if there is any discrepancy or problem**

The concentration of the appropriate antibiotic in each specimen should then be assayed using the routine method. When the assays have been performed the results in milligrams per litre (mg/L) should be entered onto the form against the correct analyte. Remember to multiply by the dilution factor if the sample was diluted.

NOTE: Returns are currently only accepted in the mass unit milligrams per litre (mg/L). Do not return results in molar units, for example micromoles per litre (µmol/L), or per cent units, for example milligrams per 100 millilitre (mg/100 ml).

Change of method or sub-method

If the method and sub-method details on the form are correct no more information is needed. If the method or sub-method has changed please enter a comment with the web-reporting result entry. Alternatively, the method changes can be notified by phone, fax or e-mail before the return of results.

At the time of writing the main current method codes are:

Manufacturer	Method	Method code	Sub-method	Sub-method code
Abbott	TDx / FLx	TDX	PFIA	PFIA
Abbott	Abbott	ABBOTT	Architect iArchetect	ARCH iArch
Abbott	Abbott	ABBOTT	Axsym	AXM
Advia	Advia	ADVIA
Beckman	Beckman	BECKMAN
Behring	Emit	EMIT	Wet manual Auto analyser	WM AA
Behring	Petina	PETINA
Bioassay	Yeast	YEAST
Biostat	Biostat	BIOSTAT	Biostat PFIA Seradyn	BIO PFIA SER
Boehringer Mannheim	Cedia	CEDIA
CDx90	CDx90	CDx90
Cobas	Cobas	COBAS
Dade	Dade	DADE
	HPLC	HPLC
LabFx	LabFx	LABFX	PFIA	PFIA
Launch	Biokit	BKT
Olympus	AU400/600/640 2700/5400	OLYMPUS
Ortho	Ortho	ORTHO
Roche	Roche	ROCHE	PFIA	PFIA
Roche	RocheK	ROCHEK
Roche	Roche Modular	ROCHEMP
Sapphire	Sapphire	SAPPHIRE
Siemens	Siemens	SIEMENS	Centaur	CENT
Siemens	Siemens	SIEMENS	Advia	ADVIA
Siemens	Siemens	SIEMENS	Dade	DADE
Siemens	Siemens	SIEMENS	Technicon	TECH
Unknown	No method
Vitros	Vitros FS	VITROS FS
Vitros	Vitros S	VITROS S

Sending results

From April 2011, the scheme moved to an all web-reporting basis for data entry and report download. However, if any participant experiences difficulties in doing this then the completed results form can be faxed, emailed or posted to us.

If you need to fax your form please keep a note of the date and time in case of problems. Result forms are sometimes accidentally faxed back-to-front and we receive only a blank sheet of paper! Also please beware of black vertical lines on faxes resulting from dirty or scored fax equipment. These have been known to obscure important data such as decimal points!

We encourage participants to return their results by web-entry because the electronic record of date and time sent is easily audited. Results do sometimes get lost in the post or in hospital’s internal mail systems.

Statistical Analyses

Antibiotic-Specific results

The following statistical parameters are determined for all antibiotics in each distribution and presented to the participants on their monthly report.

All Laboratory Trimmed Mean (ALTM) This is calculated from the all laboratory mean after trimming any mistakes (outliers) lying >two standard deviations from the mean.

The standard deviation (S.D.), coefficient of variation (CV% = S.D./ALTM*100), and number (n) of returns contributing to the ALTM are calculated also.

Method Laboratory Trimmed Mean (MLTM) This is calculated from the method specific mean after trimming any mistakes (outliers) lying >two standard deviations from the mean.

The standard deviation (S.D.), coefficient of variation (CV% = S.D./MLTM*100), and number (n) of returns contributing to the MLTM are calculated.

In addition, the Sub-method Laboratory Trimmed Mean (SMLTM) is calculated for each immunoassay sub-method but only the parameters for the sub-method used by the participant are published on their monthly report form. All sub-method statistics are however published in the Annual Report. As before, the SMLTM is calculated from the sub-method specific laboratory mean, after trimming outliers lying >two standard deviations from the mean. The standard deviation (S.D.), coefficient of variation (CV% = S.D./SMLTM*100), and number of returns contributing to the SMLTM are calculated also.

Laboratory-Specific Parameters

The following statistical parameters are calculated for each participating laboratory, for each quality control antibiotic in each distribution. They can be found on the monthly report along with the antibiotic-specific parameters described above.

Percentage Error (% ERROR or BIAS) This is calculated for each antibiotic and is the difference between the laboratory result and the target concentration. This is calculated as follows: % ERROR = (Result-Target)/Target*100. The target value is usually a clinically relevant concentration range.

The Mean % Error +2SD (MEAN +2SD) This is calculated for each antibiotic and is the (modulus) mean of the percentage error for the last six distributions plus two S.D.’s about that mean. It is not calculated if the participant has submitted less than three results in the last six months.

Cumulative Bias (CBIAS) This is the cumulative trimmed mean %ERROR or trimmed mean BIAS over the last twelve distributions. Outliers >two S.D. from the mean BIAS are trimmed before CBIAS is calculated.

These parameters are used to calculate the score and so the performance of the laboratory (see below).

The Laboratory Performance Scoring System

Laboratory performance is assessed statistically. How a laboratory performs is based its ability to be close to the target concentrations for six successive distributions. The extent of inconsistency (imprecision) and inaccuracy (bias) before a performance can be considered poor takes into account the accepted variability for results to be useful clinically. The published limits of imprecision and bias of the analytical technique used will normally be considerably better than the minimum requirement for clinical suitability.

It is generally agreed that for the results of an aminoglycoside assay to be clinically useful the assay result should be within +/- 25-30% of the "true" concentration (Reeves, D.S. & Wise, R. (1978) In: Laboratory Methods in Antimicrobial Chemotherapy Eds. D.S. Reeves, I. Phillips, J.D. Williams & R. Wise. Churchill Livingstone, Edinburgh pp. 137-143). The same limits are applied to all the antibiotics in the Scheme. The scoring system aims to reflect this view.

Each quality control sample has a target concentration - the ALTM. Please note that the ALTM is calculated to >4 decimal places but is rounded to two decimal places when printed on the report form.

Before October 1998 the weighed-in concentration was used as the target concentration. However, the ALTM and the weighed in concentration have shown agreement and the target was changed to the ALTM for all analytes, except 5-flucytosine as there are currently <13 returns for this quality control sample.

Occasionally the weighed-in concentration is deliberately chosen to be below the normal therapeutic range and/or less than the lowest (non-zero) calibrator of commercial immunoassay kits in common usage. Since April 2009 concentrations of 0.0 mg/L have also been included. For these distributions the returns are not scored and the ALTM is supplied for information purposes only. Such samples are distributed to determine for educational purposes, reproducibility in the field, of how methods deal with very low or zero concentrations. From April 2010 participants returning positive results above a determined threshold for zero-spiked samples have been penalised. This change to the scoring system was approved by the Microbiology Steering Committee.

The Procedure for the Calculation of Laboratory Performance Score

Firstly, the number of returns made by each laboratory for each antibiotic over the six-monthly period is checked. If three to six returns have been made the results for the laboratory are included and a performance Score calculation is made for this antibiotic. If a laboratory has made fewer than three returns for an antibiotic, the results are excluded from the analysis.

For each included laboratory and for each subscribed antibiotic, the individual result is compared with the target concentration and the percentage error (percentage bias) determined (% ERROR = (Result-Target)/Target*100). Note: Because the printed target is rounded up to two decimal places (but not rounded up for the purpose of calculating % error) a laboratory may notice a small percentage error even when their return and the printed target are identical. For example a Target of 5.473 mg/L will be rounded to 5.5 mg/L when printed.

For each laboratory and antibiotic the mean percentage error (m) over the last six distributions (that is, over six samples assayed over a six-monthly period) and the standard deviation (s) about that mean are calculated.

For each laboratory and antibiotic, the modulus of m is determined. Simply, this means that if the mean error is negative it is made positive. The MEAN+2SD is determined from the formula:

MEAN+2SD = Modulus (m) + 2s

Based on this result, each laboratory is then assigned to a performance group (1-11) and a performance Score (of -1, 0, 1 or 2) for each subscribed antibiotic according to the look-up table below.

MEAN+2SD	GROUP	SCORE	INTERPRETATION
0-20	1	2	OK	
>20-30	2	2	OK	
>30-40	3	1	borderline, possible problem	
>40-50	4	1	borderline, probable problem	
>50-60	5	0	poor, problem or blunder	
>60-70	6	0	poor, problem or blunder	
>70-80	7	0	poor, problem or blunder	
>80-90	8	0	poor, problem or blunder	
>90-100	9	0	poor, problem or blunder	
>100-200	10	-1	very poor, problem or blunder	
>200	11	-1	very poor, problem or blunder	

In general terms, a MEAN+2SD value of < 30 indicates a very good performance. A value of >50 is considered poor and >100, very poor.

Laboratories should remember the following when reviewing their monthly score:

· Remember performance is not scored on the basis of a single result but on a statistical analysis of the last six monthly returns.

· Poor accuracy (assay bias) and poor reproducibility (assay imprecision) may both contribute to a borderline or a poor Score.

· A blunder^* is not ignored when Scores are calculated. Therefore a single blunder can result in a laboratory having a poor performance Score. This poor Score will remain for up to six months and should act as a reminder to the staff to avoid further blunders!!!

^*Definition: a gross mistake, derived from the Danish word blunde, to slumber.

The Monthly Report: Getting the Most from the Scheme

Your monthly report form looks something like the example below. It contains lots of information that may be of use to you. It is available on line at the same time as the result form for the following distribution.

Laboratory Performance

The method is ignored in assessing LABORATORY PERFORMANCE, which is based on your last six returns. As explained above your LABORATORY PERFORMANCE is calculated each month on a rolling basis using your previous six returns. You will need a minimum of three monthly returns before your performance is analysed.

Method Performance

In contrast, METHOD PERFORMANCE is assessed each month and can also be found on the report form. Mean results reported are trimmed as described above under Statistical Analyses. Every month you will receive details of method performance for only those antibiotics to which you subscribe.

Each monthly report is individually tailored and headed by the Distribution No., your unique Lab. No. and the Date the samples were distributed. Then for each antibiotic there are four blocks of information:

TOP RIGHT - Your Laboratory Performance

Target value This is the target concentration (in mg/L) – the ALTM for all antibiotics.
Your result The result (in mg/L) which you returned to us
Your % error The % value by which your result differed from the target value
MEAN+2SD The modulus of your mean % error over the last six distributions + two standard deviations about that mean
GROUP Your performance grouping based on your MEAN+2SD
SCORE Your performance Score based on your group

Scores and Groups are determined as described under Laboratory Performance Scoring System above.

A SCORE of 2 is acceptable performance over the six-monthly period

A SCORE of 0 or -1 indicates poor or very poor performance, but remember this may be due to a single blunder made in the last 6 months.

TOP RIGHT - Your Method and Sub-method Performance (found under your laboratory performance).

Details of your method, (e.g. TDX) and sub-method (e.g. NK = Abbott reagents)

Method Mean The trimmed mean result of laboratories using your method.
Sub-Method Mean The trimmed mean result of laboratories using your sub-method.
Sub-Method S.D. The standard deviation about this trimmed mean.
Sub-Method CV% The % coefficient of variation.
Sub-Method No. The number of laboratories, using this sub-method.

PLEASE NOTE That sub-method information is only available to users of that particular sub-method.

TOP LEFT - Overall Performance Analysis - for this distribution

Here overall performance and individual method performance data is presented in tabular form. The first column gives the total number of returns for each method. The next column shows the trimmed mean value for all the methods used, method SD and CV (%).

Using this table, individual methods can be compared as regards their bias (compare the mean with the target), imprecision (compare the CV%’s of the different methods) and popularity (compare the number of returns). Due to the rapid increase in the number of new methods being used, only methods with >13 participants are shown on every report form. If your method has <13 participants it will only show on the reports of those participants using the method.

TOP MIDDLE - The Distribution Histogram

The histogram shows the distribution of results for all laboratories and for laboratories using your method. Method specific results appear as grey columns, all method results appear as white column. Your method return is shown by an arrow.

PLEASE NOTE The individual method histogram is only made available to users of that particular method.

MIDDLE - Summary of the Previous Five Returns

Here are summarised laboratory performance data for the previous five returns. These data examined together should help a laboratory in determining any possible reasons for being given a sub-optimal (borderline) or poor performance score. Below these data is a box. It is here that any analyte or distribution specific comments will be made.

Confidentiality and Divulgence of Participation and Performance Records

All Participants

· Confidentiality is an important aspect of UK NEQAS participation.

· Details of a participant’s performance will normally never be revealed to another individual or organisation.

· Details of participation and performance will only be divulged to any other third parties with the express, written permission of the Head of the participating laboratory.

Notwithstanding the above statements the fact that a laboratory participates in a UK NEQAS is not regarded as confidential. The identity of participants (Head of Department and name of laboratory) and the analytes for which they are registered may be revealed on request to a Scheme Organiser or, the Authority, Trust, hospital or private company, within which the laboratory is situated.

UK and Irish Clinical Laboratories only

The exception is the case of UK or Irish Clinical Laboratories (public or private) who become considered as persistent poor performers (see Laboratory Performance Scoring System above). The Head of the participating laboratory will normally be required to agree in writing to abide by the current Joint Working Group on Quality Assurance (JWGQA) procedures as a condition of participation (see Special Conditions below).

Sample Reliability, Homogeneity and Integrity

As the specimens sent are meant to be for EQA purposes, they need to be of consistent (homogeneous) composition. The antibiotic assay samples are prepared by spiking a pooled volume of human serum or plasma with a stock antibiotic solution of known concentration and then making up to a predetermined volume in calibrated glassware. After spiking, the solutions are thoroughly mixed according to a fixed protocol to ensure homogeneity. The solution is then dispensed into vials. Samples from first and last dispensing are assayed to confirm homogeneity has been achieved. The sample preparation standard operating procedures ensures sample integrity, since no samples from previous distributions are present in the sample preparation area.

Despite the serum and antibiotic solutions being initially sterile, it is impossible to guarantee that all the samples dispensed will remain sterile.

For this reason, it is recommended that the samples are assayed upon receipt and if not, stored at 4^OC or frozen. Sterility checks are performed randomly.

In-house sample stability studies have shown that samples are stable for 1 year when frozen once and stored at -20^OC.

Flucytosine is degraded by light. It is important therefore to keep these samples out of direct sunlight. It is recommended that all samples are stored in the dark.

It is very unlikely that a "rogue" specimen will be sent out which contains different antibiotic concentration from other samples in the same distribution. However, if a laboratory returns a result more than 30% from the ALTM they are automatically sent a repeat specimen. If, upon re-assay in the participants laboratory, the original specimen continues to give an incorrect result, and the repeat specimen gives a correct result, the original return of that laboratory will be deleted from our records and replaced by the result obtained with the repeat specimen.

Problems, Queries and Complaints

If you have a problem with the Scheme, or wish to make a complaint, please feel free to contact the Managers, either by letter, fax, email or telephone (please note that the number shown on the scheme’s paperwork is an answerphone). Every effort will be made to deal with your communication quickly and effectively.

In the case of UK and Irish laboratories, problems relating to the Scheme, including complaints from participating laboratories, which cannot be resolved by the Organiser, Steering Committee or NQAAP will be referred to the Chairman of the Joint Working Group on Quality Assurance (JWGQA).

Conditions of Participation

The only general conditions of participation are that laboratories agree to:

· Handle samples as though they were clinical samples, giving due regard to the appropriate safety procedures in their institution.

· All reports, and the data they contain, issued by UK NEQAS Organisers being copyright. They may not be published in any form without the permission of the appropriate Steering Committee.

· Take all actions necessary to pay their annual subscription promptly upon receipt of an invoice.

· Give written notice to the Scheme when they wish permanently to discontinue participation for any reason (for example, laboratory closure).

Special Conditions of Participation by UK Clinical Laboratories in UK External Quality Assessment Schemes (EQASs).

External Quality Assessment (EQA) is designed to provide objective evidence of the quality of individual investigations and analyses and is essential for clinical laboratories. EQASs have developed over the past 40 years and are now available from a range of providers covering most clinical laboratory services. EQASs have been accredited as meeting certain standards by the JWG on QA and this responsibility for EQA scheme accreditation has now been passed to CPA (UK) Ltd. CPA (UK) Ltd. has, after consultation, drawn up standards for EQASs and schemes will be required to comply with these standards to maintain or obtain scheme accreditation.

From the start, British EQASs have had the aim of maintaining high clinical laboratory standards by providing help, support, and education to participants, in a confidential setting. The professions within pathology have taken responsibility for maintaining high standards and, in doing so, have avoided the need for a licensing system. The confidentiality of EQAS performance has always been important. However, an adjustment to the level of confidentiality was required in 1990. The participants were notified of this change after it had been approved by the professional bodies.

The JWG on QA, in consultation with its parent professional bodies, now considers that a further adjustment in confidentiality is required to address the possible situation where, despite full professional support and educational assistance, a participant has an unresolved, persistent unsatisfactory performance.

Each accredited EQA Scheme has a Steering Committee that advises the Organiser on the overall operation of the Scheme. Executive responsibility for maintaining satisfactory professional standards of investigations or analytical work in pathology laboratories in the UK is vested in the National Quality Assurance Advisory Panels (NQAAPs). The NQAAPs report to the JWG on QA which has ultimate responsibility on behalf of its parent professional bodies, for professionally approving, monitoring and supporting EQASs in the UK.

Recommendations of the Joint Working Group for Quality Assurance: Conditions of EQA Scheme Participation

The Joint Working Group for Quality Assurance (JWG) is a multidisciplinary group accountable to the Royal College of Pathologists for the oversight of performance in external quality assurance schemes (EQA) in the UK. Membership consists of the Chairmen of the National Quality Assurance Advisory Panels (NQAAPs), and representatives from the Institute of Biomedical Sciences, the Independent Healthcare Sector, the Department of Health and CPA (UK) Ltd.

1. The Head of a laboratory is responsible for registering the laboratory with an appropriate accredited EQA scheme.

2. The laboratory should be registered with available EQA schemes to cover all the tests that the laboratory performs as a clinical service.

3. EQA samples must be treated in exactly the same way as clinical samples. If this is not possible because of the use of non-routine material for the EQA (such as photographs) they should still be given as near to routine treatment as possible.

4. Changes in the test methodology of the laboratory should be notified in writing to the appropriate scheme organiser and should be reflected in the EQA schemes with which the laboratory is registered.

5. Samples, reports and routine correspondence may be addressed to a named deputy, but correspondence from Organisers and NQAAPs concerning persistent poor performance (red – see below) will be sent directly to the Head of the laboratory or, in the case of the independent healthcare sector, the Hospital Executive Director.

6. The EQA code number and name of the laboratory and the assessment of individual laboratory performance are confidential to the participant and will not be released by Scheme Organisers without the written permission of the Head of the laboratory to any third party other than the Chairman and members of the appropriate NQAAP and the Chairman and members of the JWG. The identity of a participant (name of laboratory and Head of Department) and the tests and EQA schemes for which that laboratory is registered (but not details of performance) may also be released by the Scheme Organiser on request to the Health Authority, Hospital Trust/Private Company in which the laboratory is situated after a written request has been received.

7. A NQAAP may, with the written permission of the Head of a laboratory, correspond with the Authority responsible for the laboratory, about deficiencies in staff or equipment which, in the opinion of the NQAAP members, prevent the laboratory from maintaining a satisfactory standard.

8. Laboratories’ EQA performance will be graded using a traffic light system; green will indicate no concerns, amber poor performance, red persistent poor performance, with black being reserved for the tiny number of cases that cannot be managed by the Organiser or NQAAP and that have to be referred to the JWG. The criteria for poor performance (amber) and persistent poor performance (red) are proposed by the EQA scheme Steering Committee in consultation with the EQA Provider/Scheme Organiser and approved by the relevant NQAAP.

9. When a laboratory shows poor (amber) performance the Organiser will generally make contact with the participant in accordance with the Scheme Standard Operating Procedure for poor performance. Within 2 weeks of a laboratory being identified as a persistent poor performer (red), the Organiser will notify the Chairman of the appropriate NQAAP together with a resume of remedial action taken or proposed. The identity of a persistently poor performing laboratory (red) will be made available to members of the NQAAP and JWG. The NQAAP Chairman should agree in writing any remedial action to be taken and the timescale and responsibility for carrying this out; if appropriate, this letter will be copied to accreditation/regulatory bodies such as CPA (UK) Ltd, UKAS and HFEA who may arrange an urgent visit to the laboratory. Advice is offered to the Head of the Laboratory in writing or, if appropriate, a visit to the Laboratory from a NQAAP member or appropriate agreed expert may be arranged.

10. If persistent poor performance remains unresolved (black), the NQAAP Chairman will submit a report to the Chairman of the JWG giving details of the problem, its causes and the reasons for failure to achieve improvement. The Chairman of the JWG will consider the report and, if appropriate, seek specialist advice from a panel of experts from the appropriate professional bodies to advise him/her on this matter. The Chairman of the JWG will be empowered to arrange a site meeting of this panel of experts with the Head of the Department concerned. If such supportive action fails to resolve the problems and, with the agreement of the panel of experts, the Chairman of the JWG will inform the Chief Executive Officer, or nearest equivalent within the organisation of the Trust or Institution, of the problem, the steps which have been taken to rectify it and, if it has been identified, the cause of the problem. The Chairman of the JWG also has direct access and responsibility to the Professional Standards Unit of the Royal College of Pathologists. Should these measures fail to resolve the issues, the laboratory will be referred to the Care Quality Commission for further action.

11. Problems relating to EQA Schemes, including complaints from participating laboratories, which cannot be resolved by the appropriate Organiser, Steering Committee or NQAAP, will be referred to the Chairman of the JWG.

Joint Working Group for Quality Assurance in Pathology, August 2010.

Dr Alec J Howat (Chair)

JWG on Quality Assurance

May 2011

Note: Professions and organisations represented by the JWG on QA include: The Royal College of Pathologists; The Association of Clinical Biochemists; The Institute of Biomedical Science; The Association of Clinical Pathologists; The Association of Clinical Cytogeneticists; The Pathological Society of Great Britain and Ireland; The Association of Medical Microbiologists; The British Society for Haematology; The British Society for Immunology; The British Society for Clinical Cytology; The Clinical Molecular Genetics Society; The British Blood Transfusion Society; Clinical Pathology Accreditation (UK) Ltd.

Manual Appendix : Why is my assay performance Score not as good as I expect?

NEQAS antibiotic assay scoring is based on statistical parameters comparing your results with a target concentration.

For each individual sample, the difference between the target and your return is the %ERROR or BIAS. Over six distributions your %ERROR is determined for each return and from these your mean %ERROR (or mean BIAS) is determined.

The sample standard deviation (SD) about this mean %BIAS gives a measure of the irreproducibility of your returns over the 6-month period.

Remember: Your performance Score is calculated from your returns for the last six distributions. The following applies.

1. VERY LOW TARGET CONCENTRATIONS

Occasionally we send out samples with a very low target (e.g. gentamicin 0.2 mg/L) and, since April 2009, zero concentrations. The results for these samples give participants an idea of how well their method performs with the low concentrations that may be found in the pre-dose samples from once-daily aminoglycoside regimens.

2. NON-RETURNS

If a laboratory has only one or two returns in a 6-monthly period no Score will be calculated. If a laboratory has three to six scored returns a Score will be determined from these returns.

3. BLUNDERS (single gross errors)

Blunders are not ignored when calculating a laboratory’s Score. A single gross error (blunder) in a six-monthly period is likely to make your laboratory Score 0 or -1 because of a single large percentage error. Blunders should be simple to spot on the monthly report. Performance will hopefully return to normal when the blunder moves out of the 6-month window.

4. PENALTIES

The scores of laboratories returning one “greater than” value or failing once to return a value (NULL returns) will drop to zero, although they will keep their mean+2SD. Laboratories returning “greater than” values and/or failing to return more than once during any 6 month period will be sent a “poor-performer” letter.

From April 2010, participants will also be penalised for returning positive values above a certain threshold for zero-spiked samples.

5. BIAS (systematic error)

If a method has a consistent bias this might adversely effect laboratory performance, but only if the bias is very large (greater than ±20-25%). Such a degree of bias is very unlikely and no commercial immunoassay kits have such a high bias. In the case of gentamicin, which is not a pure drug but a mixture of closely related aminoglycosides, each immunoassay kit will show a slight bias depending on the exact composition of the gentamicin in the sample and the degree of cross-reactivity the kit antibody shows with each of the gentamicin components.

6. VARIABILITY (random error or irreproducibility)

Irreproducibility is the most likely cause of borderline performance and the statistical tests (mean, SD) used to determine performance assume the irreproducibility is random error. However not all irreproducibility is due to random error; trends in performance will also be interpreted by the statistical tests as irreproducibility, even if the trend is towards improved performance.

Below are six simulated % errors for three fictitious NEQAS participants, A, B and C. The % errors for all three laboratories are the same.

For example:

	Lab A	Lab B	Lab C
Month	Percentage error obtained with monthly NEQAS sample
January	22	-8	-8
February	15	-8	12
March	12	-6	15
April	-8	12	-6
May	-8	15	-8
June	-6	22	22
Mean BIAS	4.5	4.5	4.5
SD (VAR)	13.38	13.38	13.38
MEAN+2SD	31.27	31.27	31.27
SCORE	1	1	1
Trend	Getting better	Getting worse	Irreproducible

Laboratory A is getting progressively better (nearer the target, smaller % error)
Laboratory B is getting progressively worse (further from target, larger % error)
Laboratory C is showing true irreproducibility (error fluctuates from month to month)

If a laboratory Score is borderline because of irreproducibility due to a trend towards improved performance then a glance at the % errors for the last six distributions will confirm this (these data are printed on each monthly report). Such a trend could be due to a change in method or sub-method or to a reformulation or realignment of the calibration of a method or sub-method. Once consistency has been re-established performance should (blunders apart!) return to an acceptable level.

The Scheme Organiser is always happy to provide written supporting information if a laboratory feels their Score has temporarily worsened because of a trend towards improved performance.

 END