QUALITY ASSURANCE IN FISH DISEASE
DIAGNOSIS
Olga Haenen1*, Eva-Maria Bernoth2*, and David
Groman3*
1 ID-DLO Institute for Animal Science and Health, Fish
Diseases Laboratory, P.O. Box 65, 8200 AB Lelystad, The Netherlands;
e-mail: o.l.m.haenen@id.dlo.nl (6th September 199 onwards -
o.l.m.haenen@id.wag-ur.nl)
2Aquatic Animal Health Unit, National Office for Animal & Plant
Health, Department of Agriculture, Fisheries & Forestry, GPO Box 858,
Canberra, ACT 2601, Australia; e-mail: Eva-Maria.Bernoth@affa.gov.au
3 Aquatic Diagnostic Services, Atlantic Veterinary College,
University of Prince Edward Island, Charlottetown, PEI, Canada C1A 4P3;
e-mail: groman@Upei.CA
2.a QA: what?
2.b QA: where and for whom?
2.c QA: why?
2.d QA: how?
2.e What system to choose?
3.a ISO 9001
3.b EN 45001
3.c USDA guidelines: to be added
3.d Canadian recommendations on QA
5.a ISO 9001
5.b EN 45001
5.c USDA guidelines: to be added
6.a ISO 9001
6.b EN 45001
6.c USDA guidelines: to be added
6.d Canadian recommendations on QA
Return to Main Page
A workshop on Quality Assurance in Fish Disease Diagnosis was held during
the 9th International Conference of the European Association of Fish
Pathologists (EAFP) in Rhodes Greece, from 19th - 24th September 1999. In
this workshop, a working document on how to standardize your work, and one
step further, how to gain accreditation (ISO 9001, EN 45001, USDA) of
diagnostic testing was discussed, together with a plan for interlaboratory
quality assurance (QA), with respect to fish diagnostic laboratories.
It has resulted in a practical guide, about the steps required to reach
these goals. The guide was published in an issue of the EAFP Bulletin (Vol
19(6) 1999) and is made available here for reference.
Eva-Maria Bernoth acted as chairperson for the Workshop
Apart from the discussion, a number of formal presentations were made:
Return to Table of Contents
Return to Main Page
Human laboratories are legally required to follow a set of quality
assurance procedures. Although there are few regulatory requirements for
quality assurance in veterinary laboratories, the quality of laboratory
results is just as important and would be expected in any legal challenge
to decisions based on laboratory results. Every veterinary laboratory,
whether it is a full service regional facility or a practioner's office
laboratory, should have a quality assurance program.
A quality assurance program in a veterinary laboratory is important to the
laboratory, to the veterinarians who use the laboratory, and to the public.
A laboratory quality assurance program has one overall objective - to
provide confidence in the validity of the laboratory's test results and
services. Veterinarians who receive timely laboratory results and are
confident in their accuracy and reliability are more likely to make valid
medical decisions on their patients. The public in turn will have increased
satisfaction with the quality of veterinary health care.
The specific components of a quality assurance program will vary between
laboratories, depending on the extent and complexity of the tests
conducted, however, a quality assurance program for a veterinary laboratory
should usually address the following: quality goals, internal quality
assurance, external quality assurance, quality of operations and resources,
and documentation of quality.
Return to Table of Contents
Return to Main Page
"Each lab should define a set of measurable quality goals. The most
important goals to set initially are specific quality goals for the
accuracy and precision of a test aimed at reducing analytic errors to the
minimum required for a valid veterinary application of the test" (from:
Bellamy, 1999).
2.a What is Quality Assurance (QA)?
When we just look at the words themselves: "the guarantee for quality". For
a fish diagnostic lab it means, that the results of the used test are
correct. These results may be used to make a diagnosis of a disease
outbreak, or to confirm absence of a disease, and are based on qualitative
investigations. This implies that there should be standard operating
procedures (SOPs) at the laboratory for every test.
The level of quality (QA) can differ:
- standardization and internal laboratory Quality Control (QC) is a
start for QA
- accreditation according to ISO 9001, European Standard EN 45001 or USDA
norms is more advanced
- interlaboratory QA is an important value addition to the
accreditation
2.b Where and for whom should QA be practiced?
From the local fish diagnostic lab to the EU reference lab, each at their
own level. Who would be interested in your lab working to standard
procedures, and possibly being accredited? The costumer, the vet, the fish
farmer and the government officer because they can count on quality of your
diagnosis or experiments. It makes your position as a lab stronger, because
you can demonstrate that you deliver quality work, and you may print a "Q"
on your letters. There should be some economic reason, otherwise the costs
would be easily be too high.
2.c Why QA?
It is important to prove to a client that you "did what you "said" (wrote
down in your protocol), and you "say what you did" (write down in your log
book or on the form) what you did. For the employee, QA gives more insight
into its their work.
Overall, it means that you have to work strictly to a regime, which you
describe beforehand. To start with intralaboratory QC you write down the
way you have worked until now, and make this the standard way. If your lab
crew changes, the test is still well described and can be taken over by
another technician staff member who has passed the working-introductory
program, as laid down in documents.
Why would you accredite your fish diagnostic tests? To be certain that the
test results are right, not only within your lab and according to your
knowledge, but also related to reference samples from other labs and
according to experts from other labs. In this way you can standardize your
tests, and demonstrate that the results of your tests are always the same
for a certain sample, independent of the day, technician staff member,
weather conditions, mood, or whatever. It implies a lot of validation work,
dependent on whether the test is a qualitative or a quantitative test: how
specific is the test, how sensitive, etc.
2.d How can you reach this abstract form of quality?
You are not the first lab to do so, so there are guidelines present in this
paper (see references), but also at the accrediting firms. It takes a big
effort in the start, but the advantages in QA can soon be seen. For a
minimum QA you could standardize your tests: write down how you do your
test, make this the standard way, and double check results of the test by a
second person. Use reference samples (virus, bacteria etc., preferrably
from other labs) as positive controls. For accreditation: see 3-8.
2.e What system of accreditation should you choose?
If you have a small fish diseases lab with few samples per year, and carry
out some research in fish diseases, you should probably choose
intralaboratory QC or ISO 9001. It means that the first step of QA is
reached, but the accreditation agency is not checking the scientific
contents of your tests. As an example, they do not assess: why do you use
this cell line instead of that one. Therefore, you show that you use your
tests reliably and in a consistent manner, but you do not actually show
that you are using the most appropriate test.
This latter assessment is the case with European Norm (EN) 45001: they
critically look into the set up of your test or experiment, and ask for a
strong validation of your test: for example prove that the cell line is
always susceptible to the virus, test the specificity of the test, test
that the temperature fluctuations of the incubator do not influence the
test results, etc.. You could also consider partly accrediting your lab
tests according to EN 45001. Necropsy is always difficult to objectively
standardize, but, for instance, virus isolations methods for export
releated certification on notifiable diseases can be easily standardized,
based on the regulations of the European Commission.
Example: At AAHL (Geelong, Australia) they have laboratory accreditation
(to ISO/IEC Guide 25) for Veterinary Testing under the NATA Veterinary
Testing Laboratory scheme, and certification to the quality management
standard ISO 9002.
Note : Of the quality management standards that apply to laboratories the
standard ISO 9001 would apply if research activities were included in the
certification ( i.e. design, development). Standard ISO 9002 can be used if
research activities are not included in the certification. We do not
further go into ISO 9002 here.
Return to Table of
Contents
Return to Main Page
3.a ISO 9001 criteria Internal QA
* What steps are undertaken to develop the end
product (f.i. diagnosis) out of the delivered products or services?
Standard Operating Procedures (SOPs) have to be worked out.
* How do you judge good or wrong related to the
product? How can you trace back the process if the product was wrong?
* Describe the process of production or service
(in SOPs)
* Define controls at the start, during the
process, and at the end (including positive and negative controls, double
checks of results reading etc.)
3.b Internal QA according to EN 45001
Make SOPs of your tests and reagents, based on the example SOP of the QA
handbook
Use a log book, to document all relevant actions during lab testing, like
titre calculations, which are not in the SOP or the result form. The
procedure on how to use it should be composed and available.
* How is the organization of the tasks in
the primary process?
* With what frequency are internal audits held?
* How are quality documents at department or
lab managed?
* What is the procedure for ordering materials
at the lab or department?
* What is the procedure of non-standard
activities?
* What validation of immunodiagnostic and
virological tests do you implement? (See 7)
* Do you also externally quality control your
immune diagnostic tests by the proficiency testing (see 4)?
* How do you manage and archive samples and
reports?
3.c Internal QA according to USDA guidelines
Will be added prior to the workshop.
3.d Canadian recommendations on QA (Dept of Fisheries and Oceans
Canada)
It is essential in aquatic animal health certification programs that
disease diagnostic tests be consistent and reliable. The results of tests
can have a significant economic impact on an aquaculturist by determining
whether a facility can sell products to other provinces or countries. Data
from these diagnostic tests are also used to determine whether zones are
free of specific diseases or not. The disease designations for zones are,
in turn, important when assessing disease risks of proposed transfers of
aquatic organisms.
To improve and enhance the consistency and reliability of diagnostic tests,
it is proposed that a new Quality Assurance (QA) Program be implemented for
finfish disease diagnostic laboratories. This program will be developed by
Department of Fisheries and Oceans, and written up as a separate Quality
Assurance Manual.
The Manual will be a policy document, outlining the intent of the QA
Program. The main purpose of this program will be the production of high
quality, or reliable laboratory results. Reliable in this case means
accurate and precise results of appropriate sensitivity and specificity for
the use to which the test has been put and for the test itself. In
addition, a function and goal of the QA Program is to provide evidence of
this quality, not only to the diagnostician but also to clients and to
program managers.
Reference Laboratories
A key element of the QA/QC program will be the identification of reference
laboratories for specific finfish diseases and disease agents. The basis
for this recognition will be the presence of expertise and specialized
equipment to test for the specific diseases or disease agents.
Reference laboratories will be available to confirm the detection of
diseases or disease agents by other diagnostic laboratories, and
investigate the identification of new diseases or disease agents.
Return to Table of Contents
Return to Main Page
An example is the inter-laboratory proficiency testing scheme for EU
National Reference Laboratories for fish diseases. It is based on the
Coucil Directive 91/67/EEC. In the last EAFP Conference Dr. N.J. Olesen
presented a paper about it (Hastings, 1997). In short, coded lyophilized
preparations with or without virus from naturally or experimentally
infected fish tissues are sent to participating laboratories, accompanied
by written procedures and recording protocols. The labshave to titrate the
samples on standard cell lines and identify them. The aim is to obtain
uniformity between the reference labs. Apart from this aim, this type of QA
is useful to evaluate methodologies. The interlaboratory variation is
calculated and the median of the test results.
Return to Table of Contents
Return to Main Page
5.a. Quality of Operations and resources related to ISO 9001
The following subjects have to be documented:
* responsibility of the director: the whole
quality system, job descriptions, and looking after the quality system to
work in practice
* description of the quality system in a
quality handbook
* how are contracts between institute and
clients made?
* What steps are taken to fullfill a clientÌs n
order of a client?
* What paper forms and computer systems are
used and in what way?
* How do you choose suppliers and what is the
purchasing procedure?
* Description of the measuring apparatus used
in the process
* Intermediate quality control of the product
or service
* How are faults or complaints dealt with?
* How should products be stored and transported
without loosing their quality?
* Which data are recorded to register the
quality of the product or service?
* The institute should internally audit the
process, to test if the QA works in practice
* How are courses being done?
* After-care to the client related to the
product or service: how is it done?
* Application of statistical techniques in
quality control
5.b Quality of Operations and resources related to EN 45001
The following subjects have to be documented:
* The Institute: mission, tasks,
third-party insurance
* Organization and personnel: organization
structures (several levels), education and tasks, work pressure,
independence and impartiality, secrecy, primary process: for instance:
diagnostic testing of fish diseases, the elements of support of the primary
process.
* The Quality System: quality policy and aims;
quality handbook and further quality documents, judgements of the quality
system.
* Housing and Security: housing, rules about
use of rooms and facilities, Security system of housing
* Acquisition of materials and apparatus:
buying policy, judgement of suppliers, acceptance policy of objects for
diagnosis
* Acceptance, control, storage, and
distribution of goods: acceptance and control, preparation of samples and
administrative tagging of the samples, distribution, storage.
* Management and archiving objects (samples)
and apparatus
* Control of diagnostic testing: methods
(standard operating procedures = SOPs) and practice, internal delivery of
samples and apparatus, calibration of apparatus
* Automated systems: general management of
automated systems, automated data transmission (like ELISA readers) and
reporting
5.c Quality of Operations and resources related to USDA
guidelines
The following subjects have to be documented: text will be added prior to
the workshop.
Return to Table of Contents
Return to Main Page
6.a
In practice, the quality book for ISO 9001 of our lab (ID-DLO,
Lelystad) has the following contents:
* Organization and personal personnel of
the institute: organization form, tasks, structure, organisation management
, personnel personal management, conference structures
* The quality system: quality policy and aims,
security and environment protection, the quality book, judgement of the
quality system
* Organisation and responsibility of the
primary process: responsibility, structure of the research organization,
program policy, project policy, research policy at dept. level
* Practicing the primary process: initiation of
research, preparation of research policy, management of the practice of
research, ending the research, patents and licencies
* Accepted samples and goods: buying policy,
media and reagents, acceptance policy related to samples
* Management of samples
* Management of technical apparatus: buying,
registration and identification, maintenance and
callibration/justification, automized automated systems
* Services and facilities: housing, animal care
taking, maintenance of installations and instruments, documentation,
statistical support, public relations, logistic and further facilitating
services
* Annexes: general data ID-DLO, list of quality
documents, relation with ISO 9001 norm
* Safety regulations should be documented and
accessible at every laboratory or department.
6.b Documentation of EN 45001 (Sterlab):
General working documents and procedures:
For instance: use of a log book, organization of the tasks in the primary
process, internal audits, management of quality documents at department or
lab, ordering materials, example of standard operating procedure (SOP),
model document for method of preparation, procedure of non standard
activities, validation of immunodiagnostic and virological tests, external
quality control of immune diagnostic tests, management and archiving
samples and reports
Organization of the lab:
Organization scheme with persons, description of each job, task matrix of
the lab, cv's per person, settlement program per person
Standard Operating Procedures (SOPs):
Each with the set-up: title, aim, definition/declaration, principle,
precision (validation), requirements, procedure, results interpretation and
registration, storage of samples and other materials, registration,
remarks, references, annexes
Results of tests:
The results of each sample should be read by one person, and the result
should be checked by a second person. The data of each sample tested at the
lab should be well documented, preferrably on standard forms. The results
of samples tested at the lab should be well documented, preferrably on
standard forms. EN 45001 wants extra control of each reading. If the
standard operating procedure is altered, this should be written down and
explained with reason in the log book and the standard form. It should be
proven that this other method is not negatively influencing the result of
the test. The report with the results should also be checked by a second
person. All notes, forms and letters should be kept in store for at least 5
- 10 years.
Validation testing:
Validation of tests is a separate chapter (see 7). All SOPs and test
results related to validation have to be well documented. Also,
calculations of validation data have to be accessible in the log-book or
another file.
Temperature data of fridges, incubators etc.:
Every day or other time span the temperature of the apparatus which are
used for the tests at the lab have to be registered. For that we use
digital thermometers, which give the minimum and maximum temperature in the
last period since last resetting. In that way you can note malfunctions of
the apparatus, and understand why for instance the cell line is killed
during the weekend at too high temperatures, and take measures. In your SOP
you note the temperature tolerance limits of the fish cell lines, bacteria
etc., and in the documentation of the apparatus you define the accepted
temperature range.
Technical apparatus and maintenance data:
The frequency of maintenance check of each apparatus (from pipetting
apparatus to incubator) by the technical service should be defined in a
SOP, and the maintenance reports should be documented and kept for control.
Storage forms of samples and reference samples etc. etc.:
From each sample it should be easy to find out through forms, if it is
stored, and if so, where (what box no. etc.), at what site (fridge,
-20oC or -70oC for instance). If a vessel container
is removed, its' number should be crossed out and a stripe should be made
through it in the storage overview forms, so that you know at all times
from the documentation what is in store. The reference samples should also
be stored at defined places. The results of testing the reference samples
at your lab, and the origin of these samples should be well documented.
They are needed as controls, especially for the validation of your tests.
Safety regulations:
General safety procedures should be known to all employees, and documented
at a central place per department or laboratory.
6.c Documentation of Quality: Canadian recommendations (Dept. of
Fisheries and Oceans):
The Quality Assurance Manual and related operations documentation will
contain the following:
* a quality policy statement, including
objectives and commitments; the organization and management structure of
the laboratory, its place in any parent organization and relevant
organizational charts, and the relations between management, technical
operations, support services and the quality system;
* the laboratoryÌs scope of test activities;
the control and maintenance of the quality manual and related operating
procedures documentation
* job descriptions of the key management and
technical staff or reference to them;
* identification of the authorized signatories
for certificates and reports;
* validation of test methods used at the
laboratory;
* sampling procedures;
* the handling of test items (samples);
* records and a descriptive listing of the
major items of test equipment and on any reference measurement standards,
including reference materials, used;
* reporting the test results;
* monitoring the test results;
* feedback and corrective actions whenever
discrepancies are detected, or departures from documented policies and
procedures occur;
* departures, in exceptional circumstances and
for valid technical reasons, from the laboratoryÌs documented policies and
procedures, or from standard specifications;
* dealing with feedback from customers;
* protecting confidentiality and proprietary
rights, both the laboratoryÌs own data and know-how, and information
derived from clients;
* internal audits and management reviews.
* Qualifications and training of the staff to
ensure compliance with the quality system and related operating procedures;
* the retention of records;
* if work has to be subcontracted, procedures
to ensure that subcontractors are competent and comply with the
requirements of this Guide;
* procurement of support services and supplies;
* customer service including the control over
communications with the customer.
Return to Table of Contents
Return to Main Page
One way to evaluate lab procedures is by internal and external audits. They
are held for instance once a year. The auditor checks whether the
implementation of the QA documents is correct at all levels of the process,
and for instance, if whether the vessels containers with virus no. X can
indeed be found in the freezer at the site, which was documented in the
forms. The internal auditor is a colleague from the same institute, the
external auditor is someone from an official accreditation agency. In both
cases, a report is made of the audits, and action points are formulated to
improve QA. If the quality appeared to be too low, the external agency may
withdraw the approval of the test.
To validate the SOPs at your lab, there are schemes highlighting which data
need to be collected. Table 1 presents roughly, what validation should take
place in which case (data from ID-DLO, Lelystad, The Netherlands)
Table 1. Validation of qualitative and quantitative tests for EN
45001
1 = always needed
2 = if relevant
3 = not necessary
|
Quantitative
method# |
Qualitative
method* |
|
Low concentation
level |
High concentration
level |
| Test: |
New
Method |
Standard
Method |
Changed
Method |
New
Method |
Standard
Method |
Changed
Method |
Correctness
|
1 |
1 |
1 |
1 |
1 |
1 |
3 |
Repeatability
|
1 |
1 |
1 |
1 |
1 |
1 |
3 |
| Reproducibility |
1 |
1 |
1 |
1 |
1 |
1 |
3 |
Range of detection
|
1 |
2 |
2 |
3 |
3 |
3 |
1 |
Range of determination
|
1 |
2 |
2 |
3 |
3 |
3 |
3 |
Linearity
|
3 |
3 |
3 |
1 |
3 |
3 |
3 |
Selectivity
|
1 |
3 |
3 |
1 |
3 |
3 |
1 |
Robustness
|
1 |
3 |
3 |
1 |
3 |
3 |
1 |
(#) Like ELISA or Serum neutralisation assay
(*) Like virus isolation and typing by IFT
Validation is important to determine and prove the quality of your test.
First define the confidence level you want to reach (f.i. 95%).
Correctness:
The correlation between the mean from a set of tests (10 samples f.i.) and
the individual data. It can be determined comparing your sample with
certified reference samples, for instance your standard control sera with
positive and negative control sera from another reference lab. It can also
be determined by participating in a proficiency test (see chapter 4).
Related to a changed method, this change can be validated by comparing the
results of a set of samples from the original and changed test. it also can
be used for recovery experiments, in which the to be tested component is
added to the sample.
Repeatability:
This is a measure for the variance between results of testing identical
samples with the same methods at exactly the same circumstances. In
practice: the variation in test results of 10 times the test by one person
at the same day at one lab. For an ELISA for instance, use sera with a low,
medium and a high titer, and test them each 6 times. It is calculated by
CV% = SD/X (mean) * 100%.
Reproducability:
Are the results of a certain sample tested at your lab the same when tested
at other labs? It can be calculated from results within the laboratory
(internal reproducability) or between labs (proficiency tests), see 4. It
tells something about especially systematical failures in a method. The
variable circumstances can be for instance: different technicians,
different apparatus, different days etc.. For an ELISA for instance: test
sera with a low, medium and high titer each 6 times. It is calculated from
the relative standard deviation, with CV% = SD/X (mean) * 100%.
Range of detection:
Are the results of a certain sample tested at your lab the same when tested
at other labs? It can be calculated from results within the laboratory
(internal reproducability) or between labs (proficiency tests), see 4. It
tells something about especially systematical failures in a method. The
variable circumstances can be for instance: different technicians,
different apparatus, different days etc.. For an ELISA for instance: test
sera with a low, medium and high titer each 6 times. It is calculated from
the relative standard deviation, with CV% = SD/X (mean) * 100%.
Range of determination:
3*range of detection.
Linearity:
For instance how linear is the titration result in your ELISA? It can be
determined by linear regression
Selectivity or specificity:
aHow selective is your agar medium to the pathogen you want to isolate? How
selective is the antiserum related to the different viruses? What is the
influence of metabolites of a certain chemical component on the result? The
specificity is calculated by deviding the no. of negative samples in a test
by the real no. of negative samples.
Example ID-Lelystad: selectivity of the IIFT serum: tests all relevant fish
viruses in your test: there should be no cross reaction, and no repeats are
necessry.
Robustness:
What happens with the test result if one of the procedure steps is
forgotten, or, if the temp of the incubator has been 5 degrees too high
etc.. It has to be defined for all critical possibilities of failure in a
test. It cannot be calculated, but you can add a table with the
consequences of abnormalities related to the test related to the test
result. This should be one of the first validations to make in a new
test.
Example ID-Lelystad: what is the influence of :
* concentration of media: test each ingredient
at 3 concentrations (0.1, 1 and 10x), in trplo per cell line
* incubation time of the test (0.5, 1 and 2x)
* medium exchange, like culturing the cell
lines with EMEM instead of HMEM
* temperature of incubation: test all cell
lines in triplo at 15, 20, 26, 30 and 37öC.
Return to Table of Contents
Return to Main Page
* Identify customers/stakeholders that may be
interested in your lab being accredited.
* Get a vision of where your lab may be in the
next five years (does it need accreditation; are the customers going to
change; will you be doing fully-cost recoverable contract work; will you
have to compete for work?)
* Choose what system you may usefully apply.
* Contact a consulting firm to advise you on
details.
* Do a proper budgeting for the selected
system.
* Do a cost benefit analysis.
* Decide whether you can 'afford' it.
Then you start with the 'real' work, getting the data together, see 3-7
The costs for QA can be very high, especially if external accreditation
agencies are involved. In the appendix 1, 2 and 3 examples are presented on
costs made by different institutes to accredite their tests. The cost
benefit analysis is important. However, if your lab is known to be a
qualitatively good one, it might pay back to some extent.
Return to Table of Contents
Return to Main Page
* Working to QA diminishes the scientific
output of the organization, because it takes too much time
* Engaging on an accreditation process MUST be
budgeted for in advance, both in staff resources (time) as well as real $$
figures.
* Motivation of each participant in the process
is necessary to work with QA, and is more important than any written letter
in the QA book
* If you want to talk about international QA
the need of central production of standards (antisera, cell lines, pathogen
batches, and other biologics) is unmissable, as is the regular
interlaboratory proficiency testing.
* QA? Is it better to make the standard work
for you rather than you work for the standard (J. Tanti).
Return to Table of Contents
Return to Main Page
* QA is becoming more and more important in
fish diagnostic labs
* There are many QA possibilities, from SOPs
without accreditation until EN 45001
* QA is expensive, but might pay back because
of the gained quality and status of the lab
* there is a need for a simple manual for small
and bigger fish diagnostic labs, about how to start and work on QA, with
practical examples
* the US and Canadian QA system and the QA
system in Europe and Asia could be better coordinated and benefit from
eachother
* as a result of the discussion during the
workshop the EAFP Council will further focuson QA towards other legal
authorities
Follow up:
From the discussion it is clear, that QA is alive as a new topic in fish
diagnostic laboratories. The need for practical advice on how to start with
and implement QA was unanymously expressed by the audience. Apart from the
actions of the EAFP Council some volunteers are needed to pursue this
further (see the note in this bulletin). Reactions can be sent to one of
the authors of this working document.
Acknowledgements
The authors are grateful to the invited speakers of the QA
workshop for their interesting lectures: Dr. D. Alderman from the Office
International des Epizooties, Dr. S. LaPatra from the QA/QC Committee of
the Fish Health Section of the American Fisheries Society, and Dr. Alex
Thierman from US Department of Agriculture.
Return to Table of Contents
Return to Main Page
Bellamy, J., 1999. Re: Quality Assurance in Veterinary Laboratories. In:
Update and Letters to the Editor. Vol 15 (1): 13-15. Published by The
College of Veterinarians of Ontario, Canada.
Dybkaer R. et al., 1993. A quality manual for the clinical laboratory
including the elements of a quality system. Proposed guidelines. Scan. J.
Clin. Lab. Invest. 53: Suppl. 212: 60-84.
European Standard, 1989. EN 45001. General criteria for the operation of
testing laboratories. CEN/CENELEC, The joint European Standards
Institution, Brussels, Belgium, UDC 620.1.008.6.
European Standard, 1994. EN ISO 9001. Quality systems. Model for quality
assurance in design, development, production, installation, and servicing.
CEN, European Committee for Standardization, Brussels, Belgium. UDC
658.56:006.83.
Hastings, T.S., 1997. Quality Assurance in fish and shellfish disease work.
Bull. E.A.F.P. 17,6: 261-263.
ISO 9001 : 1994 Quality systems - Model for assurance in design,
development, production, installation and servicing. Available from the
International Organization for Standardization (ISO), or the national
Standards body in individual countries.
ISO 9002 : 1994 Quality systems - Model for quality assurance in
production, installation and servicing. Available from the International
Organization for Standardization (ISO), or the national Standards body in
individual countries.
ISO 8402 : 1994 Quality management and quality assurance - Vocabulary.
Available from the International Organization for Standardization (ISO), or
the national Standards body in individual countries.
Laboratory Accreditation : ISO/IEC Guide 25 - 1990 General requirements for
the competence of calibration and testing laboratories. (soon to be
replaced by ISO 17025). Available from the International Organization for
Standardization (ISO), or the national Standards body in individual
countries.
Lloyd, W.E. & Stahr, H.M., 1980. Quality Assurance in laboratory practice
[veterinary procedures]. Proceedings of Annual Meeting, American
Association of Veterinary Laboratory Diagnosticians (USA), no. 23 :
417-419.
National Association of Testing Authorities (NATA) 1. Guide to the
Development of a Quality System for a Laboratory. 2.General Requirements
for Registration 1992. 3. Supplementary Requirements for Accreditation -
Veterinary Testing. Available from NATA , Australia, 7 Leeds Street,
Rhodes, NSW 2138.
Osuch, M.A.V., 1996. Preparing for your ISO quality management guideline
audit and registration. Genetic Engineering News, Sept. 1996: 17.
Quality Management : ISO 9000 : 1993 Quality management and quality
assurance Standards. Available from the International Organization for
Standardization (ISO), or the national Standards body in individual
countries.
Rohle, R., 1993. Externe Qualita tssicherung in Deutschland. CCKL bulletin,
March 1993 8(1): 1-2.
Unites States Pharmacopeia XXII, 1989. National Formulary XVII, general
chapter "Validation of Compendial Methods". The US Pharmacopeial
Convention, Inc., Rockville, Maryland, p 1710.
WHO, 1993. Report meeting on "Quality Assurance related to health
laboratory technology". April 21-23, 1993, Geneva.Return
to Table of Contents
Return to Main Page
QUALITY MANAGEMENT SYSTEMS -An Australian example
Compiled July 2, 1999
Relevant Facts Using Laboratory XXX as an Example
Institute Size: 270 Employees
Annual Budget: approx. 7.1 Million euro's
The following costs are for certification and accreditation of Diagnostic
Services only.
Quality Management Systems
* Certification Standard: AS/NZS ISO 9002:1994
AS=Australian Standard; NZS=New Zealand Standard (adopted the
International Standard ISO 9002:1994 which was published in 1994)
* National Association of Testing Authorities
(NATA), Australia accreditation obtained: "Accredited Laboratory in the
Field of Veterinary Testing"
Costs (euro's)
The process of registration and accreditation required approx. 24 months to
complete.
Salaries:
1 QA Manager (25% for first 6 months in training - then full time for 18
months) and approximately 20 other staff members contributing approximately
10% of their time.
| Total Salary Cost (2 Years): Salary + On-Costs (20%) = |
88,503.-
|
| Training (senior staff and internal auditors): |
3,218.-
|
| New Equipment (including computer software): |
6,436.-
|
| Registration and Certification: |
3,218.-
|
| TOTAL COST |
euro's 101,375.-
|
Maintenance Costs
| QA Manager (50%: 30% on maintenance; 20% on development): |
9,655.- |
12 Staff Members contributing 5% of their time
(internal audits, equipment calibration and monitoring, document
management): |
9,655.- |
| External audits, annual registration fees: |
1,609.- |
| TOTAL MAINTENANCE COSTS: |
euro's 20,919.- |
Notes
The following consideration needs to be emphasised. Prior to accreditation
and certification, Laboratory XXX had infrastructure, systems (engineering
support) and many of the requirements for accreditation and certification
already in place. Other institutes may not be starting off at the same
level. Much of the process is concerned with documentation and maintenance
of records. Establishment of computer systems to underpin this process is
essential for an organisation of this size. Even with the level of
sophistication present at Laboratory XXX the total time to complete the
process was underestimated by 50%. Costs do not include overheads.
ANNEX 2 : EXAMPLE OF COSTS OF QA (amounts in
euros)
A second example of veterinary institute Y
Relevant Facts Using Laboratory Y as an Example
Institute Size: 700 Employees
Annual Budget: approx. 54,453,626.- euro's
The following costs are for certification and accreditation of Diagnostic
Services and research.
Quality Management Systems
* ISO 9001: all research and related
sustaining departments
* EN 45001 : 75 tests at 6 labs
* Good Laboratory Practice (GLP): animal
experiments
Accredited Laboratory in the Field of Veterinary Testing
Costs (euro's)
The process of registration and accreditation required at least 48 months
to complete.
Salaries:
2 QA Managers (full time) and approximately 8 other staff
members contributing approximately 10% of their time.
Total Salary Cost (per year): |
1,089,072.- |
| Training (senior staff and internal auditors): |
13,613.- |
| New Equipment (including computer software): |
9,075.- |
| ESTABLISHING CERTIFICATION AND ACCREDITATION: TOTAL COST: |
euro's 1,111,760.- |
Maintenance Costs
| 2 QA Managers (full time) |
181,512.- |
| 8 Staff Members contributing 10% of their time |
72,604.- |
External audits, annual registration fees:
ISO:
EN 45001 |
9,075.-
11,344.- |
| TOTAL MAINTENANCE COSTS : |
euro's 274,535.- |
ANNEX 3: EXAMPLE OF COSTS OF QA (amounts in
euros)
A third example
Here are estimates for QA costs for a lab, which would be classified as a
moderate sized veterinary diagnostic laboratory at a university.
Total Budget including salaries: euro's 1,554,200.-
Cost for Establishing Certification/Accreditation year 1
| Salaries |
10% of budget |
| Training |
5% of budget |
| New Equipment |
5% of budget |
| Registration and Certifications |
5% of budget |
| TOTAL COSTS |
25% of budget |
Maintenance Costs Yearly:
| Salaries for QA (contribution by all personnel) |
10% of budget |
| Internal audits, standards, calibration etc. |
2.5% of budget |
| External audit programs and registration fees |
5% of budget |
| TOTAL MAINTENANCE COST PER YEAR |
17.5 % of budget |
Return to Table of Contents
Return to Main Page
In September 1996, representatives from federal and state government, the
private aquaculture industry and university researchers met in Denver to
discuss the use of advanced technologies in aquatic animal medicine. The
general consensus of the group was that the use of advanced technology was
not a new topic but one in which concerns have intensified as a result of
recent applications of extremely sensitive DNA test procedures. The test
procedures were new enough to the aquatic animal health field that they may
have lacked specificity validation and standardization regarding use,
application, and interpretation of results. The results of these tests
could result in economic loss to the aquaculture industry and loss of
valuable fish stocks to the sport fishing industry. A series of questions
were then sent to the states, fish health protection groups and private
industry for response and thoughts on the use of advanced
technologies.
In response to this letter the Great Lakes Fish Health Committee submitted
a response to the American Fisheries Society Fish Health Section President
JoAnn Leong voicing its concerns about the use of advanced technologies.
The Great Lakes Fish Health committee requested that the American Fisheries
Society Fish Health Section take the lead role in developing a unified
approach to the following areas:
* Develop a QA/QC program for fish diagnostic
laboratories;
* Develop a process through which new fish
health technologies can be validated and incorporated into regular use;
* Develop guidelines for resolving inconsistent
test results when multiple diagnostic techniques are used to detect the
same pathogen 'Gold Standards'.
As a result of the letter from the Great Lakes Fish Health committee the
American Fisheries Society Fish Health section developed a committee at
first to deal with the issues of QA/QC and advanced technologies, but
realized this chore was more than one committee should handle thus a
second committee has now been established to develop means of testing and
standardizing new methods for use within the diagnostic laboratory.
QA/QC
It is the intent of the Quality Assurance/ Quality Control committee to
accomplish the following:
* Create the framework for an overall QA/QC
program for the American Fisheries Society Fish Health Section.
* Develop guidelines for individual
laboratories to create their own QA/QC documents.
* Provide guidance to individual laboratories
developing their own QA/QC documents by establishing an ad hoc QA/QC
document review committee within the American Fisheries Society Fish Health
Section.
* Provide input to the committee rewriting the
American Fisheries Society Fish Health Section Bluebook so that we have
good Standard Operating Procedures (SOP) that are in a format consistent
with ISO 9000 guidelines and add a section to the Bluebook for
QA/QC.
A special session was organized by the American Fisheries Society Fish
Health Section to encourage dialogue on this issue as part of the 3rd
International Symposium on Aquatic Animal Health. Topics included in this
special session included:
* The use of non-validated techniques in
the diagnosis, inspection and certification of aquatic animals.
* Development of a process to validate and
standardize diagnostic methods.
* Consistent use of diagnostic methods and
requirements for confirmation.
* The use of results obtained by non-validated
techniques and the potential legal and political ramifications.
* Management implications of detecting nucleic
acids or other components of aquatic animal pathogens.
Individuals from industry, research, Veterinary services, federal and state
government were invited to give their viewpoints and participate in the
discussion.
Several key points raised by the speakers were as follows:
* Any application of a new diagnostic test
must be evaluated for practicality, value, reliability and reproducibility.
Practicality refers to utility in the laboratory; simplicity, speed, cost,
and changes required of current SOP's or technical abilities of laboratory
personnel. Value includes the importance of the disease, the utility of the
test in determining management options, the specificity and range of the
test, and archival potential. Reliability and reproducibility refer to
frequency of false positives, false negatives, assay failure and laboratory
to laboratory variation.
* The development and evaluation of a disease
diagnostic test and a test for inspection/certification purposes are very
different. A PCR assay for inspection purposes requires much more
stringent quality control measures than would be needed for a diagnostic
test. The use of PCR for inspection purposes has distinctive advantages in
sensitivity but requires stringent quality control and each application
should be critically evaluated before the results are used to determine
the types of important decisions that can occur in inspections.
* The processes included in getting a
diagnostic test put on line in a veterinary diagnostic laboratory.
* OIE and its role in aquatic animal health.
With the rapid development of novel techniques for the detection of fish
pathogens, the issue of assays and the benchmarking of new assays against
known 'GOLD STANDARDS' have become very important. While the fish health
community needs to address this issue, we are not alone in that the Office
International des Epizooties has formed a working group to consider this
problem in veterinary medicine.
In conclusion the area of Quality Assurance/ Quality Control is
increasingly important as new more sensitive diagnostic assays are
developed and used in aquatic animal medicine. Also there is a need for
formal guidelines by which new methods must be compared with established
benchmarks before they can become accepted as standards (the AFS/FHS has
established a committee to begin work in this area). This is a huge task
that needs to be addressed and worked on cooperatively with OIE, EAFP,FHS
and JSFP to harmonize international standards for the validation and
certification of fish disease diagnostic assays.
GLOSSARY
AFS American Fisheries Society
EAFP European Association of Fish
Pathologists
EN 45001 European Norm standard no. 45001
(Sterlab)
FHS Fish Health section of the AFS
GCP good clinical practice
GLP good laboratory practice
GMP good management practice
GVP good veterinary practice
ISO International Organization
for Standardization
OIE Office International des
Epizooties
QA Quality Assurance
QC Quality Control
SOP Standard Operating
Procedure
Sterlab see EN 45001
USDA Unites States Department of Agriculture
Return to Table of Contents
Return to Main Page
Document Updated: 1st March 2000