Certification Information for October 3 presentation.

The National Organization for Competency Assurance (NOCA)

•  Established quality standards for credentialing organizations.

•  Established in 1977, NOCA serves as a clearinghouse for information on the latest trends and issues of concern to practitioners and organizations focused on certification, licensure, and human resource development.

The National Commission for Certifying Agencies (NCCA)

•  Accreditation body of NOCA.

•  Certification programs apply to be accredited by the NCCA if they demonstrate compliance with each accreditation standard .

NCCA Standards summary (full text found on NOCA Web site at http://www.noca.org/docs/StndFeb02.pdf )

Note: The NCCA Standards exceed the requirements set forth by the American Psychological Association and the U.S. Equal Employment Opportunity Commission.

NCCA has determined that a Certification Program must:

•  Conduct certification activities in a manner that upholds standards for competent practice in a profession, occupation, role, or skill (henceforth referred to as "profession") .

Essential elements:

•  Delineate the certification activities

•  Provide justification for the appropriateness of the certification activities

2) Structure and govern the certification program in ways appropriate for the profession that ensure autonomy in decision making over essential certification activities.  

Essential elements:

•  Show governance structure, policies, and procedures that protect against undue influences that could compromise the integrity of the certification process

•  Provide for autonomy in decision making regarding important aspects of the structure, policies and procedures for certification, such eligibility standards, the development, administration and scoring of the assessment instruments; selection of personnel; and operational processes.

•  Promote the purpose of the certification program in the development, administration, and scoring of the assessment instruments.

•  Avoid conflicts of interest between certification and education functions ; the certification agency must not also be responsible for accreditation of educational or training programs or courses of study leading to the certification.

3) Include individuals from the certified population on the certification board or governing committee. Must also include voting representation from at least one consumer or public member.  

Essential elements:

•  Establish a structure for ensuring appropriate stakeholder involvement by designating certain representative positions on the governing body.   To ensure a balance of program input, the governing body may implement a rotating system of representation over a set period of time

•  Establish bylaws and/or policies for selection of individual to serve on the governing board.   This information must show that the selection of these individuals prevents inappropriate influence from a parent or outside body.

    4) Provide sufficient financial resources to conduct effective and thorough certification and recertification activities.  

Essential element:

•  Financial reports of the certification program must demonstrate adequate resources available to support ongoing certification and recertification processes.  

5) Must have sufficient staff, consultants, and other human resources to conduct effective certification and recertification activities.

Essential elements:

•  Key staff and non-staff consultants must possess adequate knowledge and skill to conduct certification activities

•  Certification program must have adequate resources to conduct the activities (e.g., process applications, administer the assessment instrument, store records) of the certification program.

6) Establish, publish, apply, and periodically review key certification policies and procedures concerning existing and prospective certificants (e.g., determining eligibility criteria, application for certification, administering assessment instruments, establishing performance domains, appeals, confidentiality, certification statistics, discipline, and compliance with applicable laws.

Essential elements:

•  Publish documents which clearly define the certification responsibilities of the organization including:

•  The purpose of the certification program

•  Eligibility criteria and application policies and procedures

•  Materials outline all examination processes and procedures

•  A detailed listing and/or outline of the performance domains, tasks and associated knowledge and/or skills

•  A summary of certification activities (number of candidates examined, pass/fail statistics, and number of individuals currently certified)

•  Discipline , nondiscrimination, and confidentiality policies and procedures

•  Appeals policies and procedures

•  Confidentiality policies must (a) ensure that candidate application status and examination results are held confidential, and (b) delineate the circumstances under which this information may be disclosed.  

•  Policies and procedures must be published and must include guidelines by which candidates may question eligibility determination , assessment instrument results, and certification status .

•  Disciplinary policies must include procedures to address complaints, which may concern conduct that is harmful to the public or inappropriate to the discipline.   These policies must ensure appropriate treatment of sensitive information and fair decision-making.

7) Publish a description of the assessment instruments used to make certification decisions as well as the research methods used to ensure that the assessment instruments are valid.  

Essential element:

•  Procedures related to assessment instruments must address development and validation, eligibility requirements, and administration

8) Award certification only after the knowledge and/or skill of individual applicants have been evaluated and determined to be acceptable.

Essential Elements:

•  If, at the time of application for accreditation, any current certificants are granted certification without having met the examination requirements established for certification, a rationale must be provided to explain how the competence of those individuals was evaluated and found to be sufficient.  

•  Once a program is accredited, " grandfathering" or any other procedure for granting a credential in the absence of evaluating the knowledge and/or skill of an individual is not acceptable .

      9) Maintain a list and provide verification of certified individuals.

   10) Analyze, define, and publish performance domains and tasks related to the purpose of the credential.  

Essential elements:

•  A job analysis must be conducted leading to clearly delineated performance domains and tasks, associated knowledge and/or skills, and sets of content/item specifications to be used as the basis for developing each type of assessment instrument (e.g., multiples-choice, essay, oral examination).

•  A report must be published linking the job/practice analysis to specifications for the assessment instrument.

  11) Employ assessment instruments that are derived from the job/practice analysis and that are consistent with generally accepted psychometric principles.  

Essential elements:

•  Assessment instruments must be products of an appropriately designed and documented development process

•  The content sampling plan for test items or other assessment components must correspond to content as delineated and specified in the job/practice analysis.

•  An ongoing process must exist to ensure that linkage between assessment instruments and job/practice analysis is maintained.

•  Certification programs must follow a valid development process that is appropriate for assessment instruments.

•  A systematic plan must be created and implemented to minimize the impact of content error and bias. Qualified subject matter experts must review assessment content.

   12) Set the assessment instrument cut score consistent with the purpose of the credential and established standard of competence of the profession.

Essential elements:

•  Cut score must be set using information concerning the relationship between assessment performance and relevant criteria based on the standard of competence.

•  A report must be published documenting the methods and procedures used to establish the standard of competence and set the cut score, along with the results of these procedures.

   13) Document the psychometric procedures used to score, interpret and report assessment results.

Essential elements:

•  The certification program must describe procedures for scoring, interpreting, and reporting assessment results.

•  For responses scored by judgment, developers must document training materials and standards for training judges to an acceptable level of valid and reliable performance.   Any prerequisite background or experience for selection of judges must also be specified.

•  Candidates must be provided meaningful feedback and information on their performance on assessment instruments. Such information must enable failing candidates to benefit from the information, and if psychometrically defensible, understand their strengths and weaknesses as measured by the assessment instruments.

•  Reports of aggregate assessment data in summarized form must be made available to stakeholders without violating confidentiality obligations.

   14) Ensure that reported scores are sufficiently reliable for the intended purposes of the assessment instruments.

Essential element:

•  Certification programs must provide information to indicate whether scores are sufficiently reliable for their intended uses, including estimates of errors of measurement for the reported scores.

   15) Demonstrate that different forms of assessment instruments assess equivalent content and that candidates are not disadvantaged for taking a form of an assessment instrument that varies in difficulty from another form .  

Essential elements:

•  Equating or other procedures used to ensure equivalence and fairness must be documented including a rationale for the procedure used.

•  When assessment instruments are translated or adapted across culture, certification programs must describe the methods used in determining the adequacy of the translation or adaptation and demonstrate the information attained from adapted and source versions of the assessment instruments produce comparable test scores and inferences.  

   16) Develop and adhere to appropriate, standardized, and secure procedures for the development and administration of the assessment instruments.  

Essential element:

•  Assessment instruments must be administered securely using standardized procedures that have been specified by the certification program sponsor.  

   17) Establish and document policies and procedures for retaining all information and data required to provide evidence of validity and reliability of the assessment instrument.

   18) Establish and apply policies and procedures for secure retention of assessment results and scores of all candidates.

   19) Require periodic recertification and establish, publish, apply, and periodically review policies and procedures for recertification.

   20) Demonstrate that its recertification requirements measure or enhance the continued competence of certificants.

   21) Demonstrate continued compliance to maintain accreditation.  

Essential elements:

•  Annually complete and submit information requested on the current status of the certification agency and its programs

•  Report any change in purpose, structure, or activities of the certification program

•  Report any substantive change in examination administration procedures

•  Report any major change in examination techniques or in the scope or objectives of the examination

•  Submit any information it may require to investigate allegations of lack of compliance with NCAA Standards.

Full text of NCCA Standards available on the NOCA Web site at http://www.noca.org/docs/StndFeb02.pdf )