JLab_logo

 

 Bevel: Click
for Word Document

GO TO:

 

Assessment Program Description

 

PROGRAM ELEMENTS

Integrated Assessment Scheduling (IAS)

·         Roles and Responsibilities

·         Process Steps

·         Timeline

Management Self-Assessment (MSA)

·         Roles and Responsibilities

·         Process Steps

·         Timeline

Independent Assessments (IA)

·         Roles and Responsibilities

·         Process Steps

·         Timeline

Other Assessments

Corrective Action Plan

 

DOCUMENT HISTORY

Revision Summary

Approvals

 

BACK TO:

Assessment Plans and Reports

Assessment Administration

 

 

Contact:

Bruce Lenzer,

QA/CI Assessment Specialist

Date:

03/01/2018

 

 

Rev:

0.4

 

PURPOSE

 

Jefferson Lab performs assessments as part of continuous improvement to identify, incorporate and subsequently affirm organizational and process effectiveness in terms of both positive performance and areas which are non-conforming or need improvement.  Assessments ensure management and operational activities are executed in accordance with established procedures based on contract and regulatory requirements.  However, the primary thrust of the program is designed to promote continuous improvement.

 

SCOPE

 

Jefferson Lab’s Assessment Program complies with the requirements outlined within:

 

·         JSA/DOE Contract DE-AC05-06OR23177 (which includes the Contract Clause H-50 “Implementing Contractor Assurance”)

·         Contract Clause I.88 Management Controls Contract

·         Contract Clause I.89 Performance Improvement and Collaboration

·         Jefferson Lab's Quality Assurance Program Description

·         Integrated Safety Management System (ISMS) Program Description

·         CAS Program Description & QA Program Description

·         JSA/DOE Contract DE-AC05-06OR23177 Part III, Section J, Appendix E Federal and State Laws and Regulations

Jefferson Lab uses a web-based Assessment Management System (AMS) to plan, schedule, and record the various assessment activities.  The system’s functions ensure consistency by:

 

·         Utilizing a requirements risk-based assessment planning method, risk rank requirements to determine priority assessment topics and subsequently plan and schedule them each fiscal year.

·         Ensuring consistent assessment performance and documentation methodology.

·         Maintaining assessment data for historic and comparative purposes. 

 

 

 

 

 

 

 

 

 

 

 

NOTE: Jefferson Lab provides training to individuals performing assessments and monitors their progress to ensure all assessments are conducted in a consistent manner.  On-line training can be accessed at https://www.jlab.org/div_dept/train/webbasedtraining.html  Assessment Basics Unit 1 and Unit 2 Instructor Lead course.  Only those who are appropriately trained and qualified may perform an assessments at Jefferson Lab.

 

 

OVERALL RESPONSIBILITIES

 

Chief Operating Officer (COO), Chief Financial Officer (CFO), Chief Information Officer (CIO), Associate Directors, Division Managers, Quality Assurance/Continuous Improvement (QA/CI) Manager, Division Safety Officers, Thomas Jefferson Site Office (TJSO), Jefferson Science Associates (JSA) Representatives, Subject Matter Experts; When requested:

·         Assign Lead Assessor (LA) or Point of Contact and various SME’s to assessment topics as appropriate.

·         Perform adequate review of topics and provide relevant justifications based on risk to ensure assessments reflect current laboratory operations.

·         Develop an effective Action Plan for Findings or Opportunities for Improvements (OFI) identified as a result of an assessment.  Assign appropriate personnel to ensure timely completion corrective actions and actions to implement improvements.

·         Approve/accept assessment results as appropriate.

QA/CI Assessment Program Manager (APM)

·         Manage the Jefferson Lab Assessment Program. Serve as mentor to lead assessors, team members and relevant individuals involved in the process or planning, performing and reporting results of Assessments.

·         Maintain the Assessment Management System, On-line help, Training, Program Description, and Procedures.

 

 

PROGRAM ELEMENTS

 

Requirement Risk Based Assessment Planning

Jefferson Lab applies a requirement risk-based approach to establish assessment frequency for programs associated with various contract requirements, regulations and DOE directives.   The intent is to focus assessment resources on those contract requirements and related programs where compliance or conformance failures would have the greatest impact on the laboratory in terms of environment, safety, health, quality, business or operations.

 

Criteria Determination for Management Self and Independent Assessments.

Jefferson Lab utilizes the “Criteria Review and Approach Document (CRAD)/ Lines of Inquiry (LOI) Methodology” to determine to what extent an assessment topic’s requirements have been met. 

 

CRAD’s – summarizes a program’s requirements and authorizing document(s) (e.g. Contract requirments, DOE Orders, and/or related Jefferson Lab Programs) in the form of specific Criterion.

§  Cites the specific Criterion surrounding requirements which must be met to demonstrate compliance or conformance.

LOI’s –questions or guidance to objectively determine if requirement criteria have been satisfied. 

 

Note: Using a Graded Approach, CRAD/LOI’s are required for Independent Assessments (IA), strongly recommended for Management Self Assessments (MSA), and optional for less formal reviews.

 

Reporting

Assessment results are summarized in definitive, objective, verifiable, and measurable report format.  Summarize compliance or conformance description in positive terms, capabilities, and noted exceptions.  All results are classified, or categorized, as follows:

 

Noteworthy Practice (i.e. “Findings” for Office of Science Reviews): Positive aspects of the topic being assessed that may be used as a model for other programs across the Lab or Office of Science facilities.

Opportunity for Improvement (OFI) (i.e. “Comments” for Office of Science Reviews): Deviation from best management practices, isolated or minor deviation from procedural requirements, or recommendations that would contribute to continuously improving the topical area of assessment.

Finding (i.e. “Recommendation” for Office of Science Reviews): Non-compliance with a requirement which must be considered and corrected.

 

Provide a summary of instances, conditions, or work evoluntions observed to demonstrate the validity and effectiveness of a program.  As an example: if a Machine Tool Assessment looked at 100 machines and found 10 to have non-compliant safety devices. However, from a positive perspective, this means that 90% were found compliant with requirements.

 

Report content is reviewed and approved by relevant management and appropriate Laboratory representatives.

 

Corrective Actions

Activities required to achieve compliance with CRAD and LOI’s, or would contribute to a program’s continuous improvement goals.  These are documented within a Corrective Action Plan (e-CAP) by the Event Owner.  Issues needing corrective actions are assigned an Issue Owner who monitors Corrective Action progress to completion.  This is done within the Jefferson Lab Corrective Action Tracking System (CATS).  See the Issues Management Program Description for addition information and guidance relative to e-CAP development and CATS action tracking.

 

Training

Jefferson Lab provides training to individuals performing assessments. On-line training can be accessed at https://www.jlab.org/div_dept/train/webbasedtraining.html  Auditor/Assessment Basics QA01kd Unit 1 and Instructor Led Unit II. Only those who are appropriately trained and qualified may perform assessments at Jefferson Lab. The APM and other experienced Assessors are available for coaching inexperienced or newly trained assessors.

 

 

INTEGRATED REQUIREMENT RISK BASED ASSESSMENT PLAN & SCHEDULE

The Jefferson Science Associates (JSA)/Jefferson Lab Integrated Requirement Risk Based Assessment Plan and Schedule is the annual fiscal year assessments, reviews, and audit plan and schedule which is agreed to with input from DOE TJSO.  Planning activities include:

·         Management Self-Assessments

·         Science Reviews (Note: Some are similar in nature to Independent Assessments. Eg. Accelerator Readiness Reviews)

·         Corporate Reviews

·         Independent Assessments (performed with independent internal resources or by external independent resources contracted by Jefferson Lab)

·         DOE planned assessments and reviews

·         Lab Director reviews

·         Audits performed by the JSA Internal Audit Department

·         Reviews, Critiques and other less formalized assessments in the form of a Review

 

The Integrated Requirement Risk Based Assessment Plan and Schedule is, at a minimum, reviewed annually by Jefferson Lab’s Associate Directors, Division Managers, DSO’s, ESH&Q Liaisons, JSA Corporate Audit, and JSA Corporate Leadership. The final Plan and Schedule is approved by the COO, Laboratory Director, and the TJSO.

 

Purpose

Jefferson Lab applies a requirement risk-based approach to establish assessment frequency for programs associated with various contract requirements, regulations, and DOE directives.  The intent is to focus assessment resources on those programs where compliance failures would have the greatest impact on the laboratory mission in terms of environment, safety, health, quality, business, operations, R&D, and science, Assessments ensure requirements are met and the organization complies with continual improvement as cited in contract Clause I.89 Performance Improvement & Collaboration.

 

Assessment frequency, based on final risk ranking score from Risk Registry is as follows:

 

Risk Score 1 - 4 = Low, Color Coded Green; No routine assessments

o   Assessments are planned as needed or mandated by contract, DOE Order, or other regulation.

Risk Score 5 - 12 = Moderate, Color Coded Yellow

o   Planned ever 5 – 8 years or as or mandated by contract, DOE Order, or other regulation.

Risk Score 15 – 25 = High, Color Coded Red; Consider for assessment.

o   Planned no less than 3 years, or as or mandated by contract, DOE Order, or other regulation.

o   Some higher risk requirements or topics may be planned annually regardless of mandate

 

Scope

Jefferson Lab’s Assessment Management System (AMS) maintains data from the integrated requirement risk-based assessment planning activities.  This data is used to generate the lab’s annual Integrated Risk Based Assessment Plan and Schedule.

 

The Integrated Risk Based Assessment Plan and Schedule is driven by contract requirement clause I.88 Management Controls, and flowed down to Jefferson Lab’s Quality Assurance Program Description.  The Integrated Risk Based Assessment Plan and Schedule depicts the assessments, reviews, and audits scheduled to be conducted by Jefferson Science Associates (JSA)/Jefferson Lab, JSA Corporate, DOE, and other entities during the upcoming fiscal year. 

 

Roles and Responsibilities

NOTE:  Management authority may be delegated at the discretion of the responsible manager.

 

APM

·         Maintain the AMS.

·         Prepare a proposed Integrated Assessment Plan annually  and Transmit for DOE input and final JSA approval.

·         Update draft Integrated Risk Based Assessment Plan as needed.

·         Finalize draft Integrated Risk Based Assessment Plan.

 

Subject Matter Experts (SMEs) & Associate Directors, Division Managers and Division Safety Officers

·         Annually review AMS data to assure all topics are identified and risk estimates are current.

·         Provide inputs to the Integrated Risk Based Assessment Plan.

 

ESH&Q Associate Director

·         Submit Integrated Risk Based Assessment Plan to COO and Lab Director for concurrence and final Approval.

 

Lab Director

·         Approve Integrated Risk Based Assessment Plan.

·         Formally transmit to TJSO.

 

Process Steps

 

Maintain the AMS

When

Who

Action

Periodically when necessary

APM

·         Review AMS data to ensure identified topics include contract requirements, operational, hazards, regulations, and DOE directives.

·         Ensure assessments address CAS risks based on the CAS Risk Registry.

·         Verify that periodic assessments are scheduled at frequencies based on requirements and risk(s).

·         Ensure all management self- assessments, independent assessments, reviews, audits, DOE assessments (requested by TJSO), and JSA internal audits (requested by JSA Representatives) which are required by the JSA Jefferson Lab Management and Operating contract, federal or state regulation are determined through the annual integrated risk based assessment plan, and others as may requested by management are entered into the AMS. 

Independent of risk rankings and priorities include “Discretionary” or opportunistic assessments into the AMS after they have been approved or confirmed. Discretionary or opportunistic assessments may result from notable events, assessment significant findings, recurrence trends evidenced by an assessment, non-compliances from occurrences, new implemented contract requirements, and/or from other review mechanisms such as Safety Warden Inspections, Safety Observations, Trend Analysis, etc. 

 

Prepare the IAS

When

Who

Action

Annually

(March thru August)

 

APM

·         As a result of inputs from AD’s, Division Managers, DSO’s, SME’s etc., determine planned Assessments utilizing the Integrated Risk Based Assessment Planning process and tools. Incorporate AMS data into the draft Annual Integrated Risk Based Assessment Plan.

APM

and

SMEs

·         Conduct meetings with division’s management, DSO’s, ESH&Q Liaisons, Corporate Auditor, and Corporate Liaison to develop the Annual Integrated Risk Base Assessment Plan. Planning includes early TJSO input if known, identification of recurring assessments, review of assessment topic history, consideration of contract and regulatory requirements, and related topical risks.

APM

·         Provide draft Annual Integrated Risk Based Assessment Plan to QA/CI Manager, ADs, Division Managers, CFO, CIO, CPO, COO and Division Safety Officers and JSA Representatives. Make changes, modifications, or updates as necessary.

APM

·         Transmit draft Annual Integrated Risk Based Assessment Plan to TJSO for review by first week July.

·         TJSO Review and supplies inputs for External Reviews, Joint Assessments, any addition Assessment topics.

·         Finalize Assessment Plan for transmittal to TJSO by the Lab Director.

 

Transmit

When

Who

Action

July

APM

·         Submit draft Annual Integrated Assessment Plan with TJSO inputs to ESH&Q AD for finalization.

August

ESH&Q AD & COO

·         Meet with COO and DOE TJSO Site Manager to discuss, resolve difference, and finalize agreed upon Annual Integrated Risk Based Assessment Plan.

·         Submit final Annual Integrated Assessment Plan to Laboratory Director and cover letter for transmittal to TJSO.

Mid-Sept.

Lab Director

·         Review and, if acceptable, approve final Annual Integrated Risk Based Assessment Plan

·         Transmit approved final Annual Integrated Risk Based Assessment Plan to TJSO.

On or before

October 1

QAPM

·         Notify division and department managers and JSA of Annual Integrated Assessment Plan approval.

·         Activate e-plan and e-report forms for first fiscal quarter.

 

 

MANAGEMENT SELF ASSESSMENT (MSA)

A Management Self-Assessment (MSA) is an assessment performed by individual(s) directly related to, or responsible for, the assessment topic.  The scope is generally limited to a single operation or process but may be broader depending on the purpose and scope.  Jefferson Lab uses an on-line Assessment Database to record and document performance of MSA’s including the assessment plan and report.  Refer to the on-line help Assessment e-Plan and e-Report for step-by-step instructions.

 

Purpose

MSAs are internal assessments generally performed by line management and their staff.  Team members include staff, SME’s, and/or sub-contractors engaged by Jefferson Lab.  MSAs are performed in accordance the Annual Integrated Assessment Plan and Schedule.  Noteworthy Practices, OFI, and Findings are recorded and a Corrective Action Plan (CAP) is initiated when required to ensure appropriate activities are scheduled to correct issues or improve processes.  The e-CAP form is used to generate corrective actions or actions to implement improvements and upon approved, transfers the contents of the e-CAP to The Corrective Action Tracking System (CATS). CATS is used to monitor and track to completion assigned corrective actions or actions to implement OFI's.

 

Scope

MSAs are a sampling of the organization's performance in a particular area of interest which may include conformance to requirements, as well as operational processes and practices.  MSAs are performed to collect data to ensure programs and processes, related to a specific Assessment, Audit, or Review topic, are being performed adequately and comply with requirements and to continually improve processes.

 

Roles and Responsibilities

NOTE:  Management authority may be delegated to a qualified employee at the discretion of the responsible manager. 

 

Assessment Program Manager (APM)

·      Assure Lead Assessor is trained, qualified, and has necessary resources required to perform an adequate assessment.

·      Verify the MSA e-Plan and e-Report.

·      Supply Lead Assessor or POC comments as needed on the e-report as well as reported results in the form of Findings and OFIs.

 

Lead Assessor (LA)

·      Obtain training (if not current or has not taken the on-line training).

·         Assessor should be an SME of the assessment topic and trained or experienced to Lead Assessments.

·         If not an SME, the Lead Assessor should recruit an SME to participate as a Co-Lead.

·      Develop the e-Plan for the MSA.

·      Incorporate comments from those involved or who will be interviewed regarding the MSA e-plan.

·      Coordinates logistics for performing the Assessment.

·         Schedules conference rooms for in brief, out brief, and interviews as needed.

·         Prepares data and information package for assessment team.

·         Prepares and conducts in brief, out brief, daily meetings, interviews etc.

·      Perform the MSA.

·      Prepare and complete the MSA e-Report.

 

Point of Contact (POC)

·      Similar as to Lead, but more coordination is needed to ensuring assessment e-plan and e-report information is completed in the AMS.

·      Understand the JSA Jefferson Lab Assessment Program and Process, obtain Auditor/Assessment Basics Training (if not current or has not taken the on-line training).

·      Coordinates Assessments with internal and external entities.

·      Facilitates obtaining an Assessment Plan from internal or external assessor.

·         Provides external Assessor the Word Assessment Plan and Report template.

·         Translates the Word Assessment Plan and Report into the Assessment System e-Plan and e-Report for the topical assessment being performed.

·         Attaches any documents as needed to the e-plan and e-report.

·         Submits the e-plan and e-report for e-signatures.

 

Associate Directors, Division Managers, QA/CI Manager, CFO, CIO, CPO, JSA Representatives, Subject Matter Experts when requested:

·      Review the MSA e-report and supply comments.

·      E-sign e-reports as required.

 

Process Steps

 

Plan

When

Who

Action

Within the appropriate fiscal quarter

APM

Approximately, two weeks prior to the start of a fiscal quarter:

·        Initiate MSA e-Plan template via the AMS.

·        Transmit MSA e-Plan template to Lead Assessor (LA) or POC.

LA

Within the fiscal quarter:

·        Complete e-Plan template:

o   State the purpose or reason why the assessment is being performed and a general scope of the effort.

o   Describe the requirements against which the assessment will be made. These could include DOE orders, ES&H Manual chapters, work control documents (OSPs, TOSPs), division or department goals, DOE/SURA contract requirements, metrics, etc.

o   Identify Team Members, personnel to interview, and documentation to review.

o   Identify previous assessments and corrective actions to ensure there are no recurrences requiring root cause analysis.

o   Define the specific areas, procedures, processes, etc. that will be assessed.  Note any special requirements or limitations.

o   Add Criteria Review and Approach Document Criterion with Lines of Inquiry as appropriate.

APM

·        Review and concur with e-signature the MSA e-Plan (This will generate a MSA e-Report Template and email notifications sent to LA.)

 

Perform

When

Who

Action

Within the fiscal quarter or

Three months after receipt of notification

LA

·         Perform the Assessment.

·         Complete MSA e-Report template as appropriate to determine to what degree Criteria and Lines of Inquiry have been "Met."  Include a compliance description to justify determinations.  Activities include:

o   Interview affected personnel.

o   Observe work performance.

o   Verify and review documents and records.

o   Perform physical inspections.

·         Determine and summarize results including Findings, OFIs and Noteworthy Practices. This includes a focus on positives not just any nonconformances or non-compliances.

·         If applicable, complete a requirements health matrix to demonstrate both the areas where performance to meet requirements in acceptable, where partically met compliance exists, and/or where non-conformances or non-compliances in the form of a Finding were discovered.

·         State the summary conclusion and assessment effectiveness of the topical area assessed.

·         When applicable, discuss the implementation of previous corrective or preventive actions in the assessment effectiveness statement. 

·         Make an overall decision regarding having met, partially met, or not met performance of the requirement(s) being assessed.

·         Prepare a summary matrix in the form a visual aide to depict over effectiveness of requirements versus elements assessed against the requirements.

·         Confirm and submit MSA e-Report for review and approvals.

·         By review of historical assessments, if recurring issues are discovered, a corresponding finding must be included in the e-report and include an action to conduct causal analysis as a part of the post assessment Corrective Action Plan.

 

Verify

When

Who

Action

Within 45 days of Assessment Completion

APM

Associate Directors, Division Managers, QA/CI Manager, CFO, CIO, CPO, JSA Representatives, ESH&Q Manager

ESH&Q AD

·         Review, provide comments, and, if acceptable, approve (e-sign) MSA e-Report.

Assessment System

·       Flags MSA e-Report as complete in the assessment database.

 

*Time frames indicated are "typical." Exceptions are considered for assessments which require more rigorous or encompass a larger in scope.

 

 

INDEPENDENT ASSESSMENTS (IA)

An Independent Assessment (IA) is performed by qualified individuals outside the organization (or process).  Individuals are selected based on their particular expertise of the subject matter, but are not directly responsible for the process.  IAs can be performed by the Thomas Jefferson Site Office, other DOE organizations, or outside 3rd parties at the invitation of Jefferson Lab, Jefferson Lab QA/CI Assessors, or JSA Corporate Auditor. Under no circumstances should a topical assessment, process, or procedure owner be leading an IA nor be part of the Assessment team as this constitutes a conflict of interest.  Depending on an assessment topic, the scope of an Independent Assessment is typically a total program and often involves many departments. However, in some cases, a topical assessment may be a subset or specific sub element of a requirement. Jefferson Lab uses an on-line Assessment Database to record and document assessment activities.  Refer to the on-line help Assessment e-Plan and e-Report for step-by-step instructions.  However, in the event a 3rd party lead assessor is engaged, they should use the Jefferson Lab Plan-Report Template to document the assessment activities. This form can be found at https://jlabdoc.jlab.org/docushare/dsweb/Get/Document-141817/*.* . For consistency, independent 3rd party lead assessors are requested to follow the Jefferson Lab process as much as reasonably possible.

 

NOTE: An IA performed jointly with the Thomas Jefferson Site Office (TJSO) is processed in accordance with the Lead Assessor's organization.  If Jefferson Lab is the lead assessor, it complies with this program. 

 

Purpose and Scope

IAs are more formal and generally encompass a broad topic, or whole program.  They may involve Jefferson Lab staff as subject matter specialists and other resources as may be necessary to ensure the effectiveness of the assessment.  IAs are performed in accordance with a documented plan to ensure accuracy.  Findings are documented, tracked to completion, and recommended corrective actions are reviewed by others for relevancy.  The Corrective Action Tracking Tool is utilized to assign, monitor, and ensure completion of corrective actions and actions to implement OFI.

 

Roles and Responsibilities

NOTE:  Management authority may be delegated to a qualified employee at the discretion of the responsible manager.

 

QA/CI Assessment Program Manager (APM)

·          Review and concur with e-Plans and e-Reports.

 

Lead Assessor (LA)

·      Obtain training (if not current or has not taken the on-line training).

·         Assessor is preferably an SME of the assessment topic and trained or experienced to Lead Assessments.

·         If the individual assigned as a “Lead Assessor” is not an SME, another team member who is an expert of the topic, requirement, and/or subject being assessed must serve as an assessor on the team.

·         Some aspects of an IA may be delegated to one or more assistant assessors and/or an administrative assistant.

·      Develop the e-Plan for the MSA.

·         Using the e-plan in the Assessment Management System for the particular assessment or for external assessors, use the forms and process supplied by the APM.

·      Incorporate comments from those involved or who will be interviewed regarding the MSA e-plan.

·      Coordinates logistics for performing the Assessment.

·         Schedules conference rooms for in brief, out brief, and interviews as needed.

·         Prepares data and information package for assessment team.

·         Prepares and conducts in brief, out brief, daily meetings, interviews etc.

·      Perform the MSA.

·      Prepare and complete the MSA e-Report.

 

Point of Contact (POC)

·         Similar as to Lead, but more coordination as needed and ensuring assessment e-plan and e-report information is completed in the AMS.

·      To understand the JSA Jefferson Lab Assessment Program and Process, obtain Auditor/Assessment Basics Training (if not current or has not taken the on-line training).

·      Coordinates Assessments with internal and external entities.

·      Facilitates obtaining an Assessment Plan from internal or external assessor.

·         Provides external Assessor the Word Assessment Plan and Report template https://jlabdoc.jlab.org/docushare/dsweb/Get/Document-141817/*.*.

·         Translates the Word Assessment Plan and Report into the Assessment System e-Plan and e-Report for the topical assessment being performed.

·         Attaches any documents as needed to the e-plan and e-report.

·         Submits the e-plan and e-report for e-signatures.

 

 

Associate Directors, Division Managers, QA/CI Manager, CFO, CIO, CPO, JSA Representatives, Laboratory Director When requested:

·         Verify IA e-Report for completeness and accuracy.

·         Approve e-Report with electronic signature.

 

Process Steps

 

Plan

When

Who

Action

Typically 30 days prior to start

APM

·        Transmit* IA e-Plan Templates to Lead Assessors (LA).

*see note in bold block in the LA actions section.

If the IA involves interfacing with JSA vendors and suppliers, ensure both the Subcontracting Officer’s Technical Representative (SOTR) and the Subcontracting Officer from Procurement are notified and involved with assessment activities.

LA

or

POC

·        Complete the IA e-Plan template or manual form as appropriate.

o   State the purpose or reason why the assessment is being performed and a general scope of the effort.

o   Describe the requirements against which the assessment will be made. These could include Contract Requirements, DOE orders, ES&H Manual chapters, work control documents (OSPs, TOSPs), division or department goals, DOE/SURA contract metrics, etc.

o   Identify Team Members*[1], personnel to interview, and documentation to review.

o   Identify previous assessments and corrective actions to ensure there are no recurrences requiring root cause analysis

o   Define the specific areas, procedures, processes, etc. that will be assessed.  Note any special requirements or limitations. Add Criteria Review and Approach Documents with Lines of Inquiry as appropriate.

·        Submit the e-Plan or manual hardcopy plan to team members, individuals being assessed, and interviewees as well as other as defined in the Plan.

·        Offer 5 days for review comments and or feedback.

*Team Members are selected for their experience and knowledge of the subject matter.  They are functionally independent of the assessed organization.

 

If members are not certified auditors/assessors, training is obtained by completing Jefferson Lab’s QA01kd course.

APM

·        Review and concur with IA e-Plan or manual plan document, if approved, Assessment System is set to generate the IA e-Report Template and auto notify the LA. If an assessment involves an external assessor the report may be done using the JSA manual Word report template at https://jlabdoc.jlab.org/docushare/dsweb/Get/Document-141817/*.*. Note: On a case by case basis, other report documents and formats used by the independent assessor may be accepted and uploaded to the AMS e-report. This function may be performed by the LA, POC, or APM

 

Perform

When

Who

Action

Prior to Start

LA

or

POC

·        Coordinate Activities:

o   Reserve meeting rooms for interviews, daily Team Briefings, and In & Out-Briefs.

o   Schedule observation tours, inspections and site/area visits.

·        Transmit IA schedule to Interviewees, copy to assessed organization’s management, DSC, and DSOs include:

o   In & Out-Brief details.

o   Attach copy of IA Plan & CRAD/LOIs.

·        Provide meeting details to APM.

 

NOTE:  If Lead Assessor is external (or a 3rd party to Jefferson Lab), these responsibilities revert to APM, POC, the organization sponsoring the assessment, or a designee. Documents, and other information, may be submitted to team members prior to the assessment date.

Typically two weeks

LA

·         Complete IA e-Report, or other report document/form, as appropriate to determine to what degree Lines of Inquiry have been "Met."  Include a compliance description to justify determinations.  Activities include:

o   Interview affected personnel.

o   Observe work performance.

o   Verify and review documents and records.

o   Perform physical inspections.

·         Determine results including Findings, OFIs and Noteworthy Practices.

·         Focus summary on the positives, not just a non-conformance ornon-compliance.

·         For Findings, a statement of what requirement is not met is to be captured accordingly.

·         Complete a requirements health matrix to demonstrate both the areas where performance to meet requirements in acceptable, where partically met compliance exists, and/or where non-conformances or non-compliances in the form of a Finding were discovered.

·         State the conclusion on effectiveness of the area or activity assessed.

·         Make an overall decision regarding having met, partially met, or not met performance of the requirement(s) being assessed.

·         When applicable, discuss the implementation of previous corrective or preventive actions in the assessment effectiveness statement. 

Within two weeks of completion

LA

or

POC

·         Review IA e-Report, or other report document/form, for accuracy.  Disposition and/or incorporate comments. Confirm or clarify discrepancies.

·         Confirm and submit IA e-Report, or other report document/form for acceptance.

 

Verify

When

Who

Action

Typically three weeks

Associate Directors, Division Managers, QA/CI Manager, CFO, CIO, CPO, Laboratory Director, JSA Representatives

·         Review and, if acceptable, approve IA e-Report.

Assessment System

·         Flags Assessment Record as Fully Complete in the Assessment Database. E-Report is locked. An e-CAP is generated and automatic email notification sent to the Event Owner for e-CAP completion. See e-CAP process for details to process the e-CAP.

 

*Time frames indicated are "typical." Exceptions are considered for assessments which require more rigorous or encompass a larger in scope.

 

 

OTHER ASSESSMENTS

 

Other types of assessments have specific protocols and follow requirements documented within the responsible organization’s programs.  Jefferson Lab’s QA/CI Department is the point of contact and ensures these assessments are compliant and conform with Assessment process expectations

 

Other assessments included, but are not limited to:

·         Review:

o   General category that captures less formal types of assessments or recurring assessments involving a checklist (e.g. annual OSHA reviews)

·         JSA Corporate Audit:

o   Similar to Independent Assessments but more detailed.

·         External Review:

o   Performed by representatives from the TJSO, Inspector General, State Department of Environmental Quality, Hamtpon Roads Sanitation District etc.

·         Readiness Review:

o   Equipment Installation or Production Readiness Reviews

o   Accelerator Readiness Reviews (ARR)

·         Scientific Assessments or Reviews*

·         Critique:

o   Emergency Management Drills and Exercises

·         QA Vendor Evaluation:

o   Evaluates a prospective Vendor’s QA Programs

·         Effectiveness Review

o   Examines the effectiveness of corrective actions from a previous assessment, audit, review, or notable envent to ensure actions are working as intended and to prevent recurrence of non-conformance or non-compliance. Effectiveness reviews are performed by the QA/CI Department or other independent Subject Matter Experts.

 

*Jefferson Lab’s Assessment Database recognizes the following reporting results which correspond to the following determinations within Scientific Assessments or Reviews:

 

Jefferson Lab

Scientific Assessments or Reviews

 

Noteworthy Practices

Findings or Statements of fact as found

Opportunities for Improvement

Comments or Optional Considerations

Findings

Recommendations or Actions Required

 

NOTE:  The Integrated Risk Based Assessment Plan does not include safety warden inspections, work observations, and other similar, less-formal assessments.  These are separate mechanisms performed within overall program requirements specific to ESH

 

 

CORRECTIVE ACTION PLAN

A Corrective Action is an activity that corrects a finding (deficiency or non-compliance) and restores the service, component, or process to a state of compliance with specifications, procedures, or regulatory requirements.  When an assessment identifies a “finding” an electronic Corrective Action Plan (e-CAP) is developed.  When an OFI is discovered corrective actions are also indicated in the e-CAP and, if accepted, tracked in the Jefferson Lab CATS System.

 

Purpose

An electronic Corrective Action Plan (e-CAP) is initiated when an assessment identifies a finding that requires action to bring a program or process into compliance with a requirement.  An e-CAP may also include OFI’s if an action is required to implement the improvement. 

 

Responsibilities

Event Owner when requested:

·         Develop e-CAP for Findings and OFIs resulting from an assessment. 

·         Submit e-CAP to LA.

 

Lead Assessor

·         Review, and if acceptable, submit the e-CAP for approval.

 

CATS Administrator

·         Review, and if acceptable approve the e-CAP for submission into the Jefferson Lab CATS.

 

Process Step

Upon approval of an Assessment Report, if there are Findings or OFIs an e-CAP is initiated and a link is transmitted to the Event Owner. 

 

Develop e-CAP

Determine cause codes for all Issues resulting from an Assessment. Formal causal analyses are to be conducted for all risk code 3 and 4 Issues as well as TJSO identified Priority 1 or 2 Issues.  The supporting documentation for the causal analysis is to be attached to the e-CAP for these cases. Causual Analysis techniques such as 5 Why’s,  Ishikawa Fishbone Diagramming, or Taproot are recommended tools for determining root cause.

 

Determine appropriate corrective actions for identified issues, assign owners, and estimate due dates for completion. 

 

Review and Approve

Review the CAP to ensure actions noted will actually rectify the issues or successfully implement the OFI’s.

 

IF

THEN

Acceptable

Approve the e-CAP

(This will auto populate a corrective action entry in the CATS)

Unacceptable

Contact Event Owner.

Discuss concerns and ensure edits are incorporated as appropriate. 

(Use the CATS process to arbitrate irreconcilable differences.)

 

 

REVISION SUMMARY

 

Rev #

Revision Description

Effective Date

0.4

Stategic Iniatives agenda items included:

*Alignment of Risk Registry risk levels for assessment frequency.

*Addition of including a Requirement Health Matrix on certain assessments (IA’s).

* Expectation for Assessors to highlight or focus on positive requirement performance.

Expectation of Assessors to include an overall decision if the requirements assessed are met, partially met, or not met.

* Expectations for assessors to be qualified through training and be an SME. If not an SME, they must have a co-Lead who is an expert in the topic, requirement, or subject being assessed.

03/01/2018

0.3

IAW CATS MOA-2017-14-04-01, modified Corrective Action Plan process steps to include the requirement for cause codes for all Issues resulting from an Assessment, with supporting documentation required for RC3, RC4 and TJSO P1 and P2 Issues.  Supplied recommended tools for performing causal analysis.

Other editorials to align with other changes to clarify duties and process based on comments from the COO such as:

*Links to the Word Plan and Report Form for use by external assessors.

*Listed and clarified duties of a Point of Contact (POC).

*Clarified duties of Lead Assessor including review of historical assessments for recurring issues that may trigger action to perform causal analysis.

* Expectation for LA to summarize an IA assessment performance and effectiveness with a visual aide as part of the out brief and final report.

 

12/01/2017

0.2

Combined “Risk Based Assessment Planning” and “Integrated Assessment Scheduling” into one “Integrated Risk Based Assessment Planning and Scheduling.”  Added the step to identify previous assessments and corrective actions to ensure root cause analysis is performed for recurrences. Added the CATS Administrator as the approver of the E-CAP Form.

07/20/2017

0.1

Update verbiage: “Assessment Owner” to “Event Owner” to reflect “Issues and Corrective Actions Management Procedure,” and “Corrective Action Tracking System Program.”  Clarified that Lead Assessor provides review and feedback for CAP, not approval.

05/05/2017

0

Update from original program description and procedure documents to reflect current laboratory operations which include a web-based program for scheduling, performing, and recording of assessment data.  This revision includes relevant content from:

05/10/2016

 

 

APPROVALS

 

Approved by:

                                                                 Signature:

Date:

Document Custodian

Bruce Lenzer      original signature in QACI File X7333

05/10/2016

QACI Manager

Stephen Smith     original signature in QACI File X7333

05/10/2016

ESH&Q AD

Mary Logue     original signature in QACI File X7333

05/10/2016

 

This document is controlled as an on line file.  It may be printed but the print copy is not a controlled document.  It is the user’s responsibility to ensure that the document is the same revision as the current on line file.  This copy was printed on 8/10/2018.

 



[1] Required actions:

If IA Team Members are subcontractors or from other DOE facilities (otherwise known as External Team Members):

o  Create Statement of Work (SOW) and Purchase Requisition (PR) in Procurement System requesting use of external team members.

o  Coordinate travel arrangements with external team members.

o  Contact Jefferson Lab Human Resources to arrange for registration.

o  Contact Jefferson Lab Staff Services to reserve conference rooms and laptops for team members’ use during the assessment.