Step 9: Develop, pre-test and revise lessons, learning activities, materials

This step prepares the plans and materials needed to carry out the learning intervention.

Process

Use Tool #11 or Tool #12 Lesson Plan Formats to develop lesson plans.

1. Identify existing materials that you can use as references while developing or implementing the learning intervention. Research print materials and the Internet for useful references and resources. Check the following sources:
• Standards and guidelines at the national level first (if they exist and are up-to-date) and from international standard setting bodies, such as the World Health Organization
• Documents from medical professional associations
• Published curricula and journals
• Materials available from reputable websites
• Materials that have been used with similar job responsibilities or competencies and with similar learners, especially if these materials are available in adaptable formats.
Check with experts or national resource persons working with you to make sure the resources are up-to-date and appropriate to the situation. Often resources used earlier to identify the essential skills and knowledge can be used to develop the materials. Sometimes existing materials can be used either as they are, or with minor adaptations. However, if major adaptations are needed, it may be more practical to start from scratch. Even then, portions of existing materials can be used (e.g., a section from a book, illustrations or charts) or adapted (e.g., discussion questions, slides, case studies, role plays or checklists) if the copyright allows this. Be cautious about using materials that are not appropriate for the learners and their work setting and using activities that do not meet the requirements of your learning objectives.

2. Develop the lesson plans. A lesson plan is a set of instructions similar to a "road map" for conducting the learning intervention and enabling the learners to meet the learning objectives. Usually lesson plans will need to be developed specifically to meet the training requirements. The lesson plan should be written for the person who coordinates the learning intervention. In other words, if the approach calls for a trainer to lead or coordinate the intervention (e.g., group training), the lesson plan should be written for the trainer. Write the lesson plan for the learner if he/she is in charge of his/her own learning (e.g., self-directed learning). Some interventions may require a lesson plan or set of instructions for the instructor, the learners, a supervisor and a practicum advisor for the same set of coordinated activities (e.g., site-based training).
   
   The Experiential Learning Cycle can be used to structure the lesson plan in a way that encourages learners to apply what they learn when they are working. (See Box 18)
Typically, the lesson plan should include:

• An introduction that describes why the lesson is needed. The introduction should describe the job task and the skills and knowledge that are required to perform it. The introduction covers "what's in it for the learners."
• The learning objectives
• A schedule or list showing the order in which the activities should be done
• The assessment strategy for each objective
• A list of resources required and where they are located
• Description of advance preparation
• A detailed description of each activity (what the learners and trainers need to do to achieve the objectives)
• The time each activity should take.
In general, keep lesson plans short and format them so the main steps stand out. Learners and trainers are more likely to follow the plan if the lessons plan is short and easy to read. (See Tool 11 and Tool 12 for sample lesson plan formats)
3. Make a list and develop or obtain the learning materials that are needed for each activity and lesson. Remember that materials may be needed for trainers, learners, clinical preceptors, training coordinators, managers, supervisors and evaluators.
   
   Some of the materials that may be needed are:
• clinical guidelines, reference manual or study guide
• illustrations of diagrams
• pre-/post-tests, questionnaires, observation checklists
• a slide presentation for a lecture
• a computer-based simulation for skills practice
• role plays
• case studies
• job aids
• anatomical models, clinical equipment and supplies
• reporting tools and processes for inputing information into a database
• an orientation for facilitators, tutor/mentors and/or supervisors/managers (see Step 10)
• an action plan to help ensure transfer of learning (see Step 11)
• assessment and evaluation plans and related tools (detailing indicators, intended impact and how results will be used for decision making and subsequent planning) (see Step 12).
These materials may be organized, along with the Instructional Program Overview for the instructional strategy, into a learning package consisting of
• Reference manual
• Trainer/facilitator/supervisor guidelines
• Learner's guide
• Audiovisuals, equipment, supplies or anatomical models
• Supporting reference materials and job aids.
4. Develop the learning assessment instruments (skills checklists, written tests, questionnaires, etc.). Assessments should be written according to the plan for assessments made in Step 7.

Knowledge assessment tests

• Write at least one assessment item for each knowledge objective. More than one item may be necessary for learning objectives that are considered more important or cover more information than other objectives. Ensure that each item matches the learning objective and provides learners with the opportunity to meet the criteria necessary to demonstrate achievement of the objective.
• Sequencing: Once you have written your test items, there are generally two ways to sequence them. Group items:
  • that are similar in format--e.g., put all items of the same type (multiple choice, case study, true/false)together
  • based on the objective--e.g., all items that correspond to a particular set of objectives are together.
• Evaluating tests: In order to maximize resources, pilot test the assessment when you are pre-testing your learning materials. Observe pilot test participants and ask for their feedback about:
  • Clarity--Are the directions and items simple and easy to understand?
  • Realistic--Do the items ask for answers or observe behaviors that are reasonably close to the way the information or skill will be used on the job? Is it possible to observe the learner performing skills described in the objective? (e.g., are materials and equipment available; are clients or patients available at the time when the skill is to be observed?)
  • Difficulty--Do items require learners to use what they know? Can all learners who have learned the skill or knowledge get the item right?
  • Clear (not tricky) format--Do learners have to guess or use a process of elimination? Assessment items that include false statements must be written very carefully. They can easily become too tricky.
  
  Multiple choice questions can be tricky. When writing multiple choice questions, make sure all possible responses are reasonable. Avoid negative expressions and words like "except." (e.g., Which of the following answers is NOT a danger sign?)
  
  When reviewing questions, check carefully and consider rewording tricky questions or using another type of question. In some situations, a multiple choice question can be replaced by several True/False questions or a short answer question.
  
• Reliability and validity: Tests should be valid and reliable.
  
  Valid tests measure what they were designed to assess.
  
  Reliable tests are those that can be used repeatedly with different groups of learners by different evaluators and will consistently measure what they were designed to assess.
  
  Testing with content and evaluation experts can help determine validity. You can help determine reliability by pilot testing with a representative sample of your intended learners.
• Cut-off scores that determine whether learners pass or fail should be set based on how critical the skills and knowledge are to health and safety. National standards may also determine cut-off scores. When setting a cut-off score, make sure that the score is set to separate those who have critical skills and knowledge from those who have not yet achieved an adequate level of skills and knowledge. Eliminating unnecessary information or tricky questions helps make sure that a cut-off score is meaningful.
Skills assessment instruments
• Assessment of skills competency requires observation in real situations or simulation if there are no clients. Develop skills checklists to objectively observe and assess performance of skills or procedures.
• Ensure that skills checklists contain the sequenced steps required to perform a procedure in a standardized way, along with instructions and a rating scale to determine the level of competency on each step. Clinical experts must agree that the skills checklists represent the standard way of performing the procedure or are based on the national service protocols if available and up-to-date.
• Skills checklists with more detailed steps can also be used by learners as learning guides. They could also be used by preceptors during practicum training.
5. Get feedback on the draft materials and revise them.
   Reviewers can include subject matter experts, instructional designers/media specialists, people familiar with the learner group, potential learners and users, stakeholders and funding/sponsoring organizations. Develop processes and tools to collect targeted feedback from reviewers.
   
   Sometimes the changes suggested by internal reviewers are not appropriate for the intervention or the stated learning objectives. Be especially careful not to add learning activities and content that are not directly related to the learning objectives.
6. Pre-test the materials with persons who represent the intended users of the materials when a good draft is available.
• Plan how (develop tools) to collect user comments and suggestions about readability, usability and applicability.
• Select four of five persons who represent the user group. These may include: trainers, learners, supervisors and other stakeholders. Make sure the group represents variation in the learners the intervention is developed for.
• Carry out the activities and use materials the way final materials will be used, or as close as possible to it.
• Take detailed notes about difficulties, misunderstandings and comments.
• Use evaluation instruments to collect specific user comments and suggestions; ask users to complete questionnaires; interview users either individually or in a focus group using a set of questions that address particular indicators; explore and understand any suggestions that users share.
• Make changes suggested by the users. Consider user suggestions and criticism very carefully, checking with resource persons, experts and stakeholders as needed. Incorporate changes that are feasible and in keeping with the learning objectives. Sometimes users can help work through problems with the materials or the activities and help find practical solutions. Keep in mind that not all user suggestions are appropriate and feasible.

Box 18: The experiential learning cycle
Derived from evidence-based principles of learning, the Experiential Learning Cycle provides practical quidance about designing lessons. Training and learning that is designed in accordance with the Experiential Learning Cycle: 1) is linked to real life 2) encourages the learners to express their feelings and opinions and draw on their own prior knowledge and experience 3) integrates evaluation methods that provide immediate feedback to learners about their progress. The guidance in the Experiential Learning Cycle applies to both the intervention as a whole and to the individual lessons and activities. The principles described in the Experiential Learning Cycle are applicable regardless of the learning approaches used.
Step 1. Climate Setting/Introduction Stimulates interest and curiosity. Prompts learners to begin thinking about the subject that is being introduced. Helps learners understand why the subject is important to them, how it will be useful and what relevant experience and skills they bring to the learning intervention as a whole or to a specific learning activity. Information collected during Learning for Performance Step 3 is useful in tailoring learning activities to closely match the learners' interests and needs. Recognition of relevant experience, skills and accomplishments can be highly motivating for learners, especially when it's woven into subsequent elements of the learning experience.
Step 2. Objectives Tells the learners what they will be able to do as a result of participating in the learning intervention or activity. At this stage, learners should develop a clear understanding of how the learning objectives relate to performance expectations at the work site. Gives learners an opportunity to relate the objectives of the learning intervention or activity to their individual job requirements and work-site conditions. Links learning objectives to previous sessions.
Step 3. Interactive Presentation Presents content using relevant examples; poses questions to learners; supplements explanations with visual aids and summaries to highlight key points. Provides a framework for learners--either a theory or a model--that becomes the basis for the experience that follows.
Step 4. Experiencing Provides an opportuninty to encounter a situation derived from the objective of the training (e.g., skit/drama, role plays, case studies, critical incident, video, small group task/exercise, site/field visit using a checklist to observe a demonstration of procedures). Becomes the common source of learning that learners will share and is the event that will be analyzed during the rest of the lesson. Provides learners an opportunity to practice what they have learned in an actual or simulated work setting.
Step 5. Processing/Getting Immediate Reactions Solicits reactions from the learners about their individual experiences and challenges them to think about what they learned. Gives learners an opportunity to reflect on their accomplishments and receive feedback on their progress.
Step 6. Generalizing Learners link what they have learned to the session objectives. Learners identify key learning.
Step 7. Applying Using the insights and conclusions gained from the previous steps, the learners identify and share how: the learning applies to actual work situations they will use the learning in their work situations to close the performance gap. Encourages learners to develop and use an action plan and make specific arrangements for how new skills and knowledge will be used. Answers the learners' questions: "Now what?" and "How can I use what I learned?" Encourages the learners to consider the implications of what happens in their work situation if they do not effectively apply what they have learned, i.e., consequences of performance errors.
Step 8. Closure Summarizes the events of the learning intervention or activity. Links training events to job-related objectives and determines if objectives have been met. Links learning objective to the rest of learning intervention, especially upcoming sessions. Thanks learners for their participation and contribution. Ensures them of your availability for any other questions after the session.
Adapted from: Training Resources Group, Inc. and University Associates (see References and Resources)

Helpful Hints

• Ensure that the lesson plan, activities and materials:
  
  • Hold the learners' attention from the beginning and throughout
  • Use a variety of teaching methods
  • Are realistic. Opportunities to practice in realistic settings and to work with real-life problems make the lesson more relevant to the learners.
  • Give frequent feedback; evaluate progress regularly. Do not assume that "covering the subject" means everyone has learned it. Achieving the objectives will take some learners longer than others.
• Write/produce learning materials that are:
  
  • applicable to the stated objectives
  • current and accurate
  • easy to understand
  • well-organized
  • consistent and properly sequenced so that they are easy to use
  • appropriate for and appealing to the intended users.
  There are many published resources that provide design guidance. (See References and Resources)
  
• Materials development and duplication can be very expensive. Be creative about using existing materials (national procedures if available and up-to-date, international clinical procedures, other reference manuals). Do not create materials that are not essential. Examples:
  
  • Use an existing videotape of a clinical procedure with the sound turned off if the demonstration is correct but the language or narration is not appropriate. The trainer can narrate instead.
  • Have learners use plain paper rather than creating handouts for simple activities.
  • For information that may change, consider using word-processed documents duplicated in small numbers.
  • Create materials that can be adapted or updated locally. Avoid using computer software that is not available locally or that requires expertise that is not generally available locally.

Tools

11. Lesson Plan -- /
12. Lesson Plan (Alternative Format) -- /