AECT Standard 4

Artifact 7: Formative Evaluation Plan

Context and Conditions

The Formative Evaluation Plan was created in EDET 722: Instructional Design and Assessment during my second semester in the Learning Design and Technologies master’s program. This formative evaluation plan was the final section of an instructional design document used to create a PowerPoint prototype of an instructional module on “Modeling Multiplication” for elementary students who receive special education services for math. The purpose of creating a formative evaluation plan is to evaluate the effectiveness of the instructional module to make improvements and adjustments as needed. Prior to the formative evaluation plan was a need assessment, goal analysis, learner analysis, contextual analysis, task analysis, and performance objectives and assessment items.

The first step in the creation of the formative evaluation plan was to describe the purpose, audience, and possible issues to consider. The audience includes the special education and general education teachers who work with the 3rd through 5th the module was intended for. Additionally, the audience included instructional coaches and administrators who work closely with those teachers in deciding what curriculum and instruction will best serve those learners’ needs. The possible issues identified were related to student achievement of learning objectives, the time it takes to complete the module, how students navigate and interact with the module, appropriateness of instructional level, and how it will impact students’ understanding of grade level content. The next step was to determine the necessary resources to conduct the evaluation, including the participants and the data collection instruments to be utilized. These instruments include a subject-matter expert notes form, observation notes form, interview protocol, attitude survey, and a posttest to gather student performance data.

The formative evaluation plan described what evidence each instrument is expected to provide. The next step described how the instruments would be utilized and how data would be gathered in phases, as well as how that data would then be analyzed and reported to the audience. It was decided that the data would be gathered into an evaluation report that would be distributed to the administrators, instructional coaches, and both the general and special education teachers. Additionally, an evaluation schedule was included, which specified information regarding what phases each data collecting instrument would be utilized, its purpose, its duration, and other required materials. Lastly, an appendix was included at the end of the formative evaluation plan with blank examples of the different data collection instruments to be used.

Scope

The purpose of the formative evaluation plan was to gain knowledge and skills in designing an evaluation plan and creating appropriate data collection instruments to identify areas of improvement for an instructional module. This formative evaluation plan was part of a larger design document that was administered within my school building for elementary students I provide special education services for in the area of math.

Role

This formative evaluation plan was completed as an individual project as I worked as the sole designer of the instructional design document, which included the evaluation plan. In addition to my knowledge of the content and experience as a special educator, my professor reviewed information and provided feedback and suggestions for improvement before creating the PowerPoint prototype that this formative evaluation plan was used for.

Instructional Design

This formative evaluation plan reflects the MR model by implementing evaluation instruments and focusing on aspects of formative evaluation. According to the MRK model, formative evaluation includes purpose, audience, issues, resources, evidence, data-gathering, and report (Morrison et al., 2013). The instruments were created to assess the learners’ mastery of the instructional module’s objectives (Morrison et al., 2013). The formative evaluation also reflects the evaluation phase of the ADDIE model as an instructional designer utilizes instruments to determine if objectives could be achieved and what adjustments could be made to improve the final product (Durak et al., 2016).

Related Performance Indicators

AECT Standard 4: Candidates design, develop, implement, and evaluate technology-rich learning environments within a supportive community of practice (Association for Educational Communications and Technology, 2012).

Indicators:

• Reflection on Practice - Candidates analyze and interpret data and artifacts and reflect on the effectiveness of the design, development and implementation of technology-supported instruction and learning to enhance their professional growth.

• Assessing/Evaluating - Candidates design and implement assessment and evaluation plans that align with learning goals and instructional activities.

• Ethics - Candidates demonstrate ethical behavior within the applicable cultural context during all aspects of their work and with respect for the diversity of learners in each setting.

Reflection

The creation of the formative evaluation plan taught me how to develop data collection instruments. Due to the fact it was created during my second semester in the program, I was unfamiliar with the components of an evaluation plan. As a special educator, I have always understood the importance of evaluation as I continuously monitor student data to determine if adjustments to instruction are needed. However, the process helped me to strengthen my analyzing skills when it comes to making data-driven decisions. Looking back at my data collection instruments, I would change the “Attitude Survey” if I were to recreate this plan. As I reflect back on it, it is too advanced for the students who it is intended for who have learning difficulties. Therefore, I would have had it focus on more age-appropriate language, using visual aids, and consider communication methods to ensure accessibility and understanding.

Throughout the Learning Design and Technologies master’s program, I continued to learn and improve on my ability to create design documentation reports that include formative evaluation plans. This experience taught me the objectives of an evaluation guide the utilization of data collection instruments, which provide the data for making recommendations for action and further improvements.

Artifact 8: Integrated Multimedia Project

Context and Conditions

The Integrated Multimedia Project (IMP) was created in EDET 603: Design & Development Tools I during my second semester in the Learning Design and Technologies master’s program. The purpose of the IMP was to work collaboratively with a group to design and develop an audio podcast episode, with a corresponding video podcast mini episode, a marketing poster, graphic cover art, and a website based on an educational topic.

The first step for this project was the IMP: Grand Idea, which described the topic, target audience, purpose, a description of the podcast and vodcast episodes, and how work would be distributed amongst the group members. The topic of this IMP was called “Operation ‘PD’ for First Years”. The overall theme was to provide advice and support for first year teachers or individuals who are interested in becoming educators. The description of the podcast and vodcast described the formatting and storytelling features. The podcast would be a live recording hosted by experienced educators sharing stories specific to grade level and content area while listeners could call in to ask questions. The vodcast would be pre-recorded using Google Meet as hosts would discuss their experiences as teachers amongst each other.

The next step was to create the IMP: Visual Plan and Outline to illustrate the sequence and structure of the various parts of the IMP. A general outline of the podcast episode was provided that included music clip ideas, sample dialogue, and a general idea on how to structure the episode, as well as a visual timeline and how content would be broken up. A visual storyboard was then provided for the vodcast that described the narrative, audio, visual elements, and included a visual timeline. Next, descriptions of the visuals for the marketing poster and cover art were provided. Lastly, a site map was designed for the website, which included a description of what tools would be used to design the site.

The final step for the IMP was to develop all the components of the IMP and create the website that includes those components. The “Operation ‘PD’ for First Years” website was created using Weebly. The web pages within the site included Home, About, Bios, Vodcast, Podcast, Connect, Resources. The “Home” page included a brief description of the most recent podcast available to listen and what time the next episode would air. The podcast was described in more detail in the “About” page and brief biographies about the hosts with individual contact information were provided on the “Bios” page. The podcast and vodcasts were uploaded to YouTube and links were provided under their designated pages for visitors to access. The “Connect” page included a comment box where listeners could provide their name and contact information to ask questions if they were unable to call in to ask during a podcast episode. Lastly, the “Resources” page provided links to articles where listeners could learn additional information helpful for first year teachers.

Scope

The purpose of the Integrated Multimedia Project was to work collaboratively to design and develop a quality podcast with corresponding marketing materials using various tools, such as graphics, voice, music, images, and motion video. This project allowed for the opportunity to work as a group to showcase the technical skills learned throughout the course and demonstrate instructional design skills in the process of planning the various project components.

Role

This Integrated Multimedia Project was created as a group project. I worked with two partners to contribute to the overall design and development of the various components of the project using individual knowledge and experiences as teachers to create a product that would support new or aspiring educators.

Instructional Design

The Integrated Multimedia Project relates to the MRK model as the designers used their different backgrounds and individual experiences as educators to identify what challenges or problems first year teachers may encounter that should be addressed and understand inexperienced teachers' needs. This reflects the identifying learner characteristics and needs stage of the MRK model as the group analyzed audience needs to plan for the design and development of the product (Morrison et al., 2013). Additionally, the IMP reflects the design and development phases of the ADDIE model as the group collaborated to plan how content would be presented and what materials would be used, and then developing the final products based on the design plan (Durak et al., 2016).

Related Performance Indicators

AECT Standard 4: Candidates design, develop, implement, and evaluate technology-rich learning environments within a supportive community of practice (Association for Educational Communications and Technology, 2012).

Indicators:

• Collaborative Practice - Candidates collaborate with their peers and subject matter experts to analyze learners, develop and design instruction, and evaluate its impact on learners.

• Leadership - Candidates lead their peers in designing and implementing technology-supported learning.

• Reflection on Practice - Candidates analyze and interpret data and artifacts and reflect on the effectiveness of the design, development and implementation of technology-supported instruction and learning to enhance their professional growth.

• Ethics - Candidates demonstrate ethical behavior within the applicable cultural context during all aspects of their work and with respect for the diversity of learners in each setting.

Reflection

The Integrated Multimedia Project during my second semester in the program was my first experience with editing audio and video content. These skills benefitted me throughout the remainder of the program as I frequently edited sound recordings, voiceovers, and video content, to embed those elements into projects. While the IMP did not require an evaluation prior to creating content, this is something I would incorporate if I were to recreate a similar project to better understand the target audience and their needs, establish clear goals, and ensure the content is relevant. The goal of this IMP was to provide advice and support to first year or aspiring teachers. Therefore, I think it would have been effective to conduct surveys with those individuals at the start to determine what content to provide.

Additionally, this my first experience working in a group for an assignment. This was a challenging experience as our schedules did not always align, which made it often difficult to discuss the work needed to be done. I held the leader position in the group as I was the one who would try to schedule times to meet and ensure everyone was completing their part. Through this, I gained strong leadership, interpersonal, and communication skills, which are essential skills that I will continuously use throughout my career as a teacher or instructional designer.