Competency 1:

Conducting needs assessment and evaluating learning systems

Reflection on Mastery

Formal needs assessment does not come naturally to me. My instinct coming into this program was to pool a bunch of felt experts in a room and discuss actuals, optimals, and solutions. Time in this program as a whole has given me not only the awareness of the need for a formal, measurable process for needs assessment, but also the tools with which to implement them. The courses that were more impactful were

  • ISLT 9473 Project Management
  • ISLT 9474 Needs Assessment for Learning and Performance, and
  • ISLT 9455 Formative and Summative Evaluation

This still remains a challenge, not only for myself but also on a institutional level. Other people, such as other committee members and administration officials may not have the inclination, knowledge, or time to properly implement a formal needs assessment process.

As such, my challenges in this area are 3-fold.

  1. Resisting a natural inclination to trust my gut instead of using a formal, measurable needs assessment process
  2. learning how to on board participants (faculty, committee members) who might not value the process or understand its need
  3. continue to breed familiarity with the needs assessment process and build a library of literature and tools to reference

Evidence of Mastery

Please find below 3 artifacts along with corresponding reflections.

Artifact 1: Living Grace Church 2014 Needs Assessment Report, Final Project for IS LT 9474 Needs Assessment for Learning and Performance

Reflection

This needs assessment report was challenging because it was difficult to deliver. Uncomfortable problems were uncovered and needed to be communicated – and communicated to people I knew personally. This was an challenging assignment and I as forced to grow not only as a student, but also professionally.

For instance, some of the problems I found were related to how communication worked within the church. The lead pastor thought that was one of his strong suits and had tried to build that into the fabric of the organization. Additionally, the interviews uncovered a "poverty of time," where too few people were asked to do far too much, resulting in burnout and agitation towards leadership where enthusiasm and energy used to be present.

An important part of that was to trust the process and frame the results not as a personal critique, but as objective results designed to help the organization meet its stated optimals.

Artifact 2: Formative Evaluation Report, Final Project for ISLT 9455: Formative and Summative Evaluation

Reflection

For my final project for ISLT 9455, Formative and Summative Evaluation my group completed a formative evaluation of the ISLT 7314, Reference Sources and Services. Our goals were to to determine if the elements of the course and curriculum are meeting ALA competencies; and to determine if the elements of ISLT 7314 are accurately measuring what they aim to measure in student performance.

We used a variety of data collection methods, listed below:

  • Surveys of students enrolled in ISLT 7314 at the University of Missouri;  
  • Surveys of faculty teaching ISLT 7314 at the University of Missouri;  
  • Course Material Review; 
  • Course Mapping; and
  • Student Score Mapping. 

Ultimately, we recommended that to achieve the goals of the class the following 4 actions should be considered:

  1. rewrite the learning goals as 4-part SMART learning goals so that they progressively measure progress toward mastering the ALA learning goals;
  2. increase the number of low-stake opportunities for students to self-measure their progress;
  3. create holistic unit overviews that map out each week; and
  4. develop and communicate clear criteria of mastery of learning goals.

I really enjoyed this project, much more than the needs assessment that I reference in Artifact 1. It was both fulfilling and fun to dig in deep to a learning system and determine whether or not it was actually meeting the goals set out for it.

Most importantly, we were able to identify concrete, actionable possible improvements for the course. In my prior employment at Columbia College as a de facto instructional designer (official title was Course Review Specialist, but we paired with Instructional Technologists to develop and redevelop online courses) there was no formal assessment process. This was a welcomed change. In my short time at HGSAS, I tried to bring what I had learned in Formative and Summative Evaluation to the course review process. We ultimately opted for a Quality Matters process, but I was able to incorporate what I learned into my QM reports and in our review meetings with professors.

Artifact 3: Technology Recommendations at HGSAS, completed as part of my duties as an Instructional Designer at Hazelden Graduate School of Addiction Studies

Reflection

One of my essential functions at HGSAS was to recommend and support the technology needed to facilitate learning for the online degree program HGSAS was creating, a Masters of Arts in Addiction Studies to augment their in-seat degree programs. The artifact linked here represents a summary of my findings that was communicated to the director of online learning, the dean of the graduate school, and the provost. I worked in conjunction with faculty staff to identify the optimals and propose solutions to meet those needs.

Without the proper tools in place, clunky or ad-hoc solutions stand in that gap. This can easily lead to wasted time, loss of rapport between all parties (student / professor, professor / instructional support, student / school, etc.). In a startup learning environment working from a capital account, it is imperative to keep costs down while eliminating user friction. As such, selecting the proper tools becomes a necessary, albeit largely invisible to end-users, condition for educational success.

While this was not a part of my course work at MU, I did use the knowledge and attitudes formed from my classes at MU to put together these recommendations. Throughout this process I learned to adapt the theory and optimal approaches to situations to non-optimal time-frames.

Utilization

During my time at HGSAS, I constantly was having to integrate the needs assessment process into my work as an instructional designer. This was true on at least 3 different levels:

  1. At the course level - were the courses designed and functioning such that the students were mastering the (now measurable) course objectives?
  2. At the program level - was the program, when taken as a whole, accomplishing the ends we had set for it? This applied not only to things like graduation rates, but also to each of the instructional and non-instructional aspects of the program, such as wi-fi access policies.
  3. At the individual level - was what I was doing working? Was I being effective at completing my essential tasks in such a way that benefited HGSAS?

At MACC, not only have I been in the practice of running constant needs assessments on my own classes, I've been able to incorporate these processes to varying extents in my committee work.

Most recently there was a felt need on the distance education committee that we had to implement a mentoring program for our distance adjuncts. Quite quickly in the meeting it became apparent that nearly everyone there had differing views on what needs, and therefore what optimals, governed the problem. In response to that meeting I set up a subcommittee to look at that very issue and we were able to determine a set of needs, but we also were able to see that these needs were already being addressed by another mentoring program being developed at the college and that if we went forward, we would be duplicating their efforts, fragmenting the proper utilization of campus resources and personnel.

In the future, I hope to bring these skills and attitudes to my duties as an instructor, full time, faculty member, and active committee member. If I ever go back to instructional design, either in the instructional designer role, instructional technologist role, or in management, I hope to deploy needs assessment and evaluation to ensure the success of my educational institution.