Frequently asked Questions About Assessment @ ISU

  1. What is assessment?
  2. How is assessment different from evaluation?
  3. Will assessment results be used for individual faculty evaluation?
  4. Why do we have to do this?
  5. What are the benefits of assessment?
  6. We already assign course grades. Isn’t that evidence of student learning?
  7. What are program educational objectives? What are student learning outcomes? Where can I find examples?
  8. What is direct assessment? Indirect? Which method should we use?
  9. What is an assessment plan? How do we get started?
  10. Where can we find support for our assessment efforts?
  11. When does this have to be done?
  12. What is TaskStream and how do I get access?

What is assessment?

Assessment is a very broad term that has different meanings in different contexts. In the context of higher education, the focus of assessment is on improving student learning. In general, assessment is a systematic, ongoing process designed to answer the question, “Can the program demonstrate that students have attained the desired learning outcomes?” This process includes establishing measurable outcomes for expected student learning, gathering, analyzing, and interpreting evidence to determine how well student achievement matches the expectations, and using the results to inform decisions about program modifications to improve learning.

“Assessment is commonly assumed to be about finding problems and then fixing them with new initiatives . . . .  Good assessments also identify effective practices already in place and guide educators in protecting and maintaining those practices . . . .  Identifying and protecting effective practices can be as important a function of assessment as identifying and improving areas of weakness" (Swing & Coogan, 2010, p. 7).

Assessment focuses on aggregate, program-level data, not on individual students or courses.

back to top...

How is assessment different from evaluation?

While the terms “assessment” and “evaluation” are sometimes used interchangeably, and there are similarities between the two processes, it is important to differentiate their purposes.

In general, in the context of higher education, assessment is an ongoing process for understanding and improving student learning. Assessment focuses on measuring student achievement of program-level learning outcomes. It is a process that identifies, collects, and prepares data to evaluate the attainment of student learning outcomes. Effective assessment uses relevant direct, indirect, quantitative, and qualitative measures as appropriate to the outcomes being measured. In general, when data are collected to improve learning, the process is assessment.

As a part of the assessment process, evaluation is the analysis and use of assessment data to make judgments about student performance in the aggregate and inform program improvements. Evaluation determines the extent to which student learning outcomes and program educational objectives are being met.

In this context, “assessment” refers to measuring and “evaluation” refers to using assessment information to make judgments.

As a broader concept, program evaluation refers to gathering evidence and making judgments about all of a program’s goals, not just those focused on student learning. In this context, evaluation is a process in which data are collected to make judgments of worth or value.

back to top...

Will assessment results be used for individual faculty evaluation?

Assessment of student learning focuses on what the program as a whole can do to improve student learning. When there is evidence that students are not achieving the expected outcomes, the faculty members in the program should collectively identify areas for improvement. Assessment results are not used to evaluate individual faculty. As clearly stated in the conceptual framework for assessment at ISU , “A primary purpose of assessment is the evaluation and the improvement of student learning, not the evaluation of faculty teaching or individual performance.”

back to top...

Why do we have to do this?

The primary purpose of assessment is to improve student learning. Assessment provides evidence for making adjustments to the curriculum and other educational experiences, with the goal of improving achievement of the student learning outcomes identified by each academic program. Assessment supports continuous improvement across the university, with the goal of improving student success in undergraduate and graduate programs.

In addition, assessment is a means for ISU to respond to the increasing external pressures for accountability by a variety of stakeholders. In response to these pressures, the university must provide evidence of each program’s assessment efforts, including the use of results to inform program improvements. In the broader view, assessment is not a fad that is going to go away. Calls for accountability for how public and private dollars are being spent are only going to increase, as will demands for institutions to demonstrate effectiveness in helping students learn and succeed in college and beyond. Assessment provides evidence of the quality and strengths of our programs and of our commitment to enhancing student success at ISU.

The faculty and departments of ISU, given their curricular role and responsibility, have primary responsibility for the development, implementation, and maintenance of all academic assessment activities. By engaging in an ongoing process of confirming and improving student learning, academic programs will additionally meet the accountability demands from the university’s stakeholders.

back to top...

What are the benefits of assessment?

Program assessment provides data to make informed decisions about program strengths and areas of needed improvement. Assessment answers questions such as, “Are our students learning the most important things we want them to learn? Are they learning what they need to succeed in future endeavors?” Assessment also helps answer questions such as, “If our students are not learning the important things, what are the barriers or stumbling blocks? What changes can we make (e.g., to the curriculum, pedagogy, use of technology) to help students learn more effectively?” Thus, individual programs and the institution as a whole benefit from a process of continuous improvement.

In addition to improving educational experiences, the assessment process can also benefit students by making expectations for program-level learning explicit. When students are aware of a program’s educational objectives and student learning outcomes, they are able to understand the curricular design and make decisions concerning how to focus their time and energy as they engage in the various educational experiences, both in the Foundational Studies program and in the major.

back to top...

We already assign course grades. Isn’t that evidence of student learning?

Course grades may be based in part on factors not directly related to learning outcomes, such as attendance or class participation, or the subtraction of points for an assignment that is submitted late. Therefore, it is almost impossible to determine what students know or are able to do by only looking at the grades for a course. Also, course grades are a composite of student success in meeting a faculty member’s expectations for a specific course, not an indication of student achievement of specific, program-level learning outcomes.

On the other hand, grades for specific assignments, exams, or other educational experiences that measure specific student learning outcomes can be useful information for program assessment. In many instances it is possible for graded course assignments to generate evidence concerning what students are learning as a group. For example, specific items on an exam could be linked to a specific learning outcome. The scores for those items can be used to represent learning achievement, which can serve as data for program assessment. Assessment of student presentations in a course could be used as evidence of achievement of the program’s identified outcomes for oral communication, as well as critical thinking, knowledge of the discipline, or other outcomes, based on the nature of the project and the scoring rubrics used to evaluate the presentation. Capstone courses, theses, portfolios, and internships are examples of culminating educational experiences or artifacts that can provide evidence of achievement of multiple program-level student learning outcomes.

back to top...

What are educational objectives? What are student learning outcomes? Where can I find examples?

Educational objectives are broad statements that describe what the program is preparing graduates to achieve. Educational objectives are derived from the program’s mission statement, are realistic, and are stated in general terms. Objectives are the overarching knowledge, skills, attitudes, and behaviors that the program targets with its educational experiences. They describe attributes that students should be able to demonstrate upon graduation or attain in their early careers. Objectives describe what we want our students to be, e.g., critical thinkers, when they graduate from ISU.

Student learning outcomes describe what students are expected to know and able to do by the time of graduation. They are linked to the program objectives and describe the specific knowledge, skills, behaviors, and attitudes that students should acquire as they progress through the program. Student learning outcomes are specific and measurable and are stated in terms of what the student should be able to demonstrate as a result of the learning experiences provided by the program. Outcomes are learner-centered, action oriented, and cognitively appropriate for the program level. Outcomes are measurable indicators of an educational objective. Effective outcomes are stated in the format of <one action verb> + <one something>. The action verb describes a specific performance (e.g., analyze, apply, design), and the “something” is the focus of the learning experience (e.g., steps of the design process, chemical reaction, scientific method).

Many ISU programs have posted their educational objectives and student learning outcomes on their websites. Examples of program educational objectives and student learning outcomes are also available here.

back to top...

What is direct assessment? Indirect? Which method should we use?

Direct assessments acquire evidence about student learning and the learning environment. Direct assessment methods provide for the direct examination or observation of student knowledge or skills. In direct assessment, students demonstrate learning. Examples of direct assessment are exams, portfolios, simulations, and observations.

Indirect assessments acquire evidence about how students feel about learning and their learning environment. Indirect measures of student learning ascertain the opinion or self-report of the extent or value of learning experiences. Examples of indirect assessments are surveys, questionnaires, interviews, and focus groups. Indirect assessment data are created when students reflect on their learning, rather than exhibit direct evidence of the attainment of learning outcomes.

All assessment options have advantages and disadvantages. The “ideal” methods are those that are a best fit between program needs, satisfactory validity, and affordability (time, effort, and money). In assessing achievement of student learning outcomes, it is crucial to triangulate – use multiple methods and sources of data to maximize validity and reduce bias of any one approach. Also, there must be a logical link between the assessment method and the learning outcomes. For example, assessing public speaking ability through a multiple-choice exam or by surveying students about their perceptions of their speaking abilities would be problematic.

back to top...

What is an assessment plan? How do we get started?

While the Assessment Council is not prescriptive in its approach to assessment, it has developed a general framework for assessment. This framework reflects best practices of other institutions, is informed by the literature on assessment, and provides the flexibility for programs to adapt the model to provide information that is useful to them. The Assessment and Accreditation Coordinator is available to assist you as you design and implement your assessment plan. A helpful template can be found here.

In general, the steps of assessment are:

  1. develop or refine the program’s mission statement so that it is consistent with the department, college, and university mission statements and communicates the program’s primary goals;
  2. articulate the (3-6) educational objectives of the program;
  3. identify the desired student learning outcomes (3-6) for each objective;
  4. specify the performance criteria for each outcome;
  5. map the curriculum and educational activities to the outcomes;
  6. gather evidence about how well students are meeting the outcomes, keeping in mind 1) that you don’t have to collect evidence on every outcome every year, and 2) that collecting data from a representative sample of students is acceptable; and
  7. use the evidence to make decisions about any modifications necessary to improve student learning.

No one assessment plan is right for every program at ISU. Using the general framework described above, program faculty should collaborate on developing and implementing an assessment plan that is aligned with the curriculum and student learning outcomes and provides information that is useful in making decisions about the program. The assessment plan should be evaluated periodically to ensure that it provides useful information.

back to top...

Where can we find support for our assessment efforts?

The Assessment Council and the Office of Assessment and Accreditation are here to support your assessment efforts. Available support includes:

  1. consultations on developing and implementing assessment plans, such as writing educational objectives and student learning outcomes, identifying appropriate measures, analyzing data, and reporting results, as well as on using TaskStream;
  2. custom workshops on assessment;
  3. an annual Assessment Day designed to enhance engagement in assessment;
  4. professional development opportunities for departmental A-Teams to broaden and deepen engagement in assessment;
  5. reviews of draft assessment plans and reports;
  6. mini-grants for assessment-related activities;
  7. financial support for costs associated with attending workshops and conferences centered on assessment; and
  8. dissemination of information on resources and best practices for assessment.

For further information or assistance, contact Dr. Ruth E. Cain, Assessment and Accreditation Coordinator, ruth.cain@indstate.edu or
237-8899.

back to top...

When does this have to be done?

Assessment is a process in which each program should continually be engaged. The ISU assessment model is based on an ongoing process of identifying program educational objectives and student learning outcomes, collecting and analyzing data, discussing results among program faculty, and making program modifications necessary for improving student success. Programs do not need to assess every outcome every year (unless required to do so by disciplinary accreditation requirements). Each program should have a plan for rotating through the outcomes on a regular schedule.

The Higher Learning Commission expects a report on each program’s progress on assessment by December 2013. Additionally, the Provost and the Board of Trustees expect a report from the ISU Assessment Council on the assessment progress of the academic programs annually in May.

back to top...

What is TaskStream and how do I get access?

Taskstream is a web-based software system that serves as a repository for each program’s assessment information. To set up a Taskstream account, or for information on using Taskstream, contact Dr. Ruth E. Cain, Assessment and Accreditation Coordinator, ruth.cain@indstate.edu or 237-8899. Training sessions and individual consultations are available to assist you in using Taskstream to facilitate and document your assessment processes.

back to top...