Office of Academic Programs

  1. Welcome
  2. Blackboard Resources
  3. Other Technologies
  4. Free Resources
  5. Learning Online
  6. Teaching & Learning
  7. Policies
  8. HHS

Online Assessment

Online Testing Issues

Models for Evaluation of Online Courses


Online Testing Issues


Advantages of doing your tests (or surveys) on Blackboard

•  It's a time saver. If you are giving a completely objective exam, Bb will grade it automatically and post that grade in your course gradebook.
     •  If your exam is mostly objective, Bb will grade the objective questions but you will have to manually grade the subjective questions before the grade will display in the gradebook.
     •  Note: Avoid the fill in the blank question. It is difficult to guess all the possible answers ahead of time. It is usually more trouble than it's worth. Short answer questions must be manually graded as well.
•  Blackboard aggregates the data for each question. There are no names attached so it is all anonymous. You can display this data for the class and have a clarifying discussion which provides another opportunity for learning.
     •  For objective questions, it is possible to see what percentage selected each answer choice. This can help with identifying questions that turned out to be poor.
     •  Blackboard will aggregate all answers to subjective questions as well. Essay responses may be graded anonymously if desired.
•  It saves paper and storage space. An archive of the exam data is maintained as long as the courses are on the server, which is normally two years. This should be sufficient to deal with grade appeals and incompletes.
•  It's convenient for commuting students. Students can take the exam wherever they are at a convenient time for them during the window you designate.


Advantages of using a UNCG computer lab to administer a Blackboard test.

•   You can proctor students in this fashion by standing in the back of the lab and watching their computer screens.
•  A technology glitch is less likely and if it occurs, it is easier to troubleshoot. Normally, there is a lab assistant to help as well.
•  If something goes wrong or a student has a question, you are right there to answer it.


Using the UNCG wireless to administer a Blackboard test.

•  This is a challenging issue at present. If you are using the HHP wireless laptop cart, you should be fine. If, however, you are using many more computers with one or two wireless access points, it is possible that there may be glitches such as a test not loading entirely or freezing up. If you use this method, have small groups of student login at a time.


What to tell your students

Sheet with 4 sets of Blackoboard test tips for students. (Feel free to make copies, cut them up with a paper cutter, and distribute to students.)

If you provide these tips to students ahead of time and/or put them in the test description and/or instructions fields which appear with the link to the test and after opening the test, respectively, all will probably experience fewer problems and you will get more accurate test results due to lower student stress.
Recommendation: give students a practice test before administering a real exam so they will be accustomed to the interface.

•  The number of questions, if there are essays, and how long they have to take the test. Tell them that they will not be shut out at the time limit, but that you will be able to tell if they went over time because Bb puts an ! in the gradebook rather than a grade. (If you have essay questions in the exam, there will already be an ! .)
•  The time elapsed displays in upper right header of the exam. This should help students to keep track. Just in case, encourage them to have a time device as a backup. Note! The time elapsed does not display in Mozilla Firefox.
•  When they open the exam, tell them to scroll down to be sure that the entire test has loaded before they save any answers.
•  Tell students to click the down arrow next to Question Completion Status (just above the first question). When they save a question, a little icon appears over the question number. This enables students to quickly identify the questions they have not answered. Each question number is a hot link so they can navigate to remaining questions quickly.
•  Tell students if you have chosen the test setting Force Completion. If you have, it is crucial that they not click the browser Back button because it will lock them out of the exam. Should this happen, your alternatives are to uncheck force completion until they log back in or to reset the exam (in which case any saved answers are lost). If you are getting the idea that this could be a headache, you might be correct.
•  Note: if you do not select Force Completion, students may login and out multiple times during the window of time you have given them to enter the exam. As you may recall, if a student goes overtime, an ! displays in their gradebook cell. However, in Blackboard 6.3, this did not happen if a student went overtime and logged in and out multiple times. It was necessary to access the test in order to ascertain whether a student went overtime. I have not yet had a chance to check this in version 7.0.


Discouraging cheating

There are multiple ways to discourage cheating in the online environment.
•  Create test banks or pools of questions. This enables you to draw random blocks of questions so that each student gets a different version of a test.
     •  There will probably be overlap in the questions. This random block spreadsheet enables you to calculate the chances of an overlap in questions. All you need is the number of questions in your bank and the number of questions you plan to use on an exam.
     •  You might want to create a question pool for each unit of instruction so that each version of an exam is a fair representation of your objectives.
     •  Note: if you use random blocks, Blackboard cannot computer aggregate test question data.
•  Create multiple exams. (Thanks to Sharon Morrison and Rachel Dirito for this idea.)
     •  If you have a large class, consider creating two completely different exams. Order the questions in either chronological order of presentation or by topic. Then, take the first half of the first exam and put it together with the second half of the second exam. Put the first half of the second exam with the second half of the first exam and you have four varying exams.
     •  The only problem with this strategy is that it will create 4 columns in your gradebook. To overcome this problem, set the columns for the four exams so that they do not display to students and are not part of the final grade calculation. Then, create a new column that is available and part of the final grade calculation and copy the grades for the exams into it.
•  In the Modify Test Options (access by clicking modify the exam where it is deployed), under 4. Test Presentation, click Randomize Questions.
•  In the test Creation Settings (access by clicking modify the exam under Test Manager), under 5. Display, leave checked "Specify random ordering of answers." This means that the answer to question 1 may be a. for one student but b. for another.
•  Recommendation: explain to students what methods you have used to create the exam. This may help students understand that it will not be easy to cheat and they might be less likely to try to cheat and more likely to prepare for the exam adequately. Your goal is to further their learning, not to catch them in a sting. The academic integrity process is important but it is also painful and time consuming.

Top

Models for Evaluation of Online Courses

Most of these models have to do with what a course looks like, as opposed to its implementation. What is lacking is tracking and characterizing the interaction between student and instructor, student and student, and, to some extent, student and learning materials. As well, there is little coverage of the effectiveness of learning activities.

Online Course Evaluation Project (OCEP) from the Monterey Institute for Technology and Education
"The Online Course Evaluation Project (OCEP) identifies and evaluates existing online courses in higher education, Advanced Placement© and high school. The goal of OCEP is to provide the academic community with a criteria-based evaluation tool to assess and compare the quality of online courses."

"The focus of the evaluation is on the presentation of the content and the pedagogical aspects of online courses, yet OCEP also considers the instructional and communication methods so vital in a successful online learning experience."

Evaluation Categories including Definition and Goal (pdf)

Sample Course Reviews (See the links in the green box on the right.)


Quality Matters (QM)
" The Quality Matters™ project, sponsored by MarylandOnline, Inc. (MOL), has generated widespread interest and received national recognition for its peer-based approach to quality assurance and continuous improvement in online education. With the conclusion of three years of support from the Fund for the Improvement of Postsecondary Education (FIPSE), MOL is transitioning the Quality Matters project into a self-supporting program through institutional subscriptions, license agreements, and a range of fee-based services."

Quality Matters Rubric Standards 2008-2010 edition (pdf)
Preparing for a Quality Matters Course Review (pdf)


Michigan Virtual University Standards for Quality Online Courses
MVU's Online Instructional Design (OID) standards were developed in 2001. The standards organized into the following categories: technology, usability, accessibility, and instructional design. The standards are included in spreadsheet form so that computations are made automatically. The spreadsheet is contained in a downloadable zip file. It is a good idea to read the Readiness Overview and the Course Mapper first.

Note that while the tool is free, it is copyrighted by Michigan Virtual University and may not be further distributed in any form.


Free Assessment Summary Tool (FAST)
Developed by Bruce Ravelli at Mount Royal College, Canada.
" FAST is an anonymous online survey tool that automatically summarizes students' impressions of a course and/or instructor and supplies the data directly to the instructor. ... The user can create his own questions, or choose from a pool of 374 validated questions.  The assessment or survey tool is web-based, password-protected, anonymous, and instantaneously updated."

There are some good questions in this database, including a category of questions for online courses. To access these questions, you must create a login. FAST will send you an email within 15 minutes with your login information and then you can access the site.

FAST is a 2006 MERLOT award winner.

Top

 

Page updated: 28-Sep-2011

Accessibility Policy

Office of Academic Programs
The University of North Carolina at Greensboro
229 HHP Building, PO Box 26170
Greensboro, NC 27402-6170
VOICE 336.256.1003
FAX 336.334.3238