Online Assessment Tools
A Review
by
Aldena Cherva, Phyllis Harris, Melissa Weege, Joyce Pederson
Assessment and Evaluation in the Online Classroom
University of Wisconsin – Stout
How does one measure student progress in online education? With an exponential explosion of credit online courses, this question has no easy answer as methods are continuing to be developed to meet this unique learning option. The unique nature of constructivistic learning through collaboration inherent to e-learning, requires reconsideration of more traditional assessment methods while developing and refining new ones. In choosing assessment tools for use in course planning, it is important to carefully examine how they work, identify their benefits and limitations, and consider how they could be used in the digital classroom. This paper reports the analyses of four assessment tools whose uses vary within the online classroom. A range of assessment strategies included Exam builder – a tool designed for a range of testing applications, ePals Schoolblog – a secure school-based communication and learning tool, Wikis – an editable web page tool with a range of applications, and Teacherease Instructional Management System – a K-12 curriculum management system.
Exam Builder – contributed by Phyllis Harris
Introduction
As a premier provider of online testing software, Exam Builder has clients throughout the US, Canada, Europe, Asia, Australia and South America. Some of the test applications are for Telecommunications, Biotechnology, Aerospace, Technology and Insurance industries, as well as Government.
Advantages and Challenges
Exam Builder is a tool used by educators, Training directors and Human Resource personnel to create and administer exams to diverse groups of students, employees, managers, and other staff. Exam Builder can be used to manage exams, create historical reports on training, and assist in organizing the way to deliver training to all levels. I found this software has a very detailed explanation of how it operates and is easy to operate. Creating exams is quite easy and there is a choice of what is called “Click-and-Learn and Certificate” exams. I found the Student Manager, Exam Scheduler and Image Manager easy to operate. Exam Builder has a very impressive Report Manager, allowing the operator to export to PDF or Microsoft Excel. The Report Manager also provides Analysis Reports, Exam History and Student History.
Exam Builder is not Plug and Play software. It takes a good technician with network service knowledge and skills, to set up the software securely. Security is paramount whenever there is transmission of online exams; heightened measures of security are necessary to insure protection of the test and answers.
Learning Objectives
Exam Builder is an excellent assessment tool for highly technical application. Having the ability to add images is a great advantage for the author of the exam. Personalization of images to a particular technology, such as Biotechnology or the Aerospace industry, increases the use for multiple applications.
http://www.exambuilder.com/
http://www.exambuilder.com/userguide/eb_userguide.pdf
ePals SchoolBlog – A Reflection Assessment Tool – contributed by Joyce Pederson
http://schoolblog.epals.com
Introduction
Reflective journaling through the use of Web 2.0 weblogs provides an online assessment tool that can provide formative and summative assessment information. As mentioned in Assessment and Online Teaching (p. 9), the use of reflection is a way “students are being encouraged to manage their own learning.” Unlimited applications of blogging exist in assessing the levels of students’ developing content knowledge, their abilities to construct new understandings through collaboration, and their metacognitive skills. One unique weblog that is frequently used within elementary through university classrooms is the ePals SchoolBlog© , a free weblog available through ePals© , a provider of online school-safe educational services since 1996. SchoolBlog is a communication and collaboration program that provides blogging between students, teachers, administrators, and parents in a safeguarded and restricted use format that ePals calls a “walled garden.” Schoolblog’s primary function is to provide students with the opportunities to communicate using message boards and forums with a focus on classroom activities. Users responding using both open-ended and specific prompts, can demonstrate content understanding using constructivistic peer and teacher interactions which can also be addressed by the teacher.
Advantages and Challenges
Information from the online ePals SchoolBlog Teachers Guide (2007) indicates the following user-friendly features: clear commands, manual, text formatting, curricular theme units, customizable template; customizable homepage, an assessment tool, threaded discussions and topic forums, moderation of content, and provision for RSS feeds and attachments. Its focus on security through controlled access and internet within its school-based blog gives students a safe opportunity to develop reflective skills through journal writing and discussions of other information that stimulates critical thought. Teachers can assess writing and collaboration skills through monitoring. It is a free site. ePals won the 2009 ComputED’s 14th Annual Multi-Level Award in the Internet Communication Website category.
Its general inaccessibility to the broader community limits broadening students’ experiences to community and cultures beyond their school without special arrangements. It is a restricted blog with the blog open on a school-wide or classroom-wide basis with an option to add more users who can be added by the administrator. More teacher input may be needed to guide students during collaborations to expand range of thinking. Obtaining rights to use this blog are time-consuming with an email reply needed to set up an account.
Online Learning Example
SchoolBlogs could be used within a reading classroom as a reflection tool in which teachers post questions to which students respond or students could post questions for peers to respond to after a class reading of a book. It could be used formatively at given stop points in the book. Sharing with other classes within the user network could also occur.
Learning Objectives
The student will demonstrate awareness of the development of a character by describing two examples of a personal characteristic such as loyalty in the story. Other students will discuss one of these examples in a response in which they agree or disagree.
At the beginning of a class session, in a one-minute response, the student will respond to a teacher prompt related to the key point of the assignment as well as indicating an area not understood. (The teacher can then aggregate these responses and address concern areas.
Wiki – A Peer-to-Peer Collaboration Tool – contributed by Melissa Weege
Introduction
A Wiki space is an editable Web page in which students can create pages or several pages of material or work on a group project in one place — on one site. This eliminates having to email different versions of a document back and forth as it is worked on. Documents, graphs, and hyperlinks can be loaded on to the site to be viewed by its members. Additionally, the entire Wiki document is able to be viewed by team members in one place and can see the most current version, as well as who made the most recent edits. Wiki sites are free to use and easy to access from any Internet connection.
****For a good Wiki demonstration see the following brief YouTube video: http://www.youtube.com/watch?v=-dnL00TdmLY
According to Wikipedia, the term Wiki comes from the Hawaiian language and means “fast” as it allows for quick and easy creation of Web pages. Additionally Wiki sites are easily editable by anyone allowed access to it. The Wiki is a great tool for helping students collaborate on projects to create one unified project. Instructors also have the ability to comment on student work as it is being worked on to clarify student ideas or points. Students or group members can also make comments and pages and have discussions furthering the collaborative effort on the Wiki site. I have used Wikis both in face to face courses for out of class work, in online classrooms for group work, and for collaboration with committee work. Some potential Wiki sites include:
- Wetpaint Wiki- http://www.wetpaint.com/ This is a free site, however does have a large amount of ads unless you specifically indicate that you are an educator or buy a license to use Wetpaint. It is easy to use and very straightforward.
- PB Works- http://pbworks.com/ This is also a free site and is free of ads as well. It is very user friendly and additionally works well for building E portfolios.
Weaknesses and Challenges
While a Wiki site is easy to use, it can be difficult to access it in some settings (schools, hospitals, clinics, etc.) because of firewalls. I have desired to use Wiki sites to communicate with my students at hospital internship sites, but the Wiki access has been blocked by the respective hospitals’ Internet security. So the Wiki site access is dependent upon the level of Internet security in the location of use.
Another challenge of using Wikis is that anyone can make changes to the site, especially if the site is open and not secured. Often times I leave the site open so my students do not have trouble accessing. There are times when the site’s information may not be correct then because anyone is allowed to access it. This is true in the case of Wikipedia. Here is an encyclopedia that can be edited by anyone and while authorities keep an eye on their respective information, incorrect information may still be presented.
For the Wiki to be a good collaborative tool amongst peers, all peers need to participate. As is typical in some group work, there may be a participant that does not contribute on the site making it hard to work together and produce a final project or paper. However this can happen with any tool or face to face, so it is not just the Wiki that may have this type of problem.
Online Learning Example
Wikis are largely used for collaborative work, however it can also be used to blog, create a portfolio, and various other projects. An example of its use in an online classroom would be for a group project or paper. The students could easily establish the Wiki site and invite its group members and instructor to it. As part of my graduate work, I was assigned to work on the main themes of a certain chapter of our textbook with two other students. We needed to address three to four main points, summarize them, and present them to our class. One of my group members suggested that we do our work on a Wiki site. It worked out great and was a wonderful tool for collaboration. It was easy to see the edits made by the other group members. The Wiki site also allowed for comments to be made and discussion about various points of the paper.
Currently my students are posting their lecture objectives for a unit in a pathophysiology course on a Wiki site. While this is a face to face course, this type of collaboration would also work well in an online course. For this Wiki site, when everyone has contributed, they will be able to print each page for a study guide for the unit exam. Their Wiki site is: http://unit2patho.pbworks.com/
Online Learning Objective
Students will collaborate in groups to write a final paper.
Course Tracking Tool – Teacherease – contributed by Aldena Cherva
www.teacherease.com
Introduction
This particular product appears to be a functionally rich student and curriculum management system. It provides tracking for student progress, lesson planning, matching of lesson plans to school standards, and curriculum mapping across the environment being supported.
Of particular note are the capabilities of the product to communicate with students and parents – hence – it is more suited to K-12 education. I can see value in up to date status and progress of each student. I can also see value is marrying lesson plans to school goals and standards. Elaborate lesson plans are possible down to and including day to day plans. Each of these plans can be tied to goals or standards and overlapping lessons or voids in lesson plans may be identified. This capability, of course, assumes that all pertinent classes and lesson plans are entered into the software.
There is a very sophisticated grade book included with this software that will track any conceivable possible status of an assignment. That is of course if the teacher enters to a great level of detail.
Advantages and Challenges
For the school without such a system this one appears to be easily integrated and used. To have ready ability to communicate with parents and students is almost a given today. This product will also produce paper status reports for those parents that do not have Internet capability. Involvement of parents is critical to the success of a student; this tool provides one involvement.
Although not a K-12 teacher I can see great value is having data bases of school goals and standards that I could match with my specific lesson plans. Captured lesson plans also allow for substitutes to readily come into a class if I am unable to due to illness or other reason.
A sophisticated grade book would lend itself to tracking trends by student. I imagine that such analysis as frequency of late assignments, incomplete assignments or habitually low grades would be of use. I’m guessing that a teacher would use a paper grade book in the same way. I know that I use my grade book in Blackboard to perform such analysis.
I can see great advantage for the school administrator. If each teacher were to use this tool consistently administration of such items as adherence to standards, goal satisfaction and even performance of a teacher would be greatly enhanced. The additional capability of curriculum mapping across various grades, schools and even districts would help to ensure consistent teaching and learning was occurring. It also would point up gaps and overlap in teaching and learning.
This software appears to be labor intensive. I know that the vendor claims that grading is simplified but indeed every grade has to be entered. In fact to utilize some of the more extensive analysis such as an assignment submitted but not graded requires even more data entry. I can see a teacher finding it easier to just use a paper grade book. Strict guidelines would have to be implemented to take advantage of all the deliverables of this system.
I could find no ability of this software to interface with course management software. I am sure that such an interface could be developed but I don’t find the hooks already in place.
There would be significant data entry required at the onset of such a system to enter each and every goal and standard the school might have. Perhaps the most difficult challenge but maybe the most beneficial would the, establishment of such goals and standards.
Online Learning Objective
This particular tool does not lend itself to specific learning objectives but rather gives the teacher the wherewithal to monitor the progress of his/her students in their fulfillment of learning objectives. The correlation of graded assignments to learning objectives and goals and standards would aid the teacher in assessing effectiveness of such graded assignments.
Conclusion
The emergence of online course delivery is a recent phenomenon that will expand learning opportunities to learners in a variety of settings. Measurement of student’s levels of achievement will be an integral part of overall course evaluation. Does the course make a difference in a student’s understanding of a subject area? How can the standardized method of testing be applied in the online classroom? Are there ways in which evaluation can be done using authentic samples of their work as a means of measuring not only content knowledge but ongoing constructivistic critical thinking as well? How can we align the curriculum with assessment to more easily manage our classrooms, especially as more students participate in this option? The tools reviewed in this paper addressed these issues and have provided an introduction to them and their uses. Clearly, new applications of online assessment will continue to be available. Becoming familiar with the ones discussed in this review will be a good first step.
Posted in Uncategorized October 17, 2009
This week I was introduced to the Inspiration software program as an assessment tool. Learning to use the tools within the program to represent my mini-course objectives was relatively easy with the clear directions provided in the manual. I have found that I have been spending a lot of time with trial and error learning on some of the tools presented in this class as I was unsure what I was supposed to do. I am learning to better understand the “language” used in the instructions and help menus used in these programs. I think that will make future work with new tools easier. I liked using the Wiggins approach to goal setting and would recommend his books to anyone who would be using that approach. I like the emphasis on student levels of understanding, putting the student at the center of the planning. I have the first edition of Understanding by Design (1998) from ASCD which has probably been updated since its first printing. It puts curricular planning and assessment into a meaningful perspective. I enjoyed the readings for the module and even though the Bonk article may have been somewhat dated, it did provide an overview of tools that can be used within the online classroom. The updated information was also appreciated. The case study interested me as it presented a complex assessment approach that visually represented goals and results. As I mentioned in my response, it reminded me of the Delphi Technique which I participated in a university setting. The emphasis on verifying value in the curricular goals reminded me of the work done by Heidi Hayes Jacobs. I like to look assessment within a contextualized situation. As I learn the tools being presented in this course, I am trying to figure out how they will “fit” in my future instruction.