|
Memorandum
To: John A. Sebert, ABA Consultant on Legal Education
From: Peter W. Martin, Legal Information Institute, Cornell Law
School
Date: 12/7/2001
Subject: A Report on the LII's Two Multi-Law School
Courses Conducted via the Internet in 2000-2001
I. Background
In 1996 we (Cornell's Legal Information Institute or LII) invited
a number of law schools to join us in an Internet-based course.
Three accepted: the University of Colorado, Chicago-Kent College
of Law, and the University of Kansas. For three successive years
students at those institutions and Cornell studied "Copyright
and Digital Works" with me and each other. The course made
use of "off the shelf" Internet software (Web pages, Web-based
conferencing, desktop video conferencing, and e-mail) and succeeded
in adapting the law school interactive paradigm to this quite different
educational environment.
Begun before the ABA's 1997 "Temporary Distance Education Guidelines"
the course, nonetheless, fit within them being: a) "disseminated
from one law school and received at another law school" (see
section 1 of the guidelines), b) highly interactive (see section
4) and c) based on commercial grade technology (see section 5).
In diverse settings including two CALI conferences (1998 and 1999),
three AALS annual meetings, an ABA-sponsored distance education
conference (Nov. 1999) and meetings of the section's Technology
Committee the LII reported on this three-year experiment as it proceeded.
In the spring of 1999, we concluded that, rather than continuing
to evolve the initial model further, we should take what we had
learned about available technology and appropriate pedagogy, pause
a year, and create a fresh pair of distance learning courses. In
doing so, we hoped to break free from the weekly "real time"
video conference sessions that were central to the original model.
We carried through with that plan and did, in fact, offer two totally
asynchronous distance learning courses last year (2000-2001). This
report summarizes what we did and how and concludes with several
tentative conclusions emerging from this most recent phase of the
LII's ongoing exploration of distance learning in law.
II. The Courses
The LII's first on-line course focused on an area of advanced copyright
law. The 2000-2001 offerings were: 1) Introduction to Copyright
and 2) Social Security Law. The former is a high demand area that
many schools are forced to lump together with other topics of intellectual
property in a survey course. The freestanding course we offered
gave students with strong interest in the field an opportunity to
go deeper, while allowing the participating schools to reconfigure
their intellectual property offerings. Social Security is, by contrast,
a subject that receives negligible curricular attention in U.S.
law schools despite great importance and legal complexity. It is
one of a host of important legal topics around which a critical
mass of student interest and faculty expertise often cannot be found
within a single institution. Courses covering such topics are, in
our judgment, prime candidates for a distance learning structure
that enables schools to pool teaching resources and students.
Both Copyright and Social Security Law are upper-class electives.
Although normally they carry no prerequisites, their position in
the curriculum assures that students enter with a solid understanding
of the types of legal materials that frame the respective areas
(statutes, regulations, appellate decisions), with skills of study
and analysis developed during least one full year of law school
study, and with a command of legal concepts and vocabulary that
can be deployed to gain understanding of these new fields. As taught
in most schools neither course focuses significantly on the development
of professional skills or involves experiential learning in the
way a clinical course does or an evidence, procedure, trial practice
or negotiation course may. Instead they are organized around their
respective legal domains. Whether taught in a classroom or on-line,
they characteristically aim to build a solid understanding of the
salient features of the respective fields, plus a sense of how to
approach key issues from several important lawyer perspectives such
as those of advocate, counselor, advisor, arrangement-maker, policy
analyst or critic. Since neither area is static, topics of proposed
or imminent change regularly prompt analysis of contending policies
and political forces.
In short, this second round of distance learning experimentation
continued the LII's original, quite conservative concentration on
on-line instruction as a means of providing greater content depth
or breadth in the upper class curriculum through the sharing of
teaching resources and students. It neither ventured into the first
year learning experience nor into a broader range of upper-class
course types. The principal difference between these courses and
their predecessor consisted of the substitution of fully asynchronous
components for "real time" exchange. This change freed
instruction from the scheduling constraints inherent in assembling
students at the same moment across multiple time zones and academic
schedules. Less obviously, it allowed larger enrollments without
loss of interactivity or accountability. This, in turn, made investment
in reusable multi-media content economically feasible.
III. What Other Law Schools Participated and Why?
A total of seven law schools accepted the LII's invitation to join
in last year's experiment - Arizona State, Chicago-Kent, Kansas,
Rutgers-Camden, Rutgers-Newark, Seattle, and Vermont - with four
signing up for each course (Chicago-Kent participated in both).
Were distance education arrangements like this standard practice,
with most law schools participating as providers and receivers,
the basis for a school's decision to participate or not as to either
of these particular courses would have been straightforward. Assuming
adequate assurance about the quality of the materials and instruction,
the issue would reduce to considerations of curricular fit, faculty
deployment, and budget - e.g., does the course add an important
subject the regular faculty cannot cover (Social Security) or provide
desirable flexibility and increased options in an area of strong
student demand (Copyright) at an acceptable cost?
Because interest in distance education is high and experience rare,
there was a further reason for schools to join in. The experiment
offered an opportunity to observe carefully designed and executed
distance education at first hand. Few, if any, other law schools
have experience in Internet-based legal education comparable to
the LII's. Participating schools were invited to designate one of
their own faculty members as an "auditor/observer." Every
effort was taken to open up the process of course construction and
delivery. Schools were encouraged to use interviews, questionnaires,
and other means to evaluate student response and educational effectiveness.
In short, participation provided a means of experimenting with distance
education for schools that had not yet done so and of exploring
an alternative model for schools that had.
In order to share the results and compare our experience with that
of others, the Legal Information Institute ran a workshop on distance
learning course design during the summer of 2001. The weeklong conference
drew faculty members and technical staff from a broad diversity
of institutions. Schools that had participated in either one of
the 2000-2001 LII on-line courses were assured of inclusion and
contributed importantly to the proceedings (available on-line at:
http://www.law.cornell.edu/background/distance/workshop/).
IV. What Participation Entailed
As with the LII's earlier distance learning venture, all participating
schools retained responsibility for course registration, exam administration,
and related logistical matters. Students registered not with Cornell
but with their home institution. Grades and credits were local.
Participating schools were also responsible for front-line technical
support for their own students (with LII back-up), for assuring
that their students had adequate computer resources and Internet
connection, and for providing a meeting room suitable for local
discussions among enrolled students.
In order to take the course students had to have regular access
to multi-media capable computers with sound and Internet connections
capable of delivering streaming audio reliably - either in the law
school or elsewhere. As previously noted, participating schools
were encouraged to designate a local faculty member to be included
in all course communications. In short, as was true of our earlier
distance education arrangements, these courses were, to use the
language of the ABA temporary guidelines, in all critical respects
"disseminated from one law school and received at another."
The Legal Information Institute's responsibilities included: preparation
and distribution of course materials (free in digital format to
the students), instruction, performance monitoring (the on-line
analog of attendance), student evaluation and grading. Participating
schools paid a fee of $500 per student per course.
V. Pedagogical (and Technological) Approach
The basic components of this latest LII distance learning model
included:
VI. Schedule
and Exams
Because the LII's earlier distance learning course included a weekly
video conference it could not be fit completely within a single
academic term. Inconsistent start times and vacation patterns among
schools required distribution of those real-time class meetings
across two semesters. The current model, being more flexibly constructed,
did not require all students to be "in session" at the
same time. This permitted each course to be conducted within a single
term. Copyright was offered in the fall; Social Security, in the
spring. The precise beginning and ending dates were set to correspond
to the schedules of the participating schools. Final exams were
taken during each participating school's regular examination period.
In order to allow full use of the digital materials with which the
courses had been conducted at this critical point, the exams were
administered on a "take-home" basis in schools that did
not otherwise permit students to use computers to write their exams.
VII. Scale
Our previous model of distance learning worked only with relatively
small numbers - e.g., eight students per school and 32 total. One
important question we sought to answer with the new approach was
whether it could effectively accommodate larger enrollments, comparable
to those common in specialized law school courses taught conventionally,
without sacrificing interactivity or other qualities important to
effective learning.
Working closely with the participating schools we achieved enrollment
figures that provided that scale. Ninety-five students took the
fall term Copyright course; sixty completed Social Security Law.
VIII. Placing the LII's Approach to Distance Learning in Context
The form of distance education most widely practiced by law schools
to date represents the simple extension of conventional classroom
practice through high-end videoconferencing technology. Classrooms
at more than one location are linked to permit a teacher at one
of those locations to lecture or conduct more elaborate presentation
to students who are assembled at the same time in other locations.
With additional investment in technology infrastructure those remote
students can participate in discussion with the teacher and each
other. The principal advantage of this mode of distance education
is that it requires very little adjustment of working patterns or
expectations on the part of either teacher or student. It can be
used to create highly diverse collections of students (students
gathered in classrooms in different countries, for example) and
can link faculty members with students they might otherwise be unable
to teach.
Major drawbacks to this form of distance education include high-cost
at both the sending and receiving end and its requirement that faculty
and students assemble in "real-time." The latter can become
increasingly problematic as sites are linked across time zones.
The LII's current distance education courses rely on less costly
technologies and embody patterns of instruction that make substantial
use of asynchronous exchange and pre-programmed instructional materials.
The choice reflects a conviction that, long term, this direction
holds the greatest potential gains from networked digital education.
Last year's experience only strengthens this belief.
IX. Some Preliminary Conclusions
A. The fundamental
architecture and methodology are sound
For law courses of this type, i.e. content-defined and focused
on a statutory domain, and with students already well grounded
in the study of law, the model appears effective. The LII's 2000-2001
offerings generated educational outcomes that compare favorably
to those realized in similar courses taught in a conventional,
classroom-anchored mode.
Some might argue that in light of the widely observed tailing
off of attendance and engagement among upper-class law students
that constitutes too low a benchmark. Unquestionably, our ambition
is to achieve substantially better outcomes (at no greater cost)
through on-line methodologies, and our tentative conclusions,
set out below, encourage us to believe that we are headed down
that path. But in any discussion about regulatory and administrative
structures that privilege classroom instruction (as the current
ones do), the effectiveness of conventional teaching methods inevitably
and appropriately becomes the initial basis for evaluation.
B. Student mastery and engagement
1. Gauged
by the teacher
The students enrolled in these two courses represented a greater
range of language and analytic skills, work and life experience,
and facility in doing "law student work" than a teacher
is likely to confront in a single law school student body. Since
some of the participating schools had part-time divisions, the
mix of students in both courses included significant numbers
who brought directly relevant work experience to the exchange.
Taking account of that diversity, I judge the quality of student
work product I saw through the two terms (the weekly problem
submissions, on-line discussion contributions, mandatory mastery
exercises, and final exam) to be of very high quality. Measured
in terms of: 1) understanding and mastery of course content,
2) sustained engagement, and 3) learning from one another, the
course outcomes were, overall, better than I would expect to
achieve with the same students meeting thrice a week through
the term.
With a classroom-anchored course culminating in a final exam,
there are few reliable mechanisms for monitoring individual
student progress during the term. Attendance may or may not
be effectively tracked. Class preparation may or may not be
audited by periodic queries directed at non-volunteers. Even
with the most rigorous application of "Socratic" teaching
the large upper-class course provides plenty of cover for students
who opt for a "wait and then cram for the exam" approach.
In contrast, on-line teaching methods enable a teacher to be
far more attentive to the progress of individual students. The
model of asynchronous instruction represented by the LII's 2000-2001
courses included weekly progress expectations and four mandatory
progress checkpoints (the "mastery exercises"). Placed
in a work environment that logged student contributions, it
facilitated prompt intervention when any student fell behind.
As teacher I experienced an unfamiliar level of confidence that
I was detecting student difficulty or simple procrastination
in time to make a difference.
With a conventionally taught course, the overall work load is
crudely measured by the minutes per week of class time. There
are three credit hour courses and four. Inevitably some three-hour
courses impose more demands on students than others, but, over
time, students themselves police excess in one direction, faculty
colleagues and academic administration, extremes in the other.
For an asynchronous on-line course, with no shared clock running
on any part of the student-teacher interaction, there is, at
present, no such convenient metric. Each of the presentation
or problem modules comprising these courses did carry an explicitly
stated "run time". But summing up these times yields
a useless number, for several reasons. First, most students
exercised the control over delivery offered by the on-line environment
to pause, reflect, take notes, and repeat. Consequently, completion
of any 20-minute module could easily take twice as long. Total
time spent reading, reflecting on and responding to messages
in the on-line discussion area or composing a problem submission
can only be the subject of speculation by course builder and
teacher.
So long as such courses remain the exception rather than the
norm, however, their contents and work demands can be benchmarked
against conventional ones. Both of the LII on-line courses covered
exactly the same ground as their classroom-based equivalents
at Cornell.
2. As experienced by the students
Students in both courses were encouraged to fill out a final
course questionnaire, on-line, during the week before their
final exam. The response rate was 30% in the fall, 38% in the
spring.
To begin, the students (who were predominantly in their last
year of law study) were asked to compare the on-line course
they were completing with "other specialized law school
courses with comparable credit." Specifically they were
asked about: (1) the total time and effort required, (2) their
own success in mastering the material covered, and (3) the quantity
and quality of teacher feedback and amount of discussion with
teacher and other students. Answers were consistent across the
two courses with a strong majority of students reporting that
they worked harder, achieved comparable or greater mastery,
and experienced more feedback and exchange than in a classroom
course of similar content. (Since the Social Security course
came second, it benefited from a few lessons learned during
the first term and, to the extent there were discernible differences
in student evaluation of the courses, it received slightly more
favorable ones.)
C. Course
architecture - modularity
The challenge of disaggregating the range of activities carried
out during standard classroom meetings into presentation modules,
interactive exercises, problems for analysis and submission, and
on-line teacher-student discussion forces an unusual degree of
attention to pedagogical ends and means. Done properly, the resulting
asynchronous components allow students to organize their course
work for more successful learning. The fixed length class at a
standard meeting time imposes significant burdens to which most
law faculty members and students are blinded by familiarity. The
duration is often too long, sometimes too short. In terms of student
readiness and attention, the moment is arbitrary and, therefore,
frequently the wrong one.
By contrast, students taking these on-line offerings had enormous
flexibility in how they fit the multiple course elements into
their weekly schedules. Such control over the exact time and place
of their learning was, for the students, the most highly valued
feature of the asynchronous course architecture. Not only could
students to take up a given module when they were ready and able
to focus, they could run it, pause, take notes, and return to
puzzling points. In numerous ways they could exercise a degree
of control over each step of the process quite impossible in a
real-time classroom session.
Course questionnaire results indicate that more students than
not took advantage of the capacity to pause and replay the course
presentations and also to block and copy the presentation outlines
and visited text (e.g., statutory provisions) in the course of
note-taking. As one student explained: "What was great about
the [format] was that I was able to go through the [presentation]
once but pause many times. I would write what you said and then
stop .... Then I would look at the statute or regulations and
highlight it and make notes in the margin."
In addition, modularity has a direct bearing on future reuse,
course maintenance and other important elements directly related
to the economics of this form of education, a topic addressed
below.
D. Course architecture - streaming audio linked to web-based content
1. Why
not video?
In faculty workshops and other settings where these courses
have been described and demonstrated, one question that recurs
is why we chose audio rather than video for the "teacher
presentation" components. The reasons are numerous.
To begin, there is a stark cost-benefit difference between the
two. While the educational gain from using video can be enormous
when the medium is being used to show process or action - whether
the topic be marine biology, migration trends of the past century,
volcanic eruption or cross-examination at trial - the educational
gain from adding a head and gesture to the teacher's voice is
minimal. No doubt some psychological value flows from students
being able to visualize their professor, but that can be realized
through a short video introduction.
On the cost side, the gap is enormous. Video imposes several
different types of added cost. Most obvious is the greater expense
of initial production. Students bring expectations of broadcast
quality to video material. Creating video content to that standard
is more expensive than first-rate audio by several orders of
magnitude. Furthermore, whatever assumptions one makes about
the ongoing rate of course revision in successive years, even
at levels as low as 10-20% per year video, being far more complicated
is, therefore, more costly to maintain.
Bandwidth is a totally separate matter. The streaming audio,
multi-media technology used in the LII courses operates quite
reasonably over a dial-up Internet connection. Streaming video
does not. Its use requires students to have more capable and
more expensive network connections, and it also obliges the
offering institution to have greater serving capacity.
2. Some distinct advantages of streaming audio linked to Web-based
content
Most law
teachers create in relative solitude. They write, they prepare
and deliver their courses by themselves. Use of teaching assistants
is rare. Effective though they may be in live lecture and discussion
formats, few are comfortable and as skillful before a camera.
For these and other reasons, a mode of multi-media course production
that begins with a microphone attached to the law professor's
office computer is accessible to many more teachers than one
requiring use of a studio and video crew. With current authoring
tools, high quality audio can be prepared, edited, and revised
by a faculty member, alone, in his or her office. Software tools
designed to allow presenters of all kinds to prepare audio files
to accompany their PowerPoint slides can be readily adapted
to the creation of presentations that refer to a wider range
of Web-based material. With a set of course materials on-line,
the teacher can speak about a statutory section or passage in
an assigned case and have the very text automatically loaded
in the student's browser as it is being discussed. The student
can, in turn, pause the audio in order to reflect on the text,
copy portions into his or her notes, or to follow hypertext
links that connect to related material, as for example, another
statutory section defining key terms or qualifying its apparent
meaning.
Within a modular architecture, such content can be assembled
in different combinations and configurations. It can, in successive
years, be altered by adding, subtracting, or substituting new
audio and textual material, without the need to rebuild from
the beginning.
E. Course
architecture - interactivity
1. The
mastery exercises
The interactive features included one that drew near unanimous
praise, the set of four "mastery exercises". These
mandatory problems, placed at the end of two to three weeks
worth of material, called for students to submit a few paragraphs
of analysis, via a Web form. Each problem was posted a week
before its deadline. Forty-eight hours after the deadline the
class received a generic (i.e. non-individualized) feedback
memo. It set out my view of the issues and responded to common
errors or confusions revealed in student submissions. A typical
student reaction to this course component read: "I thought
the mastery exercises were useful and the feedback was very
helpful too." Another student, perhaps more candid, wrote:
"The mastery exercises were a necessary evil. I am thankful
they were included insomuch as it is too easy to fall behind.
Those exercises forced me to keep up with the material and review
it again and again until I found the appropriate answers."
Yet another exclaimed: "Finally a law school class where
a question is not answered with another question."
2. Interactive self-assessment exercises
Also popular were the interactive exercises that allowed students
to gauge their level of comprehension following the presentations
and readings on a topic. This is one dimension in which several
expressed a desire for more: "The pop up windows for the
illustrative questions were great. I wished they popped up for
each question."
3. On-line discussion preceded by problem submissions
The on-line class discussions had both fans and detractors with
the obvious base for comparison being discussions in a conventional
class of similar size. Objectively several things seem clear
to me. Including the forms which asked every student to submit
a question or take a tentative position on a hypothetical problem
before the related on-line discussion began many more students
were involved in the exchange throughout the course, topic by
topic, than I have ever been able to bring into "real time"
classroom discussion. The evident degree of reflection and level
of discourse were high and I observed more frequent introduction
of personal experience and references to material outside the
assigned readings, including current events. However, just as
some students found this a less inhibiting venue for "discussion"
than a classroom, others exhibited at least comparable reluctance
to "speak," perhaps in part because the class included
others they did not know, from different law school communities.
Some reluctant posters said they liked the discussion environment,
nonetheless: "I like the discussion area. I like to read
all the messages even though I didn't put anything there.
:-)"
F. Sources
of student dissatisfaction
Most of the students were pleased with their on-line instruction.
A number pronounced it their best law school learning experience.
What were the complaints? Anyone who has taught an upper class
law school course can imagine many of them. To some my voice was
soothing, to a few it was monotonous. The expectation of involvement
struck a handful as "unrealistic." Said one: "I
have never been in any class where EVERY person has something
to say on EVERY topic!"
Based on a review of the full range of student feedback, I have
concluded that most individual student frustration and dissatisfaction
can be traced to one of three sources: 1) technical problems,
2) specific expectations of what an on-line course would or should
be that were not fulfilled by this one, and 3) the challenge of
dealing with digitally-delivered course readings.
1. Technical
problems
Most students reported few or no technical problems. Those with
the worst experience fell in one of two categories. They were
either at a law school plagued with frequent network difficulties
(true of at least one of the participating schools) or they
were relying on an Internet service provider (ISP) with inadequate
capacity, most frequently AOL. Learning from the fall term course,
we emphasized at the outset of the second one that students
should, as a test, run the introductory unit of the course under
the conditions (computer, ISP, time of day) most likely to prevail
later while they would be taking the course.
2. Expectations
Upper class law students know what to expect in a classroom-based
course. Currently at least, those taking an on-line course have
far less certain ground for anticipating how the course will
be conducted or how they will, as individuals, respond. To deal
with this source of potential difficulty, we placed materials
at the beginning of each course that explained and also modeled
how the course would function. This is a second area in which
our second course benefited from experience with the first.
3. Digitally-delivered readings
On-line courses can be taught from standard commercially published
casebooks. Concord does so routinely. A separable feature of
the LII's approach to date has been the use of original course
materials that, being free of the license constraints, can be
delivered on-line along with the rest of the course. This approach
permits tight integration between the readings and the on-line
instructional materials. With a full set of course materials
on the Web, our presentations can and regularly do link to portions
of the assigned readings. The same is possible with comments
posted in the on-line discussion area.
On-line delivery need not mean reading from the screen. For
both courses the readings were offered in parallel versions:
a browsable format and also downloadable word-processing files
ready for printing. The range of choices and the consequences
of some of them (i.e. the bulk and, at some schools, the cost
of printing out the full course materials) surprised a number
of students. The complaints on this score were significantly
reduced in the spring, probably also a consequence of greater
clarity in the course introduction about what to expect coupled
with specific advice about how to manage the readings.
G. Teacher
time and effort (and institutional costs)
Institutions and teachers need to conceive of the preparation
of on-line courses as being much more like writing a book than
like teaching a class. The time, level of commitment, and need
for attention to detail required to create the first version of
a multi-media on-line course can be justified only if viewed in
book-like or long-term investment terms.
When the course is constructed of modular and reusable elements
that investment can pay attractive dividends, but only when the
course is offered to successive cohorts of students. As it is
offered for the second and third time, requirements of teacher
time and effort reduce to those entailed in revising the reusable
material and interacting with, monitoring, and assessing the students.
X. Continuing
the experiment
Encouraged by last year's experience we are repeating both courses
this year. Since they were built around reusable modules, designed
with an eye to revision, we have, as we planned, been able to make
updating and other changes selectively (without starting over from
scratch).
The course architecture also contemplated re-assembly with appropriate
additional material for different student populations. To explore
this process and its potential, we have adapted last year's Copyright
Law course to produce an offering that introduces the principal
issues and features of the field to university students in such
areas as journalism, art, music, and computer science. This non-JD
course is being tested at Cornell during the current term in anticipation
of wider distribution.
The Social Security Law course will again be offered to law students
during the 2002 spring semester. Participating law schools will
include some of last year's group joined by one or more others.
XI. Further Information
Both LII courses, as offered last year, remain at the institute's
Web site. Unlike conventionally taught courses where only the residue
following completion rests in notes and memories, they can still,
months later, be inspected in full.
While the sites are not open to the public, any member of the Council,
its staff or committees are welcome to visit them. The respective
URLs are:
[removed] [Copyright]
[removed] [Social Security]
The materials are, in differing degrees, password-protected. For
anyone seeking an overview, the best place to begin is with Social
Security site. It allows access, without password, to the course
syllabus, readings, multi-media environment, and introduction. To
proceed further with that course or to inspect any of the Copyright
site requires a guest password. It is available upon request. Anyone
wanting the password for either course or further information about
them should feel free to contact me directly, by phone (607-255-4619)
or by e-mail (martin@lii.law.cornell.edu).
|