Running Head: ONLINE C-TESTS FOR TAILORED TRAINING
program name
WEB’s Life-Long Learning Centers
fiscal agent
BLUM Enterprises, Joe Anthony Blum, CEO
TIN
262-76-8390
project nature
A pilot
post-graduate project proposal · University of Central Florida · Dr. Martha Lue
project director
Joe Blum, 5378 Round Lake Rd., Apopka FL 32712 · 407-761-1611
Abstract
BLUM Enterprises intends to establish a resort-like
training, tutoring, and research institute. The engine of the project will be
Klein-Braley’s C-test (1982), which was called an excellent tool
for assessing language teachers (English Only Europe, 2003), and, judging from
a survey of 20,000 European collegiate students (Coleman, 1995), there is no
reason why C-tests and their many variations should not also become the
LL/ESL/EFL industry’s favorite teaching aid. With the help of ClozeOnline,
our software, C-tests are now easy to create, administer, and
score; in fact, they are self-scoring. ClozeOnlinePlus, our
tailored tutoring curriculum, combines the advantages of private
tutoring with those of working with breakout groups, while applying the newest
of research to foster the development of Life-Long Learning habits. Training
new teachers and coaches is our scholastic aim; putting these coaches to work
where they are needed the most is our practical mission.
WEB’s Life-Long Learning Centers
Online C-tests for training learners at all levels of competence.
The four aspects of ClozeOnline™, the online C-test.
1.
ClozeOnline™ is an assessment instrument that
uses the C-test (Klein-Braley, 1982), now administered online, for evaluating
(a) foreign language
teachers in their own country, (b) foreign language teachers in a foreign
country, (c) teacher competency in using a foreign language to teach a subject,
and (d) for evaluating students, to test the degree to which they have become
native-like. NOTE: The present consensus of C-test proponents in Europe is
that a non-native speaker must come within ten percent of a native speaker’s
accuracy rating on a tailored C-text to qualify for licensure as a teacher of
that language at that level (English Only Europe, 2003).
2. ClozeOnline™ is achievement-measuring
software
that produces classic cloze tests (Taylor, 1953), L-tests
(Kokkota, 1988), and several versions of Klein-Braley’s C-test. It can
be installed on a school’s own server or used online through private directories.
The program creates also a printable version of the tests. (Exhibits 9b, 9c, or
http://clozeonline.us/Babaii.htm.)
3. ClozeOnline™ is the engine
used for training language skills at all levels starting with
grade 2, specifically the elements of (a) active reading, (b) reading
comprehension, (c) paragraph-style text creation, (d) use of proper grammar,
and (e) meaning-based text restoration. On a peripheral level, taking tests on
a weekly basis has proven to enhance a student’s test-taking skills.
4. ClozeOnline™ is the core
element of the ClozeOnlinePlus™ outreach program. ClozeOnlinePlus™ intends to
utilize (a) the talents of parents and grandparents whose children turned out to have
better-than-average grades; (b) retired teachers and professionals who subscribe to positive
reinforcement practices and now wish to volunteer their help; and (c) select rehab volunteers
who agree to uphold the “Look for the good and praise it” concept, while practicing the craft
of leading others to enjoy the learning process.
This above stated joining of efforts is what has been given credit for the
outstanding success of several of the nation’s cohort programs. Current
research is attached as Appendix B with additional sources and summaries
available at
clozeonline.us/Research/index2.html
Method
Populations.
The
populations of the outreach program are below-average learners (on math- and language-related subjects) in grades 2
through 6. To tutor these grade levels, we gladly go to the schools. Students
in grade levels higher than these must come to the Learning Center. ESL
learners must come with a family member to increase the odds of having someone to
practice with away from school.
As to
group size in schools: At the macro-level, there is no limit to the acceptable
number of breakout groups. At the micro-level, since we deal with a quasi-volunteer
labor force, we can afford the luxury of limiting each group size to two or
three students. The methodology is outlined below. It incorporates training
techniques that I have used successfully for over forty years in the private tutoring
industry; they are techniques that closely paralleled those researched by
pacesetting educators such as Ross McDonald and Vincent Tinto, which are
techniques now used at the University of Central Florida’s SARC (UCF, 2003).
Their research has documented an average increase in test scores of almost one
full letter grade through such practices. (This breakout concept also
accommodates the trend to no longer pull poor performers out of a classroom but
to bring ESL or remedial teachers into the classroom on an as-needed basis. Our
coaches are an additional low-cost resource for helping with the
developmentally delayed.)
As to
training: When not
trained in the field, trainees start by taking a few tests in each of the grade
levels, starting with “grade2”. (NOTE: Test results are not archived for as
long as the [Student Name] and [E-mail Address] are labeled “test” and “test@test.com” respectively.) Next, they
practice the writing and copying of other passages from other readers or
textbooks in the format required for C-test processing. (NOTE: They do this
in their private directory until I or another licensed trainer approves their
tests for use in public directories.) Third, they learn to create cloze and
L-tests for phase 2b of the program (with NRL set to “0” and “1” respectively).
Fourth, new coaches memorize logic-based test questions to have fillers for
ending the first few weeks of the program with confidence-building activities.
Fifth, they learn to use a proprietary math program (using simple addition and
multiplication with a speed-test feature). Sixth, they must familiarize themselves
with the mechanics of retrieving Big Cheese quotes that are used during the
optional phase 3 of the program, including all ongoing tailored training. Last,
after being registered as a school volunteer, new coaches go to their
neighborhood school principal to offer their services for twenty weeks on a
trial basis (to randomly-selected students from randomly-selected classrooms via
randomizer.org).
Subsequent groups (including the first, if they wish to continue with phase 3)
may be billed to BLUM Enterprises, the grant(s) administrator.
As to
duration: We expect
to have the developmentally delayed caught up (at least psychologically) within
twenty weeks, i.e., the length of a typical spring or fall semester. Students
started at the end of a semester are either coached by email (until an
acceptable level of competence is achieved) or they are restarted (at whatever
point the group wishes to continue from). A group’s 20-hour research period
ends when the exit exams have been given and the questionnaires are back.
(See Appendix C.) When
45-minute sessions are the only option, twenty-five sessions may be logged
before administering the exit exam.
Procedure
Typical Week 1 (using just ClozeOnline™[the test] for 45-60 minutes at
the school).
First, the students watch as
their coach logs on to http://clozeonline.us and enters a group nickname
at [Student Name] and then the same name for the [E-mail Address] field in the
format of “name@name.com”. This is followed by
entering “grade2” (or grade3, grade4, etc.) as the [Teacher’s Name] to get to
the test samples from their textbook. Next, the coach asks all three whether
they recognize one of the test titles and waits until they have recognized at
least four. The coach clicks the first to load it.
The coach now presses the
keyboard’s [Insert] key, activates (clicks) the curser inside the text box,
uses the four arrow keys to maneuver the curser to the beginning of the first
underscore, and announces to the students that s/he is ready. Two of the
students stand (one on each side) and start to guess at the missing letters. As
they call them out, the coach types them to restore the text. When finished
restoring (the 20-25 truncated words), the SUBMIT button is clicked. The
process is repeated when the score is less than 100 percent. The third student
stands behind the coach, monitors what is going on, and takes notes (to report
to the teacher).
During this first session, the
coach restores a second sample test but asks the students to switch sides and
suggests some other variance in their response pattern to help establish a
cooperative relationship. The rest is the same; the coach starts by re-setting
the keyboard to [Insert] mode.
To start the third of their
selected topics, the coach says, “Now you try it. Who wants to go first?” . . .
“OK, which one would you like to do?” . . . “Here, you click it.” (One types, the other speaks. When finished,
they reverse their roles and restore the last of their four selections.) It is
important to stress the pressing of the [Insert] key every time the SUBMIT
button is pressed. Should they forget (which is evidenced by seeing text being
pushed out of the way), the coach shows them how to use the [Esc] key to
restore the window to its proper integrity. Upon finishing the session,
the student monitor reports to the teacher; the other two are invited to fill
in the gaps.
NOTE: (1) After the first session
with a new group, the coach asks their teacher to please confirm, via email,
the continuation of their program for the following four weeks. (2) At the
coach’s option, this first session can be done with just two students to help
control the initial enthusiasm but, since the objective is to take advantage of
group-learning dynamics, working with three students at a time is better. (3)
The coach helps with keyboarding but allows them to hone their own skills in
rotation; doing so allows at least one at a time to concentrate on the task at
hand, which is correctly restoring mutilated text.
Weeks 2 through 5: (the official start of) Phase 1a.
At the rate of four tests per
session, the group now goes through the rest of the samples but accesses them
through a private directory created for that teacher with an automated email
service for reporting test results. Starting with this session, the coach
restores only the first of the four paragraphs and helps the students do one each
(with classmates sounding out the words). As before, the coach helps with
keyboarding, but does so only with letters not assigned that day for
practicing. (At the coach’s discretion, two can take turns using the two
halves of the keyboard while the third one calls out the letters to be
restored. The coach may wish to help maneuver the curser to the next set of
underscores via the arrow keys and depress the [Insert] key).
During the last of these five
sessions, the coach confers with the teacher on the issue of continuing five
additional weeks but this time not on previously created text, but with
individually created paragraphs that, ideally, were taken from the students’
journal entries. Teachers will be asked to give official credits for each
successful creation and completion of a test . . . 1 point for each week’s
paragraph-style contribution can add a total of 10 points over the twenty weeks
(1 point each for weeks 6 through 10 and weeks 16 through 20).
Weeks 6 trough 10: Phase 1b.
The objective of this phase is to
create tests using the students’ own or selected words. In essence, each
student brings in a sample paragraph, such as an excerpt from his or her
journal, which the group then reads in unison and briefly discusses. By lot or
judgment, the coach now either selects one of the three and adds a word or two
from those not chosen or consolidates them at random and creates a new one. To
do so, s/he goes to the MAKE TEST screen, names the test with a date, e.g.,
0707 for July 7th , writes the group’s name in the page code, and
clicks MAKE TEST. The coach then (1) types and creates the test in the default
mode of NRL=50%(+), (2) switches to the sign-on screen, (3) enters the group’s
name and email, (4) makes sure the [Teacher’s Name] is still set to the correct
teacher, (5) selects the test they just created, and (6) proceeds to get the
youngsters to restore it. Once they can restore the text without errors, the
coach switches back to the MAKE TEST screen and recompiles the text in the
NRL=50%(-) setting, meaning that the number of remaining letters is reduced to
2 out of 5 instead of 3 out of 5. The even numbers remain at NRL = 50%. NOTE:
NRL=50%(-) is the official C-test setting. Time permitting, the coaches
spice up the end of each session with a riddle, word problem, or sample test
question. Upon leaving the workstation, the coach asks for feedback to, “That
was fun; did you feel some improvement today?”
Weeks 11 trough 15: Phase 2a.
Phase 2a is identical to phase 1a
in concept and format but with the NRL set to 50%(-) on either the same
textbook excerpts or with text from a book by another publisher at the same
grade level. However, it is different from phase 1a in that the team members
now sign in under their own name (rather than the group’s name). The purpose:
to select, by performance, possible peer assistants for the next group of
stragglers. (Only those who can restore accurately should be used for
peer-mentoring activities.)
Weeks 16 trough 20: Phase 2b.
Phase 2b is like phase 1b with
the added dimension of texts having been mutilated at the NRL setting of “1” to
create a cloze variation called “L-test”. The skills now learned are
keyword-related. Depending on their individual learning style, some students
will find this L-test simpler; others will find it more difficult . . . which
is a variance not noticed to influence performance on C-tests (Coleman, 1995).
However, since multiple measures are the new vogue, the choice to employ them
during this phase seems frugal. Research has given it high marks (Kokkota,
1988).
Week 21 to end of semester or
school year: Phase 3.
During this optional phase, no
specific structure is recommended. However, each member of the team is offered
the opportunity to select and present new textbook passages for inclusion in
the repertory for that grade. The coach is charged with the responsibility of
naming the paragraphs in a manner commensurate with the original format to give
proper credits to textbook publishers. The coach also presents three related
test questions per week in a manner similar to the following: First, s/he
either creates a keyword cloze test ahead of time (NRL set to “0”) or selects
an existing one with text that explains the reasoning behind the problems and
their solutions. During the session, the coach explains one problem to each of
the team members and charges them to help one another until all three know how
to solve all three. Each session finishes with students calling out the words
as the coach types them over the underscores. Also strongly recommended is the
use of the addition speed test borrowed from V.J. Mortensen, Co., the owner of
Mortensen more than Math™.
NOTE: Because they are tailored
to individuals. phase 3 test results are not archived. It would be difficult to
correlate them.
Legalities
BLUM Enterprises, the hub of the
distributed network, d/b/a COPI.
COPI (ClozeOnlinePlus™
International) is a cooperative consortium of independent contractors who
agreed to function as coaches on select issues of remedial education that can
be enhanced by the use of ClozeOnline™[the test] and ClozeOnlinePlus™[the
program]. BLUM Enterprises is my immediate family. ClozeOnlinePlus™
International (COPI) is the fictitious name created for this project.
My role in the network.
Through the year 2010, I shall
own the rights to ClozeOnline™. My title shall be WEBmaster and CEO, and my
salary that of a University Lecturer with an expense account for
project-related travels. By the end of that period, I shall relinquish my
ownership and title to the most productive director of our consortium [at that
time] who, in turn, will relinquish her/his control at the end of any
subsequent year during which another district director manages to bypass the
prevailing production numbers, meaning the largest number of active groups in
their district’s COP program. I expect to be retained as a consultant and be
kept on a permanent expense account for consortium-related travels.
My initial job-description
charges me to (a) coordinate the expansion of the program (initiate the field
training), (b) maintain the ClozeOnline™ software, (c) orchestrate the full
functioning of the network, (d) act as arbitrator on unresolved project-related
disputes, (e) preside at annual COPI meetings, and (f) oversee the writing of
dynamic online training manuals. (Appendix A)
The role of the Independent
coaches.
Since the buddy concept has
proven to be effective at all levels of learning, coaches are expected to
recruit and field train at least one other retired or volunteer professional
for the duration of one complete 20-week cycle. All coaches are required to be
registered in their school system as a volunteer. NOTE: The early
registering with a county school system will assure licensure by completion of
the field training.
Miscellaneous legalities.
If necessary, BLUM Enterprises
will build and operate a training facility for coaches who wish to come into
the program without being field-trained. (I expect a 20-hour seminar to be
sufficient.) Profits from such training seminars, if any, shall go, in part, to
the support of ISFGW, the International Society of Friendship and Goodwill
(which just last year celebrated its 25th year of service to the
world). Dr. Stanley Drake, its founder, is one of our consortium’s advisors.
BLUM Enterprises is resolved to
establish ties to universities both here and abroad through (a) personal
contacts (educators I had the pleasure of knowing over the years), (b) members
of the American Association of Instructors of Esperanto (AAIE), of which I am
the current Vice-president, (c) members of the International Society of
Friendship and Goodwill, of which I am also the current First Vice-president,
and (d) networked contacts through TESOL, AARP, and organizations such as
EALTA and the U.S. National Tutoring Association.
BLUM Enterprises is agreeing to
be the Fiscal Agent and Security Administrator for moneys received from grants.
As such, BLUM Enterprises is structured to pay those who log their activities
and submit a bill. The rate at which each may bill (for services actually
rendered) is as follows: The WEBmaster (presently the only fulltime position)
is paid at the average University Lecturer rate. The same applies, at a later
date, to any Board of Directors (2 per State), District Directors (1 per school
district), and Academic Coordinators (1 per university) . . . and again, only
for hours logged, journalized, and presented for payment. Coaches (no limit per
school) are paid at the district’s average teacher rate for hours logged after
the first hour each week, starting with the 21st week.
BLUM Enterprises has obligated
itself to summarize and submit, on a yearly basis, all ClozeOnline™ test data
and its correlations to the ACTFL, the American Council on the Teaching of
Foreign Languages, and the COE, the Council of Europe, via EALTA. Their sites are at http://www.actfl.org, http://www.coe.int/t/e/cultural_co-operation/education,
and http://www.ealta.eu.org respectively.
BLUM Enterprises is committed to
(a) charting student progress over time, (b) drawing correlations between that
data and pertinent ACTFL-published norms on aspects such as quality, validity,
and reliability, (c) enhancing our test whenever new technology is developed to
further accommodate students with disabilities, focused specifically on limited
English proficient (LEP) students including conventional ESL students, and (d)
collaborating efforts between and among states to yield approaches that can be
adapted in varied contexts and for effective dissemination of results to
increase the likelihood that the project will contribute to ongoing State
efforts to improve their assessment systems.
Competitive advantages.
Unlike most assessment and achievement tests, ClozeOnline™
is portable. It can be
used for both public and private schools and is presently researched for
possible inclusion in the home-school market (Post, 2004). Participating public schools, including
randomly selected private schools, are
the ones contributing to the data-collection process. After the research
period, those who desire privacy may buy their own software and stay outside
the longitudinal research process. Jeff
Blum, Cisco engineer #5577, was commissioned to support all networked
installations that were not previously set up to run CGI programs. As my son, he is also a permanent member of
BLUM Enterprises and the COPI consortium.
ClozeOnline™[the test] is dynamic in that its
arsenal of sample tests keeps growing as new books are published and new field
coaches, as part of their training, select new passages during phase 1b and
phase 2b of the process. Even those created by the [student] participants are
intended to add to the dynamics of this learning aid. (Peer-produced texts
will be in the folders “grade2peers”, “grade3peers”, etc.)
ClozeOnlinePlus™[the program] solicits from the entire
spectrum of role models, thereby bringing to the developmentally
delayed the extra diversity of skills sometimes needed to add the often missing
elements of private attention, targeted motivation, and free discussion about
culturally diverse issues (such as culture-specific quotes and proverbs found
in so many of our new books and periodicals).
ClozeOnline™ itself, as an automated test, has an almost non-existent
cost factor. This test creator
processes any language using text-based characters.
Detailed instructions for tailoring variations of the C-test are
at the site. All teachers of foreign languages are welcome to request their own
private directory. For the duration of the study, there will be no charge
Because of automation, charting students’ progress over
time will be feasible; partially because it will be pegged annually to
established standards (such as those collected by ACTFL and EALTA), and
partially because it is derived from multiple sources, meaning the whole
spectrum of competency ratings established for schools. Presently, the data
collection process utilizes automated email notifications from all
non-networked directories, i.e., every time a student clicks to finish a test,
an email is sent to the District Director and the student’s teacher, parent, or
handler. Two sample emails are shown as Exhibit 9a and 9b: one shows just the
name, test code, and score (which happens when all restorations were correct),
the other shows also the missed words (which is what is sent when the text was
not fully restored).
Finally, the “Plus” element of the program is bound to help
COPI become and remain among the least expensive community
outreach programs . . . not only because it solicits its coaches from the
spectrum of the workforce that expects probably the least remuneration for
their services and represents the least amount of threat to insecure classroom
teachers, but also because the youngsters trained by these mentors are more
likely to become and then remain productive individuals. Youngsters tend to
relate well to older mentors, especially when they come from outside the school
system (National Mentors Association, 2004). See also http://www.mentoring.org/prevalence_2002
/full_data_set_summary.pdf.
Budget projection
My long-term vision for BLUM Enterprises is to become the
leading vacation-style ESL training facility for successful grandparents and
professionals, who wish to create for themselves a rewarding part-time tutoring
activity in a neighborhood school or industry, regardless of the country in
which they live. This part of the program is expected to be self-supporting
once the training facility is built. The first building, anticipated to be
built near the Apopka Elementary school, is estimated to cost between
$180,000-$190,000 U.S. dollars. Its construction we hope to see underwritten.
Its upkeep will be financed with funds coming in from tuition grants, sublet
charges, and seminar attendees. The orchestrating of the training institute and
the subletting of the Life-Long Learning Center will be the dominant element of
my WEBmaster job description.
The costs of maintaining the amenities for the facility
(structures other than classrooms) will also be born by income generated from
those attending the one-week or two-week training session. No more than
$110,000-$120,000 dollars are anticipated to be needed for the original
necessities such as housing and a dining facility. Hookups for motor homes are
nearby. The classroom structure has been designed and will be scheduled for
construction later this year. We are also researching the feasibility of using converted
school buses for on-site training in schools where facilities are too primitive
to accommodate this online program.
Year 1 (2006-7):
To facilitate the simultaneous launching of the program in
different parts of the English-spoken world (with funds from the grant and the
collective advice from my many scholarly mentors), I will orchestrate the
drafting of an online training program that will utilize current research on
the subject of dealing with breakout groups and tailored remedial training for the
developmentally delayed. The estimated cost for such a setup is expected to be
in the range of $50,000- $75,000 dollars, depending on the cooperation I get
from online universities. It seems as though the National Tutoring Association
will be able to help me keep these costs to a minimum. Adding this
course-development expense to the set-up costs, the budgeted for Year 1 is set
at $385,000 dollars with, hopefully, at least half of the total coming from a
NCLB grant.
Year 2 (2007-8):
I estimate the expenses for this year to be between
$200,000 and $300,000 dollars . . . with one half of that allocated to payroll
for trainers, administrators, and proctors, and the other half used as payroll
for the coaches in the field. This second year of the project presumes about
one thousand students in the program at $150.- each (1000 x $150 = $150,000).
Year 3 (2008-9):
This is the year when attrition may set in, meaning, some
of the coaches may want to move into private enterprises. It will be the rate of
flight from Cohort schools to the private sector that will determine the
ongoing training expenses for the program. With proper recognition and
courteous acceptance by the schools, this flight ratio might turn out to be
minimal. I anticipate that the “make a meaningful difference” element of COPI
will be attractive enough to keep at least some of the coaches where they are
needed the most—in their own neighborhood. The official NCLB budget form is
attached as Exhibit 10.
Miscellaneous.
After training and certification, coaches may either act as
independent contractors within our COPI consortium and tithe in return for
being able to use our network, or they may create their own network with
software purchased from us. While registered with us, we pay them for their
services to Cohort schools (from grant funds); when servicing other contracted
industries, coaches will be remunerated directly by those whom they serve. The
budget for this will be dictated by the size of the grant. While there is money
from the grant, we will pass it on to those who help with the teaching and
coaching in supported schools. NOTE: Though most of our coaches may not need
the extra income, it is still of paramount import to have enough money coming
in to pay those who do need it. I estimate that $100,000 dollars per year,
starting with Year 2, will be the minimum for servicing an average-size school
district’s [1000] stragglers.
In compliance with section 427 of
the General Education Provision Act
(1994), no forbidden restrictions are placed on any aspect of the
selection process. “Unity through Diversity” is woven in and throughout our
entire network. Age, gender, race, color, religion, national origin, or
physical handicaps are non-issues in determining a coach’s licensure and
acceptance into the consortium. To accommodate those whose senses are impaired,
instructions are (will be) available on cassette and disc as well as in
large-scale print and braille (with the braille version being compiled by
consortium member, Dr. Rasheen Abdulabey, Director of Georgia’s Council for the
Blind).
Sec. 97.101(c) through (i) do not
apply to our program in that we do not monitor and archive behavior, we track
only performance on tests created and administered through ClozeOnline™.
References
Babaii,
E., & Ansary, H. (2001). The
C-test: a valid operationalization of reduced redundancy principle? System 29, 209-219. Tehran, Iran: Shiraz University.
Blum,
J. A. (2003). C-tests: their
evolution and future—a way of boosting report cards, including the teacher’s?
Graduate research project, MA TESOL at University of Central Florida. Retrieved January 30, 2004, from http://screenbot.biz/Research/Blum.htm.
Coleman,
J. A. (1995). Progress, proficiency and
motivation among British university language learners. CLCS Occasional Paper No.40. University of Portsmouth.
Daller,
H., & Grotjahn, R. (1999). The
language proficiency of Turkish returnees from Germany: an emperical
investigation of academic and everyday language proficiency. Language, Culture and Curriculum 12(2),
156 – 172. Bristol, UK: University of
the West of England.
English
Only Europe? (2003, October). 2nd Forum. Brussels, Belgium.
Introducing English as the medium of instruction at K.U. Leuven: A comparison
of lecturers’ and students’ language skills, perceptions and attitudes. By Dr.
Lies Sercu, Catholic University of Leuven.
Evans, P. M. (2002,
February). A principal’s dilemmas: Theory and reality of school design. Phi
Delta Kappan, 84(5), 401 – 406.
Reviewing the meaning of Gloria Ladson-Billings’ new book title, Crossing
over to Canaan. (Jossey-Bass Education Series, 2001.)
Foster, M., Lewis, J, &
Onafowora, L. (2003). Anthropology, culture, and research on teaching and
learning: Applying what we have learned to improve practice. Teachers
College Record, 105(2), 261 – 277. Columbia University.
Klein-Braley,
C. (1997). C-tests in the context of
reduced redundancy: an appraisal. Language
Testing, 14(1), 47 – 84.
Kokkota,
V. (1988). Letter-deletion procedure: A
flexible way of reduced text redundancy.
Language Testing, 5(1), 115 – 119.
Post, L., Pugach, M., Hains, A., & Thurman, A. (2002). Multiple points of entry into teaching for urban communities. ERIC Digest. ERICEDRS, 20021201.
Wilson, J. (1994, January).
Urban education: A board member’s perspective. Phi Delta Kappan, 75(5),
382 – 385.
Appendix A
Introduction to Training Manual
Who we are.
We are a potential army of professionals, including
successful grandparents, who volunteer their coaching services for no less than
one hour per week to a neighborhood school or business following the COP format
outlined below. (Detailed instructions are at clozeonline.us/Instructions.htm.)
What we do.
Through ClozeOnlinePlus™, we train a mixture of language
skills by creating, selectively mutilating, and then restoring texts using
Klein-Braley’s C-test (1982) and, during one segment, Kokkota’s L-test (1988),
which is a variation of the classic cloze (Taylor, 1953). During the first half
of the two 10-week phases, the students restore, with their coach’s help, the
previously mutilated texts from their own textbook. During weeks 1 through 5,
the texts are retrieved form directories named “grade2, grade3, etc. During
weeks 11 through 15, the texts are retrieved from the directories named
“grade2-, grade3-, etc. with the (-) after the grade level indicating that the
texts in this directory have been compiled with the NRL setting of 50%(-) . . .
the official setting. During the second half of each phase (weeks 6 through 10
and weeks 16 through 20), we add the dimension of students also “creating” the
text that the program then mutilates and the group proceeds to restore. The
last 20-minute segment of each session is intended to both hone math skills
(through a short speed drill and scenarios using logic) followed by social
confidence training in the form of meaning-based discourse—discussions of Big
Cheese quotes selected by students from the site created for that purpose,
i.e., http://clozeonline.us/cgi-bin/cheesearch.cgi).
At completion of each group’s 20-week catch-up training, two things need to
happen: (1) each team member needs to restore the text in all five of that
week’s selections in the test directory (behind “grade2test, etc.” and accessed
with the [PostTest] link) and (2) the team members, singly or collectively,
need to respond to the invitation to add their name to a signup list for helping
host or support a Big Cheese toastmaster club in their school or neighborhood. (Where
available, real toastmasters will be engaged to help with the setup and
administration of a yearly Speech Craft program for all COP graduates.)
How we do it.
Once registered as volunteers and fluent at
using ClozeOnline™[the test] and ClozeOnlinePlus™[the program], consortium
coaches may offer their services to any school or organization in their school
district and advertise themselves to be licensed members of our COPI
consortium. The only condition is that a coach’s first breakout group in a
given school or organization be coached free of charge for one hour per week
for twenty weeks and attempts be made to start new break-out groups after every
fifth hour, which, if successful, raises the number of volunteer hours to 50
during the first 20 weeks (5+10+15+20). After these 20 weeks, all hours may be
billed at that district’s average adjunct rate. In general, the billing and
paying for these extra hours is done directly when the client is a private
school or industry; it is done through us when the school meets the criteria
of the No-Child-Left-Behind Act.
Suggestions and sample monologues from the Training Manual
On “look for the good and praise it”:
“My spiritual mentor, way back when I was still a
struggling foreigner, told me that one day I would grow into the habit of
‘looking for the good and praising it’. I didn’t buy into that Pollyanna
concept for a long time; I saw too much crime and hatred around me. In fact, it
wasn’t until years later—when I was led to reason my way into believing that
there had to be some sort of god or spiritual supremacy—that I saw what Dr.
Evans was trying to tell me. Naturally, I don’t expect you to relate to this
either; you’re still much too young. But, I wanted you to know why I keep
looking for the good in everything, including you, if you let me. Are you OK
with me complimenting you when you deserve it?”
Horse manure and pony story.
“In some neighborhoods, I know it must be awfully hard to
find the proverbial pony when all you see around you is a big pile of horse
manure, but I promise you, if you hang in with me and learn the craft of
helping other team mates, you’ll be upping your chances to find one . . . or at
least up your chances of one day earning the extra money so you can move to a
place where it’s a bit easier or the pile is a bit smaller. Personally, I am
fully convinced that there is an underlying harmony even in chaos; meaning,
that there’s always a pony . . . either right in or somewhere near the pile.”
On positive vs. negative reinforcement.
Negative reinforcement is a powerful motivator. In fact, some of my most prized
accomplishments were the result of insults and derogatory feedback. The problem is, in a public enterprise this
negative good-cop/bad-cop byplay scenario is frowned on. It takes advantage of
someone’s emotional ignorance. We must
keep our coaches away from such criticism by simply refusing to succumb to it. Yes,
learn about negative feedback but keep it out of the consortium. Even with competition:
We don’t instigate it; we just let it happen. Why? Because most of the developmentally delayed
have already had more than their share of criticism. We need to stay away from
negative motivation. On a day when the
obvious accomplishment of a new task didn’t get to set a natural positive tone,
we must supply a make shift version of it in the form of sincere praise for
overall accomplishment. Each session
must conclude with positive individual responses to and from the students
through targeted (personalized) feedback questions! (The use of our speed
test in doing simple math will help with some of this.)
On negotiating meaning.
Let your team know that
you're open to negotiation . . . on anything. I used to give my children six
chances to sell me on an idea. If they were unable to do so, they were expected
to drop it for a while (which varied in length with the nature of the subject).
Researchers are almost unanimous in their agreement on this subject:
negotiating meaning, no matter how it is done, hones skills nothing else can do
quite as well. Make sure you allow at least 10 minutes during each session for
this aspect of growing up.
On writing notes and making
calls.
Successful writers of old, like Mark
Twain, are said to have written an average of three notes a day for most of
their adult life. Good coaches do the same, at least while they’re hungry. I
did it for most of my career as a coach in the ballroom dance industry. At 60+
dollars per hour, students only come back when they feel they’re getting their
money’s worth. A short three-sentence note does wonders. Naturally, three calls
per day will do the same. The most successful coaches, of course, do both:
three calls and three notes a day. While you’re new, it would really help if
you started this habit. The chances are you will be amazed and pleased with the
results.
Appendix B
Current Research
Of the many studies and peer
reviews of the C-test, I wish to share these four convenience samples: (1) an
impressive study titled, “Progress, proficiency and motivation among British
university language learners” (Coleman, 1995); (2) a study of the C-test’s
validity as it relates to the operationalization of the reduced redundancy
principle (Babaii, & Ansary, 2001); (3) Klein-Braley’s own appraisal of the
now 15-year-old C-test in the context of reduced redundancy (1997); and (4)
Kokkota’s Letter-deletion procedure, including his introduction of the L-test,
a flexible way of reduced text redundancy (1988). Many more are at
clozeonline.us/Research/index2.html
1. Coleman’s study at the University of Portsmouth was
significant in that it dealt with large numbers of participants on the subject
of “Progress, proficiency and motivation among British university language
learners” (1995). They were learners of French, German, Spanish, English and
Russian in British higher and further education. Year 3 students, since most
spend this year abroad, were excluded. Coleman reports that there were 3,000
language learners in 12 UK universities and 8 other institutions for Stage 2
(completed) and 20,000 for Stage 3 (in progress in six EU countries). This is a
joint project between the universities of Portsmouth, Duisburg, and Bochum. It
received its funding under the LINGUA program from the Commission of the
European Communities, the British Council, and the German Austauschdienst
(under their Academic Research Collaboration Program). Its members are at uni-duisburg.de/FB3/ANGLING/FORSCHUNG/ELPS.HTM.
The instrumentation consisted of C-tests with associated questionnaires
to explore biographical data, language-learning background, residency abroad,
attitudes, motivations, strategies, and grammatical knowledge. Coleman said of
the C-test, “The C-test is easy to construct, quick to administer, and easy,
objective, and unambiguous to score. It is also extremely reliable. To provide
a snapshot of a learner’s general competence in a foreign language, the C-test
is unrivaled.” His details describe how Phase 1 was a small-scale survey
that “showed widespread ignorance of theoretical constructs underpinning
language testing among university staff.” It continues by listing the divergent
performances and rates of progress across different institutions of Phase 2,
and examines features of learner motivation and of how it evolves during
university studies. His results showed that there are (1) both similarities and
dissimilarities between learners of different languages, (2) that there is a
shift in emphasis in motivation between students in different years of study
(with significant changes following residence abroad), (3) that there are major
differences between UK students and their counterparts in Germany and Austria,
(4) that motivation changes with age (with sex-based differences being
marginal), and (5) that there is a slight but measurable relationship between
integrative and/or resultative motivation and foreign language proficiency.”
Factor analysis confirms expectations of the principal motivations underlying
student responses. He finishes, commenting that, “This three-stage study, using
the C-test developed at the University of Duisburg, is closest to what I
believe needs to be done in select schools and universities in the U.S.” This is what my study intends to accomplish.
2. Babaii and Ansary (2001) at Shiraz University in Tehran
was a study involving engineering students taking an English for Science and
Technology course. There were 10 females and 22 males of heterogeneous
background. Their ages ranged from 20 to 30. The purpose was to explore whether
or not the C-test, as it was claimed, serves as a valid operationalization of
the reduced redundancy principle (RRP). The instrumentation used was the 1995
TOEFL (without the Listening Section) and the recommended format for the C-test
(Klein-Braley, 1997), i.e., an original set of eight short texts (that were
later narrowed down to five) with 20 mutilations each. . . . Furthermore, to
adjust to the overestimated reliability of the C-test (resulting from the
application of measures of internal consistency), these selected five were
again treated as Super items with Crombach’s alpha formula. Its reliability was
estimated at a=0.93. Under details, Babaii
and Ansary state that an attempt was made “to investigate the frequency and
type of micro- and macro-level cues that EFL learners employ to restore the
mutilations in the C-test.” The two tests were administered concurrently. The
retrospective data collection was done orally. The results: Analysis of the protocols
indicated that “there exist four major types of cues with varying frequencies:
(1) automatic processing; (2) lexical adjacency; (3) sentential cues; and (4)
top-down cues” . . . automatic processing (n=110, 16.6%) because the
intact half of a mutilated word reminds an examinee of some familiar and highly
frequent words and expressions as in [kn _ _] and [of co _ _ _ _] ([know] and
[of course]); lexical adjacency (n=363, 54.9%) through reliance on the
syntactic and lexical cues provided by the words immediately before and/or
after the mutilated word; sentential cues (n=148, 22.4%) because of
reliance on the grammatical features such as tense, referencing, and cohesive
ties with surrounding sentences; and through top-down cues (n=40, 6.1%)
such as background knowledge, topic familiarity, and similarity chains . . .
items discovered elsewhere in the text. Their conclusion: With a certain degree
of latitude, “C-testing is a reliable and valid procedure that mirrors the
reduced redundancy principal”© 2001, Elsevier Science Ltd. Their C-test is at
www.clozeonline.us/Babaii.htm.
The article’s keywords are: C-test, Language proficiency, Reduced redundancy
principal, Language testing, and Verbal protocol analysis.
3. Klein-Braley (1997), of course, has the best summaries
of how her C-test compares to others available in Europe, including the
University of Duisburg’s DELTA scores. That study’s size was small (n=81), but
all completed the entire battery of tests. The purpose of this comparison of
the various reduced redundancy approaches was to compare the university’s own
C-test to the rest. The battery of tests consisted of 2 cloze tests, 1
four-part C-tests, 2 four-choice cloze tests, 2 cloze-elide tests, and the
dictation portion of the DELTA test. CLOZE1 used the fixed 7th
word deletion rate; CLOZE2 deletions were made using a random number
table. (Exact scoring was used. Each had 20 items.) The 100 possible points on
the C-test were derived from four short paragraph-style texts with 25
mutilations each (C1, C2, C3, & C4). Of the
multiple-choice cloze tests (MC1 & MC2), one used the 26 blanks
specified by Jonz (1976); the other used the 22 blanks format used by Manning
in his (1986) study. The cloze-elide tests (CE1 & CE2) used
ANIMAL MIGRATION and ARCHEOLOGY, which were also taken from Manning’s (1986)
study. The DELTA dictation test (DICT)
assigned 50 points with 1 point subtracted for each erroneous deletion or
non-deletion. She concluded by first citing averages for the items measured,
e.g., that a difficulty ranking of (P = .5 is ideal). Knowing
this, helps understand why the C-TEST’s (.52) is better than those of
ALLCE (.46), ALLMC (.60), and ALLCLOZE at (.27). A lesser superiority was shown
in respect to reliability (KR-21). DICT (.94) and DELTA (.93) were rated
higher, but the C-TEST’s (.85) is still better than those of ALLCE
(.75), ALLCLOZE (.66), and ALLMC (.55). On rank of usability (performance
and ease of use), the C-TEST ranked higher than DICT, ALLMC, and
ALLCLOZE tied with ALLCE. The C-test ranked in the middle in ease of
construction, with DICT and ALLCLOZE rated higher and ALLCE and ALLMC rated
lower. (This less than perfect ranking should be remedied through our online
administration of it.) The C-test’s
rating on scoring should also move from second place to first with the
help of ClozeOnline™ (ALLMC, C-TEST, ALLCLOZE, and DICT). The C-test’s overall
rating is, as was expected, higher than the rest with DICTation being the only
one close in rating.
4. And I liked Kokkota’s study (1988), because it had
impressive numbers and offered a viable solution to some of the problems in
solving classical cloze tests. His participants were 18/19-year-old Estonian
first-year students at the Tallinn Polytechnic Institute. There were six groups
of 51 – 56 testees each, with each group taking eight different tests in one
term. Under Details, Kokkota states that the average English proficiency levels
and score distribution curves were equalized using the All-Union standardized
test ITA-83. ITA-83 consisted of 135 items: vocabulary, 50 items;
multiple-choice cloze, 50 items; grammar, 25 items; reading comprehension, 10
items. When pre-tested in 75 Soviet universities and institutions on about
7,500 testees (with ITA-82), it had reliabilities of .89 – .91 on different
forms, and validities of .86 – .89 (Einväli et al. 1986). The inter-item distance (IID) and the number
of undeleted letters (NUL) of the two L-tests were between those of the classic
cloze and the standard C-test. The intent was to monitor the extent of
influence NUL and IID had on item difficulty. (I renamed Kokkota’s NUL to
read NRL for number of remaining letters. The other was reported to be
confusing.) For reliability purposes, each group was exposed to each test
twice, each time taking a different test format or the same test format with a
different item deletion rate. An interval of about two months was left between
the two administrations to diminish the memorization effect. In each case, the
classic cloze or the test with the lower item deletion rate was administered
before the C-test or the L-test. Conclusions: The letter-by-letter deletion of
LDP is [often] a more flexible and powerful means of controlling reduction of
text redundancy than a cloze procedure or the C-test. It is also more general
than either, because (1) it is possible to derive both cloze procedure and
C-tests from LDP; (2) the parameters of LDP are between those of the cloze
procedure and the C-test, which are its extreme modifications; and (3) LDP has
a number of the advantages of cloze, RD and C-test, while overcoming some of
their disadvantages.” The facility curve of the L-tests was between the C-test
and classic cloze test curves, with the C-test facility curve at 60 – 70 and
the cloze curve from 36 – 48. Kokkota regards RD, the rational deletion system
(Bachman, 1982, 1985), and the C-test (Klein-Braley, 1985) as “the major
improvements on the cloze procedure” of the decade. His aim was to outline a
comparatively new letter-deletion procedure (LDP), which would flexibly match
the advantages of RD and C-test, while avoiding their main disadvantages.
Others, including those on cohort projects, are at clozeonline.us/Research/index2.html. The blinking star in front of a link leads to that study's references.
Appendix C
Feedback data
(printed in student’s native language, signed by parent, and collected
after the 20th hour)
- Of the four 5-week phases (1a[1-5],
1b[6-10], 2a[11-15], & 2b[16-20]), I found | 1a | 1b | 2a | 2b |
(circle one) to be the easiest and phase | 1a | 1b | 2a | 2b | (circle
one) to be the most difficult.
- My favorite among these four phases was
| 1a | 1b | 2a | 2b | neither | (circle one).
- My least favorite was | 1a | 1b | 2a |
2b | neither | (circle one).
- Of the other two 5-week periods, I
preferred | 1a | 1b | 2a | 2b | neither | (circle one).
- My confidence has improved | very little
| some | considerably | more than one letter grade |
- My test scores have improved | very
little | some | considerably | more than one letter grade |
- My reading comprehension has | improved
| stayed the same | I don’t know | (circle one).
- I am glad I was selected for this study
| yes | no | not applicable | (circle one).
- My parent(s)/custodian(s) were informed
about my participation | yes | no | not applicable |
- If I am allowed to continue with phase 3
of this training, I would like to be a | team leader(proton) | team
member(electron) | assistant coach(neutron) | (circle one).
- What I would really like is
_____________________________________________________
- What I think could be changed is
________________________________________________
Other comments:
___________________________________________________________________
_________________________________________________________________________________
_________________________________________________________________________________
School/Teacher: ________________________________ _________________________________
Student/E-mail: ________________________________ ___________@___________.com
Person who filled this out: | my parent/custodian | I did it myself |
(circle one).
Of the following three mascots, I relate best to the | 1st | 2nd
| 3rd | (circle one).
Signature _____________________________
Appendix C: Cohort Schools in Central Florida
A PowerPoint presentation (large pictures, slow loading!)