Unfortunately, the legislators themselves have estimated that this Race to the Top prize will not be not be enough to cover the additional expenses of creating new tests, training, deploying, grading and reporting the results of the tests that SB 736 requires. There is immense cost in creating fair standardized tests specific to every course’s curriculum. SB 736 doesn’t provide any additional funding and most citizens are unaware that the additional burden of creating these tests will come out of the local school district’s shrinking budget. It is another case of legislate something new but providing the money to make it happen.
Many of my colleagues believe that SB 736 (and last year’s SB 6) is all about publishers making money from standardized tests and the grading of those tests. A well-placed team of lobbyists can be very influential.
What is happening to curiosity-driven inquiry? With all the stress of testing, are children going lose yet more of their childhood? One question teachers and parents are asking is Do we really want instructional time devoted to more standardized style testing?; but I think the real question is What does all this testing tell us?
Concern #1 What are we testing?!
Not everything that counts can be counted, and not everything that can be counted counts. Albert Einstein
When we begin to look at assessment, one of the first thoughts that come to my mind is a quote often attributed to Albert Einstein: Not everything that counts can be counted, and not everything that can be counted counts. However, it is probably better to credit sociologist William Bruce Cameron who wrote something very similar in his 1963 book: Informal Sociology: A Casual Introduction to Sociological Thinking (1963). It appears to have made some impression on Albert Einstein who apparently quoted it and became associated with this thought. But I digress...
Concern #2 What do the test numbers really mean?!
Education is not as simple as manufacturing widgets in a factory. Most public school teachers cannot determine the type of student that crosses their threshold each August. Unlike other vocations, educators have very little control on the raw goods they are given and the events of any given household, community or school are not consistent year to year. Measuring the success of a teacher following a year of instruction is much more complex than developing a quality product in a factory assembly line.
I am trained as a scientist. Science relies on numbers to prove or disprove a hypothesis. An experiment may provide results that are different from a control group but not statistically significant. The term Statistically Significant is used when the results are unlikely to have occurred by chance alone. Cause and Effect is very difficult to establish, because observed results are not always a direct result of the treatment.
In the physical sciences, cause and effect is easier to establish because physical science subjects “behave” consistently according to the laws of energy, principles of mass, etc. Sciences that deal with living things introduce much more complex systems. There are so many variables (many hard to quantify) i.e., internal vigor, hormone levels, diet, previous experience. The complexity of living things make it very difficult to establish cause and effect, and this complexity is even greater with human subjects. When we say a teacher should not return to their job next fall because their students didn’t make significant improvement on their standardized test scores, we are implying a cause and effect relationship that doesn’t necessarily exist.
Let’s suppose that we know that the end of year test is good. The student did learn something. The test measures (correctly) that gains were made. Does this mean that the teacher was successful. Not unless there are significant numbers and significant gains and repeated success with a wide variety of students. It is a very large leap from the test correctly evaluating the student to the test evaluating the teacher in a cause and effect relationship.
I am not a statistician, and I don’t play one on TV, but I really question the validity of relying on test score improvement each year as the primary measure of how effective a teacher is. Here are some points to ponder:
- Currently, teachers have about 50 hours (per subject) to ‘move the needle’ before the FCAT is taken. Subtract classroom management, unavoidable student absences, and various interruptions, the actual time that teachers have to ‘produce learning’ is not as great as ‘a year of instruction’ might indicate.
- Although teachers are very important, the range and potential effect of other variables in the learning styles, foundational experiences, home, and emotional environment can overwhelm the effect of a charismatic, logical, knowledgeable, and talented learning facilitator. The total time students are ‘exposed’ to other stimuli far outweighs the time they are exposed to the teacher’s instruction.
- Because students have different learning styles, our very best teachers are most successful when using a specific teaching style to teach them. The factory model of education that standardized testing assumes and reinforces makes it much more difficult for them to operate in the range of learning styles. If our teachers are supposed to deliver ‘differentiated instruction’ then assessment must be differentiated as well. This is at odds with the idea of standardized testing which says that success is measured by one standard applied to all in the same way.
- In statistical language, I wonder about sample size, the randomness of selection of subjects (sampling distribution), disqualification of outlier results, standard deviation, significance of small gains or small losses (effect size). Human learning research is confounded by many more variables and measurement issues than other sciences. That is why behavioral and social sciences are called a ‘soft science’ and human learning is at the farthest end of the soft science <–> hard science continuum. Are true social science statisticians going to review this process to determine if there is statistically sound reason to award, penalize, or fire teachers based on their class’s scores on one test??!!
- My final concern has to do with the internal reliability of the instrument. SB 736 (if signed and not blocked by courts) starts July 1st. End of course testing has begun and judgements will be made next year on the results of these tests. Assuming each teacher will be giving a pre-test for their course and post-test, someone has a lot of work to do (with no money) prior to the start of school in August. What is kind of quality are we to expect of these tests? This bill is intended to be used for hiring and firing decisions next year. It is unrealistic that these tests will be an equitable, fair assessment for such huge consequences.
When SB 736 ties a teacher’s pay (or even more seriously, the teacher’s employment) to their students’ test grades we need to be absolutely certain that cause and effect can be proven in a statistically significant fashion. We are talking about a teacher’s livelihood, their future; these are real people with families they are supporting. Personally, I am not confident that our legislators have taken these validity concerns into consideration. When we make a law, it must be fair to all people under all conditions. All courses, all teachers, all grade levels in an equitable, humane, fair fashion.
I understand the frustration of the public. There are bad educators that we need to be able to get out of the classroom. It would be nice to reward the efforts of excellent educators. However, annual test gains are not the answer to a very, very complex problem.
I have thoroughly enjoyed over 25 years as a career educator. I love being with students, watching them learn and learning with them. I have been honored several times on local regional and national scale as an educator. My students have achieved great things. Even though it has been difficult financially to support my family solely on my public school salary, I feel fulfilled through my career.
However, I am very reluctant to recommend education as a career to my brightest students– given this obsession with test scores. And I doubt they would be persuaded any way.
How much did the technology savviness of President-elect Obama (and his staff) affect his campaign’s success?
If it was a major factor...
What does this say about the kind of education our students should be receiving?
Are our students sitting in classrooms more like Barak Obama’s technology infused world,
or like the classroom that John McCain sat in?
Obama's High-Tech Win Holds Lessons for Ed
His campaign's unprecedented use of technology shows schools and colleges how to inspire communities, mobilize support
Fri, Nov 14, 2008
By Maya T. Prabhu, Assistant Editor
As educators continue to reflect on President-elect Barack Obama's historic victory in the Nov. 4 election, many are looking at the Obama campaign's unprecedented use of technology to mobilize support and wondering what lessons their schools and colleges might learn from his success.
Observers have credited Obama's success in no small part to his campaign's innovative use of technology--including blogging, text messaging, and online social networks--to connect with younger voters and get them excited about politics and the election.
"We've done a huge amount of organizing using the internet, and we've used new technology in ways that really captured young voters' attention," Obama spokeswoman Kirsten Searer told the Associated Press (AP) for a Nov. 3 story.
Obama's Facebook page had 2.6 million supporters, and he had 850,000 MySpace friends. The campaign also relied on text messages to communicate with voters, finding that short blurbs were an effective way to advertise campaign stops and early voting locations.
Exit polls had the youth turnout, voters between the ages of 18 and 29, at its highest since 1972--and 66 percent of these young voters cast their votes for Obama.
Young voters reportedly accounted for 18 percent of the 133 million votes cast. This occurred in a year when a Pew Research Center poll found that nearly half of Americans between 18 and 29 used the internet as their major source of election news in 2008. Only 17 percent of youth voters said they got their election coverage from newspapers.
more of the article here...
I have played around with this website a number of times and really enjoyed the fresh way of looking at an essay, news article or report. The website is called Wordle- you copy and paste the verbal content (words) of a section of writing, it does word counts and determines the most commonly written or spoken words. The website then creates a visual representation (graphic) of these words with the words that were mentioned most frequently proportionately larger than words that were just mentioned once or twice.
So last night we had a Vice Presidential Candidate Debate: Joseph Biden versus Sarah Palin. Viewers were awash in a sea of words, and one might want to know at the end of the debate, what were the most frequently spoken words. Wordle to the rescue! BBC took the transcripts of the two candidates, let Wordle perform it’s magic and here is what they got in return:
(click the graphics below to view the full Wordle)
addendum: Stephanie Cheney visited and left a comment below recommending this blog post for more Wordle in the Classroom Ideas
What is it that educators really are battling with? Educators in battle with a culture that devalues education in favor of appearance, escapism, sports, sex, entertainment, etc. Students that don't know why they are at school – except because society has sentenced them to what is arguably no longer a privilege.
And here we are again in the cycle that asks that impossible question - how successful are you as an educator? What is the measure of your success? It has been said before but is worth repeating:
The business of education is not like other industries that can be measured merely with a return-on-investment spreadsheet that declares the number of widgets produced - and the profit margin on those widgets.
Yet we rely on the best measures that we have – the standardized test. And to prepare these students – because we all know that test taking is a skill – we have spent many of our instructional hours preparing, reviewing, and practicing taking tests. So what does it really mean?
With a tip of the hat to fellow educator and blogger: Barry Bachenheimer - http://plethoratech.blogspot.com/2007/05/1620.html
When you see those faces light up or hear the excited gasps or "COOL!" from the students it gladdens the heart of a teacher. There are few things in life are more gratifying than sharing the power of good software and effective skills with students and peers. In my mind's eye, it is like watching a rose blossom bloom in hyperspeed. I find myself wanting to pause indefinitely to enjoy the fragrance of that rose.
What were they learning? How to create a webpage with a table of over 100 different background colors last week. How to use Fireworks to create original graphics this week. Technology is a tool that can empower creative energies within. It is like freeing the captive spirit... opening the mundane jail of the soul detained.
It never ceases to intrigue me... the thought I stand amongst a classroom full of independent, unique lucid beings - attempting to alter the biochemistry of their cerebral cortex.
As the leader, facilitator, director and guide in their classroom, I have a tremendous responsibility. I must carefully structure the emotional atmosphere of my classroom to encourage thinking, challenge misconceptions, develop positive social interaction and promote teamwork.
For instance, how many students (or adults for that matter) know how to set up style sheets in a word processor? How many different tabs are there? How do you create a spreadsheet that will calculate averages or do item counts? What data is properly illustrated using a line chart versus a pie chart? What are the four main categories of fonts, how are they different, and where are they appropriately used in a document? What are the key differences between a spreadsheet and a database? When should we use a drop list or create radio buttons?
These are the types of questions that students should be able to answer after the first nine weeks in computer applications. Hopefully they will serve them well beyond this year in Computer Applications.