mathematics

International Comparisons of Achievement

Two large-scale international studies have become established to compare countries' performance in the core subjects of literacy, mathematics and science.

TIMSS: Trends in International Mathematics and Science Study

TIMSS is an international study involving 50 countries that assesses math and science achievement at four year intervals. It has been running since 1995. Students are assessed in the 4th and 8th years of school, and in their final year. The next assessment round will be in 2007.

The study uses four benchmarks (advanced, high, intermediate, low) to gather a more complete picture of trends within a country. Thus we can not only approve high performing countries like Singapore, Chinese Taipei, Korea, and Hong Kong, for having about 1/3 or more of their 8th grade students reach the advanced benchmark in mathematics, and about 2/3 to 3/4 reaching the high benchmark, but we can also note, for example, that although the Netherlands doesn't have high numbers reaching the advanced level (some 10% of 8th graders and 5% of 4th graders), it does at least do an excellent job of educating all its students, since 97% of its 8th graders and 99% of its 4th graders reach the low benchmark. It also enables us to spot trends across time — for example, in general, countries have improved their levels at the lower end, but not at the high end.

Mathematics

Grade 8 Advanced Benchmark

Students can organize information, make generalizations, solve non-routine problems, and draw and justify conclusions from data. They can compute percent change and apply their knowledge of numeric and algebraic concepts and relationships to solve problems. Students can solve simultaneous linear equations and model simple situations algebraically. They can apply their knowledge of measurement and geometry in complex problem situations. They can interpret data from a variety of tables and graphs, including interpolation and extrapolation.

Grade 8 High Benchmark

Students can apply their understanding and knowledge in a wide variety of relatively complex situations. They can order, relate, and compute with fractions and decimals to solve word problems, operate with negative integers, and solve multi-step word problems involving proportions with whole numbers. Students can solve simple algebraic problems including evaluating expressions, solving simultaneous linear equations, and using a formula to determine the value of a variable. Students can find areas and volumes of simple geometric shapes and use knowledge of geometric properties to solve problems. They can solve probability problems and interpret data in a variety of graphs and tables.

Grade 8 Intermediate Benchmark

Students can apply basic mathematical knowledge in straightforward situations. They can add, subtract, or multiply to solve one-step word problems involving whole numbers and decimals. They can identify representations of common fractions and relative sizes of fractions. They understand simple algebraic relationships and solve linear equations with one variable. They demonstrate understanding of properties of triangles and basic geometric concepts including symmetry and rotation. They recognize basic notions of probability. They can read and interpret graphs, tables, maps, and scales.

Grade 8 Low Benchmark

Students have some basic mathematical knowledge. The few items at this level provide some evidence that students can do basic computations with whole numbers without a calculator. They can select the two-place decimal closest to a whole number. They can multiply two-place decimal numbers by three-place decimal numbers with calculators available. They recognize some basic terminology and read information from a line on a graph.

Grade 4 Advanced Benchmark

Students can apply their understanding and knowledge in a wide variety of relatively complex situations. They demonstrate a developing understanding of fractions and decimals and the relationship between them. They can select appropriate information to solve multi-step word problems involving proportions. They can formulate or select a rule for a relationship. They show understanding of area and can use measurement concepts to solve a variety of problems. They show some understanding of rotation. They can organize, interpret, and represent data to solve problems.

Grade 4 High Benchmark

Student can apply their knowledge and understanding to solve problems. Student can solve multistep word problems involving addition, multiplication, and division. They can use their understanding of place value and simple fractions to solve problems. They can identify a number sentence that represents situations. Students show understanding of three-dimensional objects, how shapes can make other shapes, and simple transformation in a plane. They demonstrate a variety of measurement skills and can interpret and use data in tables and graphs to solve problems.

Grade 4 Intermediate Benchmark

Students can apply basic mathematical knowledge in straightforward situations. They can read, interpret, and use different representations of numbers. They can perform operations with three and four-digit numbers and decimals. They can extend simple patterns. They are familiar with a range of two-dimensional shapes and read and interpret different representations of the same data.

Grade 4 Low Benchmark

Students have some basic mathematical knowledge. Students demonstrate an understanding of whole numbers and can do simple computations with them. They demonstrate familiarity with the basic properties of triangles and rectangles. They can read information from simple bar graphs.

from http://timss.bc.edu/PDF/t03_download/T03_M_Chap2.pdf

2003 Performance

In 2003, the international averages were:

Benchmark Grade 4 Grade 8
advanced 9% 7%
high 33% 23%
intermediate 63% 49%
low 82% 74%

There is quite a wide variation around these means. For example, Singapore is head and shoulders above everyone, scoring 44% advanced, 77% high, 93% intermediate, 99% low at grade 8, and 38% advanced, 73% high, 91% intermediate, 97% low at grade 4. The only countries that come close are also Asian: Chinese Taipei, Hong Kong, Japan, and the Republic of Korea (for Grade 8; grade 4 figures weren't available). The highest of the remaining countries at grade 8 was Hungary at 11% advanced, 41% high, 75% intermediate, 95% low, and at grade 4 England at 14% advanced, 43% high, 75% intermediate, 93% low -- a substantial difference in results! But still a vast improvement over those at the bottom of the table. Here's 2 tables roughly grouping countries, using the top performing country in each group as a benchmark:

Grade 8 advanced high intermediate low
highest performing countries (Singapore) 44% 77% 93% 99%
Singapore, Chinese Taipei, Republic of Korea, Hong Kong, Japan        
above average countries (Hungary) 11% 41% 75% 95%
Hungary, Netherlands, Belgium, Estonia, Slovak Republic, Australia, United States        
slightly below average countries (Malaysia) 6% 30% 66% 93%
Malaysia, Russian Federation, Israel, Latvia, Lithuania, England, New Zealand, Scotland        
below average countries (Romania) 4% 21% 52% 79%
Romania, Serbia, Sweden, Slovenia, Italy, Bulgaria, Armenia        
really below average countries (Cyprus) 1% 13% 45% 77%
Cyprus, Moldova, Macedonia, Jordan, Indonesia, Egypt, Norway, Lebanon, Palestinian National Authority, Iran, Chile, Philippines, Bahrain, South Africa, Tunisia, Morocco, Botswana, Saudi Arabia, Ghana        

note that the range at the bottom end is still very large; although most of the countries in the last category at least got over 50% to the low benchmark, 8 did not -- the worst only got 9% through.

Grade 4 advanced high intermediate low
highest performing countries (Singapore) 38% 73% 91% 97%
Singapore, Hong Kong, Japan, Chinese Taipei        
above average countries (England) 14% 43% 75% 93%
England, Russian Federation, Belgium, Latvia, Lithuania, Hungary        
slightly below average countries (Cyprus) 6% 30% 66% 93%
Cyprus, United States, Moldova, Italy, Netherlands, Australia, New Zealand        
below average countries (Scotland) 4% 21% 52% 79%
Scotland, Slovenia, Armenia, Norway        
really below average countries (Philippines) 1% 13% 45% 77%
Philippines, Iran, Tunisia, Morocco        

note that there are substantially fewer countries' results available at grade 4

You can find out more about international comparisons of achievements in mathematics, science and reading at the official website for TIMSS (Trends in International Mathematics and Science Study) & PIRLS (Progress in International Reading Literacy Study): http://timss.bc.edu/

The full 2003 Mathematics Report can be downloaded at: http://timss.bc.edu/timss2003i/mathD.html

Science

Grade 8 Advanced Benchmark

Students demonstrate a grasp of some complex and abstract science concepts. They can apply knowledge of the solar system and of Earth features, processes, and conditions, and apply understanding of the complexity of living organisms and how they relate to their environment.

They show understanding of electricity, thermal expansion, and sound, as well as the structure of matter and physical and chemical properties and changes. They show understanding of environmental and resource issues. Students understand some fundamentals of scientific investigation and can apply basic physical principles to solve some quantitative problems. They can provide written explanations to communicate scientific knowledge.

Grade 8 High Benchmark

Students demonstrate conceptual understanding of some science cycles, systems, and principles. They have some understanding of Earth’s processes and the solar system, biological systems, populations, reproduction and heredity, and structure and function of organisms. They show some understanding of physical and chemical changes, and the structure of matter. They solve some basic physics problems related to light, heat, electricity, and magnetism, and they demonstrate basic knowledge of major environmental issues. They demonstrate some scientific inquiry skills. They can combine information to draw conclusions; interpret information in diagrams, graphs and tables to solve problems; and provide short explanations conveying scientific knowledge and cause/effect relationships.

Grade 8 Intermediate Benchmark

Students can recognize and communicate basic scientific knowledge across a range of topics. They recognize some characteristics of the solar system, water cycle, animals, and human health. They are acquainted with some aspects of energy, force and motion, light reflection, and sound. Students demonstrate elementary knowledge of human impact on and changes in the environment. They can apply and briefly communicate knowledge, extract tabular information, extrapolate from data presented in a simple linear graph, and interpret pictorial diagrams.

Grade 8 Low Benchmark

Students recognize some basic facts from the life and physical sciences. They have some knowledge of the human body and heredity, and demonstrate familiarity with some everyday physical phenomena. Students can interpret some pictorial diagrams and apply knowledge of simple physical concepts to practical situations.

Grade 4 Advanced Benchmark

Students can apply knowledge and understanding in beginning scientific inquiry. Students demonstrate some understanding of Earth’s features and processes and the solar system. They can communicate their understanding of structure, function, and life processes in organisms and classify organisms according to major physical and behavioral features. They demonstrate some understanding of physical phenomena and properties of common materials. Students demonstrate beginning scientific inquiry knowledge and skills.

Grade 4 High Benchmark

Students can apply knowledge and understanding to explain everyday phenomena. Students demonstrate some knowledge of Earth structure and processes and the solar system and some understanding of plant structure, life processes, and human biology. They demonstrate some knowledge of physical states, common physical phenomena, and chemical changes. They provide brief descriptions and explanations of some everyday phenomena and compare, contrast, and draw conclusions.

Grade 4 Intermediate Benchmark

Students can apply basic knowledge and understanding to practical situations in the sciences. Students demonstrate knowledge of some basic facts about Earth’s features and processes and the solar system. They recognize some basic information about human biology and health and show some understanding of development and life cycles of organisms. They know some basic facts about familiar physical phenomena, states, and changes. They apply factual knowledge to practical situations, interpret pictorial diagrams, and combine information to draw conclusions.

Grade 4 Low Benchmark

Students have some elementary knowledge of the earth, life, and physical sciences. Students recognize simple facts presented in everyday language and context about Earth’s physical features, the seasons, the solar system, human biology, and the development and characteristics of animals and plants. They recognize facts about a range of familiar physical phenomena — rainbows, magnets, electricity, boiling, floating, and dissolving. They interpret labeled pictures and simple pictorial diagrams and provide short written responses to questions requiring factual information.

from http://timss.bc.edu/PDF/t03_download/T03_S_Chap2.pdf

2003 Performance

In 2003, the international averages were:

Benchmark Grade 4 Grade 8
advanced 7% 6%
high 30% 25%
intermediate 63% 54%
low 82% 78%

There is, again, wide variation around these means. Singapore is again head and shoulders above everyone. The only countries that come close are also Asian: Chinese Taipei, Hong Kong, Japan, and the Republic of Korea (for Grade 8; grade 4 figures weren't available). The highest of the remaining countries at grade 8 was Hungary at 11% advanced, 41% high, 75% intermediate, 95% low, and at grade 4 England at 14% advanced, 43% high, 75% intermediate, 93% low — a substantial difference in results! But still a vast improvement over those at the bottom of the table. Here's 2 tables roughly grouping countries, using the top performing country in each group as a benchmark:

Grade 8 advanced high intermediate low
highest performing countries (Singapore) 33% 66% 85% 95%
Singapore, Chinese Taipei        
above average countries (Republic of Korea) 17% 57% 88% 98%
Republic of Korea, Japan, Hungary, England, Hong Kong, Estonia        
slightly above average countries (United States) 11% 41% 75% 93%
United States, Australia, Sweden, New Zealand, Slovak Republic, Netherlands, Lithuania, Slovenia, Russian Federation, Scotland        
slightly below average countries (Israel) 5% 24% 57% 85%
Israel, Latvia, Malaysia, Italy, Bulgaria, Romania, Belgium, Jordan, Norway        
below average countries (Serbia) 2% 16% 48% 79%
Serbia, Macedonia, Moldova, Armenia, Palestinian National Authority, Egypt, Iran        
really below average countries (Chile) 1% 5% 24% 56%
Chile, South Africa, Cyprus, Bahrain, Indonesia, Lebanon, Philippines, Saudi Arabia, Morocco, Tunisia, Botswana, Ghana        

again the range at the bottom end is still very large; although many of the countries in the last category at least got over 50% to the low benchmark, 7 did not -- the worst only got 13% through.

Grade 4 advanced high intermediate low
highest performing countries (Singapore) 25% 61% 86% 95%
Singapore, England, Chinese Taipei, United States, Japan        
above average countries (Russian Federation) 11% 39% 74% 93%
Russian Federation, Hungary, Australia, New Zealand, Italy, Latvia, Hong Kong        
slightly below average countries (Scotland) 5% 27% 66% 90%
Scotland, Moldova, Netherlands, Lithuania, Slovenia, Belgium        
really below average countries (Cyprus) 2% 17% 55% 86%
Cyprus, Norway, Armenia        
really below average countries (Philippines) 2% 6% 19% 34%
Philippines, Iran, Tunisia, Morocco        

note that there are substantially fewer countries' results available at grade 4

You can find out more about international comparisons of achievements in mathematics, science and reading at the official website for TIMSS (Trends in International Mathematics and Science Study) & PIRLS (Progress in International Reading Literacy Study): http://timss.bc.edu/

The full 2003 Science Report can be downloaded at: http://timss.bc.edu/timss2003i/scienceD.html

PIRLS

PIRLS is an international study of reading literacy involving 35 countries. It began in 2001, and is intended to take place every five years. It assesses performance at year 4 (around 10 years of age), although in a few cases the students are in their 3rd or 5th year of formal schooling. The PIRLS 2001 assessment was based on eight different texts of 400 to 700 words in length – four literary and four informational. Test items were designed to measure four major processes of reading comprehension:

  • Focus on and Retrieve Explicitly Stated Information.
    The student needed to recognize the relevance of the information or ideas presented in the text in relation to the information sought, but looking for specific information or ideas typically involved locating a sentence or phrase (approximately 20% of the assessment).
  • Make Straightforward Inferences.
    Based mostly on information contained in the texts, usually these types of questions required students to connect two ideas presented in adjacent sentences and fill in a “gap” in meaning. Skilled readers often make these kinds of inferences automatically, recognizing the relationship even though it is not stated in the text (approximately 40%).
  • Interpret and Integrate Ideas and Information.
    For these questions, students needed to process the text beyond the phrase or sentence level. Sometimes they were asked to make connections that were not only implicit, but needed to draw on their own knowledge and experiences (approximately 25%).
  • Examine and Evaluate Content, Language, and Textual Elements.
    These questions required students to draw on their knowledge of text genre and structure, as well as their understanding of language conventions and devices (approximately 15%).

23 of the 35 countries had average reading scores significantly above the international average of 500; the range was large, with the highest scoring country (Sweden) scoring 561, compared to the lowest scoring 327 (Belize). I've grouped them into five categories according to performance. As with the TIMSS results, the highest performing country in the group is the one whose average score is given:

  average range1
highest performing countries (Sweden) 561  
Sweden, Netherlands, England, Bulgaria, Latvia, Canada, Lithuania, Hungary, United States, Italy, Germany, Czech Republic   542-561
above average countries (New Zealand) 529  
New Zealand, Scotland, Singapore, Russian Federation, Hong Kong, France, Greece   524-529
average countries (Slovak Republic) 518  
Slovak Republic, Iceland, Romania, Israel, Slovenia, Norway   499-518
below average countries (Cyprus) 494  
Cyprus, Moldova, Turkey, Macedonia   442-494
really below average countries (Colombia) 422  
Colombia, Argentina, Iran, Kuwait, Morocco, Belize   327-422

1. the difference between the country with the lowest average and the one with the highest average

It should be noted that the range of difference between the highest 5% and lowest 5% of students in most countries was 200 to 300 points -- similar to the range in average performance across countries.

In all countries, girls had significantly higher achievement than boys. Italy had the smallest difference, with an 8-point difference compared an 11-point or greater difference for all other countries. The international average was 20 points. Countries with a difference of 25 points or more included Moldova, New Zealand, Iran, Belize and Kuwait.

For more details on countries' performance, see http://timss.bc.edu/pirls2001i/pdf/P1_IR_Ch01.pdf

Although the PIRLS, like the TIMSS, used benchmarks, the performance on the benchmarks as a whole for each country doesn't seem to be available. However, you can read about benchmark items and countries' achievements on particular ones at http://timss.bc.edu/pirls2001i/pdf/P1_IR_Ch03.pdf

The full 2001 Literacy Report can be downloaded at: http://timss.bc.edu/pirls2001i/PIRLS2001_Pubs_IR.html

tags strategies: 

tags study: 

Improving academic performance with a simple psychological intervention

Stereotype threat is a factor not only for some ethnic groups, but for women in certain areas (e.g., math, engineering), and also for older adults. Interventions that help reduce stereotype threat are also potentially helpful for those who suffer from test anxiety, or math anxiety.

Dave Nussbaum talks in the Scientific American about research he’s been involved in, showing how a small intervention aimed at reducing stereotype threat had significant long-lasting benefits for Latino American middle school students.

... In both schools, the intervention improved core course grades (Science, Social Studies, English, and Math) among Latino American students by the end of the school year by an average of roughly 0.3 points (on a 4.33 scale), reducing the achievement gap by 20-30% (the intervention had no effect on the White students’ grades). The intervention also sharply reduced the downward performance trajectory. Moreover, students in one school were followed for two years after the intervention had been completed and its effect on their grades persisted, even as some students made the difficult transition from middle school to high school. …

 Read the article

tags memworks: 

tags problems: 

tags study: 

Metacognitive questioning and the use of worked examples

The use of worked examples

We're all familiar, I'm sure, with the use of worked-out examples in mathematics teaching. Worked-out examples are often used to demonstrate problem-solving processes. They generally specify the steps needed to solve a problem in some detail. After working through such examples, students are usually given the same kind of problems to work through on their own. The strategy is generally helpful in teaching students to solve problems that are the same as the examples.

Worked-out examples are also used in small-group settings, either by working on the example together, or by studying the example individually and then getting together to enable those who understood to explain to those who didn't. Explaining something to another person is well-established as an effective method of improving understanding (for the person doing the explaining -- and presumably the person receiving the explanation gets something out of it also!).

Metacognitive differences between high and low achievers

An interesting study comparing the behavior of high and low achieving students who studied worked-out examples cooperatively found important differences.

High achievers:

  • explained things to themselves as they worked through the examples
  • tried to construct relationships between the new process and what they already knew
  • tended to infer additional information that wasn't directly given

Low achievers on the other hand:

  • followed the examples step-by-step without relating it to anything they already knew
  • didn't try to construct any broader understanding of the procedure that would enable them to generalize it to new situations

Other studies have since demonstrated that students taught to ask questions that focus on relating new learning to old show greater understanding than students taught to ask different questions, and both do better than students who ask no questions at all.

Learning to ask the right questions

An instructional method for teaching mathematics that involves training students to ask metacognitive questions has been found to produce significant improvement in students' learning. The method is called IMPROVE -- an acronym for the teaching steps involved:

  • Introduce new concepts
  • Metacognitive questioning
  • Practise
  • Review
  • Obtain mastery on lower and higher cognitive processes
  • Verify
  • Enrich

There are four kinds of metacognitive questions the students are taught to ask:

  1. Comprehension questions (e.g., What is this problem all about?)
  2. Connection questions (e.g., How is this problem different from/ similar to problems that have already been solved?)
  3. Strategy questions (e.g., What strategies are appropriate for solving this problem and why?)
  4. Reflection questions (e.g., does this make sense? why am I stuck?)

A study that compared the effects of using worked-out examples or metacognitive questioning (both in a cooperative setting) found that students given metacognitive training performed significantly better than those who experienced worked-out examples (the participants were 8th grade Israeli students). Lower achievers benefited more from the metacognitive training (not surprising, because presumably the high achievers already used this strategy in the context of the worked-out examples).

More reading

A paper by the creators of IMPROVE on their studies into the benefits of metacognitive instruction (in PDF format):

http://www.dm.unipi.it/~didattica/CERME3/proceedings/Groups/TG8/TG8_Kramarski_cerme3.pdf

References: 

Mevarech, Z.R. & Kramarski, B. 2003. The effects of metacognitive training versus worked-out examples on students' mathematical reasoning. British Journal of Educational Psychology, 73, 449-471.

tags study: 

Subscribe to mathematics