Test of Reading Comprehension 4th Edition
Virginia L. Brown, J. Lee Wiederholt, Donald D. Hammill
a. General Description:
The TORC4 is a test that has been changed over time to fit the best needs of the students being tested and stay up to date on norms and vocabulary. It was originally made to meet the need for a reliable, nationally normed, multi-dimensional and validated test for silent reading comprehension. It was based off of research concerning the nature of reading comprehension.
b. Materials provided/needed:
The materials that are provided are the test manual, examiner record form, student question booklet and the student answer form. The only other materials needed are a pencil.
c. Alternate forms:
There were no alternate forms found for this test.
a. Age ranges:
7-17 are the ages that this test focuses on.
b. Administration and scoring time:
The test will take no more than 45 minutes long.
c. Types of scores reported:
For relational vocabulary, a correct answer will be recorded as 1 and an incorrect answer will be recorded as 0. The students must mark both correct words to get complete credit for the right answer.
For subject completion, a correct answer will be recorded as 1 and an incorrect answer will be recorded as 0. The raw score is the amount of correct answers up to and including the last item passed in the five-item sequence
For paragraph construction, the student will receive one point for each sentence that is put in the correct sequential order. For example if there are four sentences to sequence into a paragraph and the student gets the first two sentences in the right order but not the last two then the student will receive two points for that item.
For text comprehension, a correct answer will be recorded as 1 and an incorrect answer will be recorded as 0.
For contextual fluency, you will start scoring with the last passage that the student attempted and you will score backwards until you reach the spot that the student correctly identified all of the words in a passage or until all of the words have been scored. So if the student got some words correct in passage 9 and all of the words in passage 8 you give the student credit for all the words in passage 1-8. Each correctly identified word is 1 point.
d. Starting points:
All subtests start at item 1
Relational vocabulary and Sentence Completion’s ceiling is reached when 3 out of any 5 consecutive items are incorrect
Paragraph Construction the ceiling is reached when student scores fewer than 3 points on any 2 consecutive items beginning with item 4 because the first four items are given regardless.
Text Comprehension the ceiling is reached when any 3 items are missed on three consecutive stories
Contextual Fluency subtest has no ceiling
A site coordinator with the experience in collecting standardization data was picked for each location and then they were trained with the new edition of this test. At each site, the participating schools were selected based on the demographic characteristics of the students closely matched with the region as a whole. This approach ended up in 1,604 students being tested. The additional examiners were chosen randomly by accessing the PRO-ED customer files to track professional who had purchased tests of reading in the last 2 years. These examiners were then asked if they would be able to test about 20 students. 20 examiners volunteered and gave data on 338 students.
b. Size and characteristics of sample (sex, age, geographic region, etc.)
The sample size for this test was 1,942 people in 14 different states. The states represented were California, Colorado, Connecticut, Illinois, Massachusetts, Missouri, Nebraska, New Jersey, New York, North Dakota, Ohio, Texas, Utah, and Wisconsin. The ages of the norm group was anywhere from 7 years old to 17 years old.
c. Date of norms:
There was no date found for when the norm was tested but the copyright was in 2009 so the norm was probably tested in 2007 because it is usually tested two years before the copyright.
a. Explanation of the procedure for each reliability type measured:
Content sampling error was investigate by applying Cronbach’s coefficient alpha method. Coefficient alphas for four of the subtests were calculated at 11 age intervals using data from the entire normative sample. The coefficients were averaged using the Fisher z-transformation technique.
Time sampling error is estimated using the test-retest method. In this measurement the test is given to a group of people, then a period of time is allowed to pass, and then the same group is tested again and the results are compared. The degree of similarity between the two test scores shows the amount of stability and reliability that that test has. This type was tested in a public school in Austin, Texas with 68 students that were divided into a 7 to 12 age group and a 13-17 age group. The time lapse between the tests varied from 1-2 weeks.
Scorer reliability refers to the amount of test error due to examiner variability in scoring. In this test, two PRO-ED staff member independently scored the same 50 protocols drawn from students in the normative sample. The students were anywhere from 7-17, where there were 25 males and 25 females from the South or Midwest.
b. Coefficients for each measure of reliability reported if applicable:
All but one of alphas are at or above .90 so highly reliable. The only exception was .89 but it rounds to .90.
The coefficients for the combined group equal or exceed .80. The coefficients for the Reading Comprehension exceeds .90.
The coefficients for this test are all at .95 or higher.
a. Explanation of each type of validity measured:
The content-description validity has three parts to it. First it gives a rationale for the test and testing formats for the subtests. Secondly it gives an item analysis and the last thing it does to give validity is that it uses differential item functioning analysis to show bias in the test. Differential item functioning analysis is looking at different population groups in the test subjects and seeing if there are any bias present. For this test they looked at gender, race, ethnicity, exceptionality, family income, educational attainment of parents.
The criterion prediction validity puts the Torc-4 test up against other tests of similar areas. The tests that they compared the TORC-4 to were the Reading Observation Scale, the Woodcock-Johnson III Tests of Achievement, the Written Language Observation Scale, and the Test of Written Language Fourth Edition.
This test also had construct-identification validity which has 6 hypotheses that it tests to show the validity of the test.
1. Since the reading comprehension ability is known to be related to age, then the TORC-4 subtests should tie together with chronological age.
2. Since the TORC-4 subtests were made to measure the ability of reading comprehension, then they should be pretty highly interrelated but not too much so.
3. Since reading comprehension is known to be a pretty significant cognitive ability then the TORC-4 subtests should be able to correlate highly with intelligence tests.
4. Since the TORC-4 looks at reading comprehension then the results should vary from the groups that are known to be good readers and those that are known to be not as good at reading.
5. Since the TORC-4 subtests were made to look at the reading comprehension by the ways different measures then the subtests should focus on one skill.
6. Since the items of a certain subtest look at similar traits then the items of each subtest should highly related with the total score of the subtest.
b. Correlation coefficients reported if applicable
The average coefficients for each subtest after being compared to the other four tests are as followed:
Relational Vocabulary: .63
Sentence Completion: .53
Paragraph Construction: .45
Text Comprehension: .42
Contextual Fluency: .43
Composite Reading Comprehension: .63
Construct- Identification Validity:
The coefficients for the correlation with age are as followed for each subtest.
Relational Vocabulary: .64
Sentence Completion: .66
Paragraph Construction: .54
Text Comprehension: .57
Contextual Fluency: .77
The correlations between the tests should be between .30-.70 to contribute variance but not have too much correlation to be redundant. The range of coefficients of the correlation is between .37-.63.
The coefficients of the correlation between the TORC-4 and a top intelligence test the WISC-IV are as followed for the subtests with the full scale WISC test.
Relational Vocabulary: .63
Sentence Completion: .75
Paragraph Construction: .81
Text Comprehension: .80
Contextual Fluency: .80
Reading Comprehension Composite: .66
The coefficients of the factor analysis for each subtest are as followed:
Relational Vocabulary: .79
Sentence Completion: .84
Paragraph Construction: .82
Text Comprehension: .70
Contextual Fluency: .71
VII. CLASSROOM USES:
a. Specific uses as suggested by manual/authors:
One specific use for this test is to identify if certain individuals will need specialized reading services and help to make decisions if the students do need specialized services. Another use for this test is to evaluate students’ progress in certain programs. The final use of this test is for research and the correspondence of reading comprehension to other areas in life.
b. Your opinion of appropriate uses:
I think that the specific uses that the manual included are very good uses for the test because all three of those areas are very important for continuing the best reading education that we can to help the students get the best experience possible. I think this is a great tool for identification and measuring the progress in the area of reading comprehension. This looks like it is a good assessment in the area of comprehension.
I like the different subtests, I think they are good subtests and activities that are very good measures reading comprehension. I like the simplicity of the administration of the test and of the scoring process for each subtest are pretty simple and easy to understand.
One thing that I didn’t like was the language of the manual sometimes. Sometimes it seemed pretty complex and didn’t make a whole lot of sense and was hard to follow sometimes.