Although most foreign language educators would agree that the opportunity to study in the country where a student's foreign language is spoken is invaluable, there is little research that assesses exactly what benefits accrue to students from such an experience. The dearth of research is particularly acute in reference to summer study abroad programs. The present study is an initial attempt to use an ACTFL-based test to evaluate gains in oral proficiency during a six-week summer study program. The study has three main foci: a) to determine the validity of the ACTFL-based Portuguese Speaking Test as a placement tool for a short study abroad program; and b) to assess gains in oral proficiency among participants in the University of Tennessee's summer study abroad program in Brazil; and c) to assess, where possible, other issues related to the initial study abroad experience.

John Carroll (1967) in his study of the language proficiency of foreign language majors in their senior year, noted that most of the majors in his sample had at least one foreign study experience. Even with this experience, most did not surpass the level of 2+ on the FSI scale. This rating is equivalent to a score of Advanced Plus on the ACTFL scale (Liskin-Gasparro, 1987).

Graman (1987) noted the same phenomenon among students in his upper division courses at the University of Utah. He administered a questionnaire to his students and found that only 8 out of 130 students enrolled in upper division Spanish courses did not have experience in noninstructional Spanish-speaking environments. Graman concluded that without outside experience, students are very unlikely to progress beyond 200 level courses. Graman also noted that upper division students who had foreign experience expressed a significantly greater degree of comfort when using the language in a listening or speaking situation.

The findings of Carroll and Graman affirm the important role foreign study plays in developing language skills that are needed for advanced language courses. While efforts have been made to make the foreign language classroom more communicative and to improve student's proficiency so as to better prepare students for advanced language classes, foreign study serves as an irreplaceable linguistic and cultural experience.

The traditional foreign study program designed for language learners includes study at a foreign institution in regular or specially designed classes, placement with families, and often additional scheduled travel. The cost of these programs coupled with the time students must commit to foreign study has led to a shorter, more intensive version of foreign study, the summer study program. Unlike academic year programs, summer programs typically arrange for special classes for students at foreign institutions that would normally be on vacation. Summer study is attractive to faculty and students alike because it requires less time and less money. However, summer study programs have been criticized for their brevity, intensity, and lack of   —779→   traditional structure. The shortness of the summer study experience can limit student contact with the host culture, which in turn limits the opportunity for language practice. While most summer study programs offer credit for work done abroad, administrators and faculty sometimes wonder if summer study programs offer classwork that is equal in rigor and content to course work given on campus.

Only a few studies have reported on the effects of summer study abroad programs. Fish (1974) and Chlebek and Coltrinari (1987) used student course evaluations to report on summer study abroad programs for high school students. At the end of each of these programs, students reported more ease and self-confidence in speaking the foreign language as well as an increase in comprehension, vocabulary and fluency.

Armstrong (1982) reported on a study of 60 high school graduates who took a six week summer course in Mexico on the Indiana University Honors Program. The students were administered the listening, reading, and writing sections of the MLA Cooperative Spanish Test (form MA) on the first and last day of class. Their gains were compared with the raw scores for high school students completing three and four years of Spanish, who participated in the study that was used to establish the norms reported in the test manual. The mean pretest score of the Indiana group was equivalent to that obtained by students completing three years of high school Spanish. The mean post-test score exceeded that reported in the manual for students completing four years of high school Spanish. Thus, Armstrong concluded that the program produced gains equivalent to a year or more of high school Spanish.

Unfortunately, no research is available to judge the amount of proficiency that a student should demonstrate in Portuguese at the end of various points of instruction. The authors of the above studies, with the exception of Carroll, did not even use a proficiency-based scale. The present study, although based on a small sample, provides some initial data on student proficiency that may be used for comparative purposes by others.

The University of Tennessee, in cooperation with the Federal University of Ceará, offers a six week course of study in Portuguese language and Brazilian culture during the months of June and July in Fortaleza, the capital city of Ceará in Northeastern Brazil. The goals of this foreign study course are to develop greater proficiency in the four language skills, broaden the appreciation of Brazilian people and their culture, and develop a broader range of life skills.

The summer program was inaugurated in 1987, with eight University of Tennessee undergraduates participating. In the 1989 program under study here, fifteen students (six from Tennessee, three from Iowa State, three from Georgetown University, and three from Dickinson College) attended classes in Fortaleza. In 1990, nine students enrolled in the summer program, and of these, six were from Tennessee, two from the University of Virginia, and one from Dickinson College.

The requirements for admission to the summer study program are one year of Portuguese language study prior to departure and an interest in Brazilian culture and language. The summer program accepts students from Portuguese programs throughout the United States and charges the same fee for Tennessee residents and nonresidents alike. Because language programs vary in the amount and type of material covered during a year, students are grouped according to their oral proficiency, evaluated at the beginning of the summer program, into two levels: intermediate and advanced. The subsequent language course is designed for each level's needs.

The summer course consists of one week of orientation in Belém, Pará, the port of entry, and five weeks of homestay, language and culture classes in Fortaleza. Classes meet for approximately four hours in the morning, four to five days a week. Two hours of class are dedicated to language learning and are taught by local faculty members, while the remaining time is dedicated to lectures by Brazilian scholars of economics, music, history, sociology, religion, etc. Excursions to near-by cities, beaches, factories, businesses or museums complement the classwork.

Classwork is divided into two segments: student-centered activities dominate the language classes while professor-directed lectures drive the culture classes. The language courses consist of structured grammar review, compositions, readings and discussions of newspaper and magazine articles, and observations of or interviews about social behavior and cultural values. Each lecture, usually conducted in Portuguese, offers a broad overview of its subject as it applies to Brazil and then concentrates on specific concerns about a major problem or topic of research.   —780→   Each lecturer is asked to bring an outline of his or her talk which students use to help them follow the lecture.

Student performance is evaluated in light of the course's objectives with a special emphasis on improvement in oral/aural skills. Students are evaluated through oral interviews at the beginning and end of the course. They evaluate their own progress in the areas of cultural information, grammar, and communication. They are also tested for their general understanding of the information presented in the lectures. How students adapt to living in a foreign culture, that is, the skills they develop and perfect in the day to day challenge of interacting with their families and the community, represents a significant part of the credit given for the culture course.

The present study is based on an analysis of data collected from the 1989 cohort of fifteen students from four universities. These students had previously completed from two to six semesters of Portuguese language study. One member of the group was a native speaker of Spanish, one was a native speaker of both Spanish and English, and the rest were native speakers of English. All but one of the participants had studied another foreign language in high school. Three of the students had lived abroad, seven had traveled in either Europe or Latin America, and five had never been outside of the United States. Although one of the students had lived in Brazil for a short time as a child, none had studied in Brazil previously.

Data was collected from three sources: measurements of oral proficiency (the Portuguese Speaking Test and a structured oral interview administered at intake by the instructors in Brazil), feedback obtained through questionnaires completed by instructors involved in the program, and feedback obtained through questionnaires completed by the students.

The Portuguese Speaking Test (PST), a semi-direct standardized test, is a simulated oral proficiency interview that closely follows the format of the ACTFL oral proficiency interview and is scored on a scale that uses the ACTFL proficiency guide lines to the Superior level and the scale used by the Interagency Language Roundtable (ILR) of the U. S. Government at levels above Superior. The PST was developed and made operational as a national testing program by the Center for Applied Linguistics in 1988. It has been found to exhibit high reliability and high concurrent validity with a face-to-face ACTFL interview (Stansfield, Kenyon, Paiva, Doyle, Ulsh, and Cowles, 1990).

The ACTFL scale provides a useful metric for understanding the spoken language proficiency of foreign language students. The points on the scale are described in functional terms so that it is possible to make assumptions about a student's level of proficiency in handling real-life situations based on his or her rating on the scale.

The PST was used as an external criterion in determining proficiency levels of the students involved in the University of Tennessee summer study abroad program. A principal goal in administering the PST was to determine if it could be used on a continuing basis each year to place students in appropriate classes in Brazil, with scores obtained prior to the students' departure from the United States. Such a use would facilitate staffing the courses and planning the curriculum. If the PST could function as well for placement purposes as an initial instructor interview, then placement decisions could be made based on a PST score.

The PST was administered as a pre-test and as a post-test. Following the administration, the PST examinee response tapes were sent to the Center for Applied Linguistics, which in turn sent them to the University of Pennsylvania, where they were scored by Maria Antônia Cowles, an ACTFL-certified interviewer and tester trainer.

Before departing for Brazil, 11 of the 15 participants in the program were administered Form A of the PST at their respective institutions, in May 1989. Upon arrival in Brazil, the participants were given a face-to-face, structured interview by the two program instructors for the purpose of determining their class level placement. This structured interview consisted of five questions. Responses to the questions were categorized according to the following scale:

The instructors were permitted to award pluses and minuses if they felt the need to do so.   —781→   Compound scores were also permitted. For data analysis, the final scores on the structured interview were coded as follows:

3+	3.50
3	3.00
3-	2.75
2+/3	2.75
2+/3-	2.65
2+	2.50
2	2.00
2-	1.75
1/2	1.50
1+	1.50
1	1.00
1-	0.75

On the basis of the interview, two classes (advanced and intermediate) were formed. The advanced class contained 7 students; the intermediate class contained 8. The structured interview lead to an apparently rather natural division: all of the seven students assigned to the advanced class received a 2+ (2.50) or above; five of the seven received the highest score of 3+ (3.50). Thus, according to the interview, there was little variation among them. Nonetheless, other data collected in this study, namely the PST, instructor evaluations and rankings of the students and student self-evaluations, revealed that this was not the case.

Two weeks into the program, the instructor of each group was asked to mark whether the class level was significantly below (--), somewhat below (-), at (), somewhat above (+) or significantly above (++) the student's own speaking ability level. The instructors were also asked to rank each student against the entire class in terms of his or her speaking ability. Students also completed the same evaluation of appropriateness of the class level to their level of speaking ability. This pre-test data for the advanced class are presented in Table 1 below.

These data show that the five students who received 3.50 on the interview ranged from 1.0 to 2.0 on the PST. Their average proficiency rating was 1.62, which is between an Intermediate Mid and an Intermediate High on the ACTFL scale. The classroom instructor also divided the five students into four different rank orderings. From the instructor's point of view, the level of the class was somewhat lower than the ability level of all students who received 3.50 on the interview (i. e., the class was somewhat too easy for them). The two students who received 1.8 on the PST agreed with the instructor's assessment, whereas the students receiving 1.5 and 1.0 felt the level of the class was adequate. Surprisingly, student C did not feel the class level below her ability level, even though she scored a 2.0 on the PST. With the exception of students B and C, the instructor's ranking of the students on the basis of class performance follows their earlier ranking by the PST exactly. The Spearman rank order correlation coefficient between the PST and the Instructor's Ranking for the six students who took the PST was .59; the rank order correlation coefficient between the interview and the instructor's ranking was .81. However, it must be remembered that the instructor's ranking of the students' ability was based on their performance in the on-site interview and not an independent measure.

Corroborating evidence of the effectiveness of the PST as a placement tool is given in the case of student G. This student, with the lowest score   —782→   on the interview and only an 0.8 (Novice-High) on the PST, felt the class level above her own, while her instructor felt the same.

The pre-test data for the eight students in the intermediate class is given in Table 2.

Based on the performance of students who took the PST at their respective institutions, the students in this level may be characterized as Novices on the ACTFL scale, even though they had all completed one year of college-level Portuguese. The mean PST rating in this group was .56, which is approximately a Novice-Mid on the ACTFL scale.

From these data, it can be seen that the PST in general agrees with the division of the class and with the instructor's ranking two weeks into the course. The rank order correlation between the PST and the instructor's ranking (for the five students who took the PST) was .89; the rank order correlation between the instructor's ranking and the interview for all eight students was .90. It must be remembered, again, that this latter correlation is not between two independent measures, as the former is. For the eleven students who had both PST and Interview scores as pretest measures, the correlation between the two was .88. This also supports the use of the PST as a placement test for oral proficiency.

The placement of student H into the intermediate group appears to be an anomaly. Although she scored as well or higher on the interview than two of the advanced students, she was placed in the intermediate class. The instructor, although ranking her as the best speaker in the class, did not feel the class level was below her level, although the student herself felt that way. The rest of the students felt the level was appropriate, but the instructor felt it was much above the level of the two students who had scored 0.2 on the PST. This evidence supports the use of the PST as a placement test in this context.

At the end of the program, students were again ranked, this time by both instructors who participated in the program. Table 3 presents student scores on the post-test PST, the ranking at the beginning of the program, and the class ranking at the end.

At the end of the program there was a narrower spread of PST scores in the advanced class. With the exception of student G, all students scored between 1.5 and 2.0 on the PST. In   —783→   ranking, instructor 1 differentiated between the students. The only incongruity between instructor 1 rankings and the ranking by the PST is for student D. Including that student, the rank order correlation between instructor 1 and the PST was .50; excluding her, the correlation is .97. Unfortunately, instructor 2 did not differentiate her rank ordering of students. Instead, she ranked all of them together into two groups. The rank order correlation between her rankings and those of the PST was .67. Student G continued to receive the lowest score on the PST and lowest ranking by her instructors.

Table 4 presents the same data for the intermediate class.

From this table we see that student H continued to be ranked first and Student O continued to be ranked at the bottom. Student O's PST score corroborated this ranking. Student I's score on the PST is surprisingly low. Consistently ranked second, this student scored at the same PST level as Student O, who was ranked lowest in the class. At the end of the summer term Student N had improved in ranking, and this improvement was corroborated by the improvement in his PST score. The number of students in this group who took the PST as a post-test was too low to meaningfully compute rank order correlations.

Table 5 presents data on the progress made by students during the course of the six-week program. As indicated earlier, the pretest PST was obtained at the end of the spring term, while the post-test PST was obtained at the end of the program. The first two columns present the pre and post-test PST scores. The next two columns present the two instructors' evaluation of the student's progress during the program: extraordinary progress (++), above average progress (+), adequate (), some progress (-), and very little progress (--). The final column presents the progress evaluation from the student's viewpoint. Here, the students compared their actual progress with the progress they had hoped to make, i. e., students did not evaluate their actual progress but the magnitude of their gains in comparison with their expected gains. A (+) means the student made more progress than he or she had expected, while a (-) means he or she made less progress than expected. A () means that the student's progress matched his or her expectations. Both classes are represented in this table.

This table reveals that the instructors were generally quite enthusiastic about the progress the students made. There is a clear difference, from the instructor's point of view, between the amount of progress made in the higher level group and in the lower group. The table also shows that students in the higher level group were generally more positive about the progress they made compared with that they had hoped to make. Only one of the seven students felt he had made less progress than he had hoped. Nonetheless, this individual did raise his PST score from 1.5 to 1.8. In spite of the students' satisfaction with their progress, their gains on the ACTFL scale were not significant. The mean gain was .12 with a standard deviation of .28. A one-tailed t-test revealed that this score gain was not statistically significant (t=1.025, p<.05,). The lack of significance is due in part to the small sample size. However, it is mostly due to the fact that increments between points on the ACTFL scale do not represent equal gains in proficiency; that is, as one moves up the scale each successive increment represents a far greater gain in proficiency than the previous increment.

In the lower group, half of the students felt they had made less progress than they had hoped. For one of these students there was no change in her PST scores. For another, the change was quite modest considering the ACTFL guidelines (0.2 to 0.8). The instructors were also less enthusiastic about the progress made by the lower level students as compared to the higher level students. However, two of the lower level students did show sizeable increases in their PST scores, which reached 1.5. These scores reflect quite a bit of improvement, even though one of the two students felt she did not make the progress she had hoped she would make.

In spite of this lack of instructor and student enthusiasm, the PST scores of the lower group did improve. The average gain for the lower level was .65 levels, with a standard deviation of .53. The one-tailed t-test revealed that the gain was significant (t=2.353, p>.05). Again, the magnitude of the gain during this short period of instruction corroborates that the Novice level of the ACTFL scale is sensitive to small gains in proficiency resulting from a short period of study.

Results from the PST and the student progress report provide insights into the cases of the «misplaced» students mentioned earlier. The first is student G, in the advanced class. Although instructors were enthusiastic about the progress she made and she herself was satisfied with it, her second PST score is surprisingly low. Perhaps being in a class above her level was an inappropriate placement for her. The second case is   —785→   Student H, who was placed in the intermediate class although she had scored higher than student G in the oral interview. Only one of the two instructors felt she made an above average amount of progress, and she herself was disappointed with the progress she made. Perhaps she would have progressed more in the higher class.

The average gain score for the 10 students who took the PST as pre- and post-tests was .33, with a standard deviation of .462. The mean pretest score was 1.06 while the mean post-test score was 1.39. This average score represents approximately the difference between moving from an Intermediate-Low to an Intermediate-Mid on the ACTFL scale. A one-tailed t-test revealed that this score gain was statistically significant (t=2.259, p<05). Thus, this increase cannot be attributed to chance alone. Table 5 does indicate that there is no relationship between the instructor's or student's subjective evaluation of progress made and gain scores on the PST. This may suggest that at the end of the program instructors and students were not able to make objective evaluations of the progress made in oral proficiency.

The students participating in the summer program were also administered a formal self-evaluation as a pre- and post-measure that consisted of three parts: Grammar, Communication, and Culture. The Grammar part contained 11 questions, Communication 13 and Culture 6. For each question in the grammar and communication parts of the questionnaire, students rated themselves on a 1 to 5 scale as follows:

The culture questions were scored on the following scale:

• 0 no knowledge at all.
• 1 I have a vague impression, but can't speak knowledgeably.
• 2 I have solid knowledge from reading, from experience, or from being told by a Brazilian.

The grammar questions focused on the use of individual verb tenses, pronouns, interrogatives, possessive adjectives and numbers which were felt to be mastered. The communication questions focused on functions such as describing family, school, home, and discussing food and health, and competencies such as using a phone, public transportation, print and non-print media. The culture questions focus on knowledge of Brazil's geography, history, political system, and units of currency, as well as the public transportation, and historical points of interest in Fortaleza.

In the following discussion, the results of one non-student individual who completed these forms and both administrations of the PST are included. This individual studied independently and worked as an assistant to the program director.

Table 6 presents the correlations between the interview, PST pre-test and the first self assessment.

The interview given the students at the beginning of the semester correlated relatively highly (.78) with the total self assessment score and had a significant correlation (.87) with one of the three subscales (grammar). This high correlation with grammar is understandable as grammar played a major role in the interview. The PST correlated about the same with the total self evaluation score (.75) as did the Interview (.78). It correlated slightly less strongly with grammar (.80) than did the interview (.87) but had a significant moderate correlation with the self assessment of communication (.64). This suggests that the PST score may be more reflective of total communication ability than the Interview.   —786→   Neither oral proficiency score correlates with the culture subscale score. This is not surprising, since one would not ordinarily expect factual knowledge about a foreign country to be directly related to the degree of oral proficiency in that country's language. Neither the grammar nor the communication subscale correlates with the culture subscale.

It is interesting to note, however, that neither the PST post-test nor the interview correlated significantly with either the total score or with any of the subscale scores on the student self-assessment at the end of the program. This may suggest that at the end of the program students were less likely to make objective self-evaluations.

The analysis of the data presented in the tables confirms the validity of the PST as a placement tool. Both when correlated with student evaluations and with instructor interviews, the PST provides meaningful information about the student's ability to speak Portuguese. The PST showed significant correlations with the student self-evaluation of communication (r=.64) and grammar (r=.80) and with the total assessment by students of their skills (r=.75). These data provide further evidence for the validity of the PST as a measure of grammar and communicative competence. The PST demonstrated a moderately strong relationship (r=.88) with the instructor interviews, which supports its validity as a placement tool. In two instances, students G and H, the evidence suggests that the PST would have helped the instructors detect a misplacement. The progress of those two students, as registered by the instructors and the students themselves, did seem to be adversely affected by placing them at inappropriate levels for their skills.

Although the gains measured by the PST were significant for the group as a whole, when divided into class levels the gain was greatest for the intermediate group and not significant for the advanced group. These gains reflect the weighting of the ACTFL Guidelines, that is, the students in the intermediate group made significant gains because increments on the scale come more easily at the bottom. So too, the students in the advanced class did not show significant gains because they started at a level from which it is difficult to advance in just six weeks of instruction.

This study points out that, subjectively at least, students and instructors were more positive about progress made when the students began at the ACTFL Intermediate level than when students began at the Novice level. A commonly held tenet of foreign study is that students with only minimal language skills, in this case the Novice level, may not have the foundation on which to build good language skills in a short amount of time. The perceptions exhibited here are consistent with that belief. On the other hand, it may be that this group of students was disappointed in their progress because they had unrealistic expectations. These are reflected in the following student comments:

Research conducted by Horwitz (1988) has emphasized that beginning students hold unrealistic beliefs about the time required to learn a language. She noted that a «substantial number of students felt that a maximum of two years is sufficient for learning another language» when studied one hour per day. Further research could focus on student beliefs about progress to be expected as a result of foreign study. Additional research could examine the effects of students' and instructors' expectations on language gains made during study abroad.

An issue that this study did not address, but that deserves further attention, is the problem of accurately measuring the Portuguese proficiency of Spanish speakers. As Jensen (1989) has stated, previous knowledge of Spanish enables students to «demonstrate an impressive degree of 'proficiency' with no instruction whatsoever». This is due to their communicative ability in a related language. Several of the students in the advanced class either spoke Spanish as a first language, were bilingual, or had studied Spanish in high school. Jensen's work might lead to speculation that the enthusiasm of the advanced level students and their instructors was due more to 'proficiency' in Spanish than to actual progress in Portuguese. The presence of Spanish in the students' backgrounds might also skew the results of the PST. For example, the meager gains of the advanced students could be due, in part, to «false proficiency» demonstrated by the Spanish speakers on the pre-test. Jensen suggests such students may require a different kind of proficiency   —787→   test; i. e., one that would measure how much communicative ability comes from mastery of Spanish and how much is actually based on genuine command of Portuguese. We believe it would be useful if future research on the PST or the ACTFL interview could address the previous language training of Portuguese students, especially those with a background in Spanish. The results of such research could also inform the development of a set of ACTFL Proficiency Guidelines for Portuguese.

Although limited by the small number of participants, this study suggests that the PST should be used as a placement tool for students entering the University of Tennessee's Brazil summer program. The results of the PST would augment the existing information on the students' applications so that classes could be formed and the curriculum shaped before arrival in Brazil. The fact that the test is prerecorded, can be given on any campus, and produces meaningful results makes it both convincing and convenient.

With regard to the second focus of this study, to assess gains in oral proficiency, it must be stated that the PST did register improvements made by the students during the six week program. These improvements were statistically significant for the students in the intermediate Portuguese class, yet smaller for students in the advanced Portuguese class. The probable explanation for this phenomenon is the ACTFL scale itself, which demands increasingly larger gains in proficiency as one moves up the scale. Such gains might be difficult to attain in six weeks of summer study. Otherwise, in the advanced level class, students and instructors were generally quite satisfied with the gains attained.

The University of Tennessee will benefit from this attempt to measure the performance of students in its summer study abroad program. Administrators and educators alike seek empirical data that will help strengthen program operations and tailor the curriculum to the needs of the students. This research contributes to the body of literature on foreign study by helping to validate an instrument that can identify student skill levels before studying abroad. With additional research, it should be possible to determine in what ways skills are improved through study abroad. And, as more data are collected, researchers will understand better the role summer study plays in developing students' language skills.

Armstrong, G. K. 1982. «Language Study Abroad for High School Students: Indiana's Program for Proficiency and Recruitment». Foreign Language Annuals 15: 365-70.

Byrnes, H. 1989. «The Rating Scale». In K. Buck, Ed. The ACTFL Oral Proficiency Interview Tester Training Manual (Chapter 2). New York: ACTFL.

Carroll, J. B. 1967. «Foreign Language Proficiency Levels Attained by Language Majors near Graduation From College». Foreign Language Annuals 1: 131-51.

Chlebek, A. and H. Coltrinari. 1987. «Summer Immersion Programs Abroad». Canadian Modern Language Review 33.3: 348-53.

Fish, I. 1974. «Foreign Language Learning and Short Term Study Abroad». American Foreign Language Teacher 4.4: 36-38.

Graman, T. L. 1987. «The Gap Between Lower- and Upper-division Spanish Courses: A Barrier to Coming Up Through the Ranks». Hispania 70: 929-35.

Horwitz, E. K. 1989. «The Beliefs about Language Learning of Beginning University Foreign Language Students». Modern Language Journal 72.3: 283-94.

Jensen, J. B. 1989. «Evaluating Portuguese Performance of Spanish-Speaking Students». In D. A. Koike and A. R. M. Simoes, Eds. Negotiating for Meaning: Papers on Foreign Language Teaching and Testing. Austin, TX: University of Texas. 119-30.

Liskin-Gasparro, J. 1987. Testing and Teaching for Oral Proficiency. Boston: Heinle & Heinle.

Stansfield, C. W., D. M. Kenyon, F. Doyle, R. Paiva, and I. Ulsh. 1988. Portuguese Speaking Test: Official Test Manual. Washington, DC: Center for Applied Linguistics.

Stansfield, C. W., D. M. Kenyon, R. Paiva, F. Doyle, I. Ulsh, and M. A. Cowles. 1990. «The Development and Validation of the Portuguese Speaking Test». Hispania 73.3: 642-51.

Hispania [Publicaciones periódicas]. Volume 74, Number 3, September 1991