Acknowledgments

The realization of this test was possible thanks to a Title VI Grant from the US Department of Education, USDE 22470. Special thanks to Dale A. Koike, Elena C. Papanastasiou, Nina Garret, Cesario Alvim and Renato Maggioli Costa de Frias who have contributed significantly for the realization of this test with their valuable comments. It has been a privilege to work with a team of Brazilians from the Summer Language and Culture Institute, Universidade Federal do Espírito Santo, in Vitória, during the preliminary preparation of this test. They are Professor Santinho Ferreira, the project consultant, Andréa Lemos Ferreira, the Institute administrator, and the instructors Regina Egito, Flávio Valadares and Ana Paula Chagas Correia. Dovis B. Pollock, Nate Harrison, Steve Milas, Natalie Savage and students of the University of Kansas have also participated in the preliminary versions of this exam and provided valuable feedback. 

This self-assessment instrument is intended to help Brazilian Portuguese students determine their proficiency classification level relative to other students in the same program. We use cutscores to divide our students into different language groups. In the future, after a given period of use of this exam, we will be able to propose a benchmark, should such a reference become necessary. 

This test was not designed for use in a competitive situation or for determining grades. It is designed to perform functions such as language gain measurement and placement in a language program, provided that the test administrator understands the limitations of subjective evaluations. Language gain measurement for instance, is still limited to larger gains in the current version of this self-assessment. A future version of this test will take into consideration language gain in short periods, such as one semester or less of language instruction. Another limitation that must be taken into consideration is the student population that is expected to take this test. This test was designed mainly for a motivated student population, such as students going abroad. This type of student will be more motivated and more careful when taking this test than students who are simply fulfilling language requirements in an average U.S. college. For more information regarding the limitations of self-assessments and other characteristics of self-assessment see the article by Simoes and Papanastasiou (Simões, A.R.M. and Elena C. Papanastasiou (September, 2002 - in print). Evaluating the usefulness and properties of a subjective assessment of Brazilian Portuguese. In Hispania, 85.3:900-910).

In addition to its functions, such as the placement of students in a language level or the assessment of language gains, this test will also inform students about the type of knowledge they might want to consider when studying Brazilian Portuguese. In other words, the content of this self-assessement can also help students to establish their goals when studying Brazilian Portuguese.

The items for this self-assessment exam were prepared using the American Council on the Teaching of Foreign Languages (ACTFL) Oral Proficiency Guidelines only as a point of departure. A different scale of language skill levels was developed simply to organize the 48 items of this exam. We acknowledge that there are no empirical data or theoretical rationale for ACTFL guidelines, except for speaking, or for the scale of this self-assessment. The purpose of creating this scale was mainly a practical one since all decisions regarding language skills in relation to this Brazilian self-assessment exam will depend on statistical results and not on this language skill scale. 

Although the items were structured according to practical language skill levels, the actual self-assessment has the test items presented in random order, so that any situation, regardless of complexity in any of the six linguistic areas included, might appear at any point in the test. Therefore, the order was intended to be unpredictable, just as any situation would present itself in a full immersion context.

The graphic below depicts the random distribution of the 48 test items.

This self-assessment is predominantly characterized by the use of specific question items. A number of authors have argued that students can accurately evaluate their own language capabilities, especially when asked to judge specific tasks rather than overall language competence (Oscarsson, M. 1978. Approaches to self-assessment in foreign language learning. Oxford: Pergamon Press;  Pierce, Bonny N., Merrill Swain, and Doug Hart. 1993. Self-assessment, French immersion, and the locus of control. Applied Linguistics, v14, n1, March, 25-42).

The current Brazilian Portuguese Self-Assessement Test uses a two-way yes-no scale. The reason for the use of the yes-no scale is summarized in the brief statement below.

Some scaling issues that were considered during the preparation of the Brazilian Portuguese self-assessment 
Prepared by Elena C. Papanastasiou © 2002 

One of the limitations of self-assessment is that people do not all use the same criteria to evaluate themselves. For example, some individuals might tend to over-inflate their ability estimates in an attempt to respond in socially desirable ways (Shrauger & Osberg, 1981).On the other hand, high achieving individuals who strive for excellence and perfection might consistently respond as if they knew less than they actually do since they are aware of their imperfections. Consequently, it would not be surprising if a low achieving individual, who strives for social desirability, responded in a similar way to a self-assessment, as high achieving individuals who might tend to focus themselves on their imperfections (so that they can be improved) (Papanastasiou & Simoes, 2002). This becomes even more of an issue if there is ambiguity in the scale that is used to assess themselves. For example, being able to do something ‘very well’ might have a different meaning for different individuals. If ‘very well’ is the highest option on a Likert scale, some people might interpret that as ‘excellent performance.’ However, low achieving individuals might consider anything that they can do adequately as ‘very well.' Consequently, any ambiguity on the scale of the assessment might add additional sources of error to the results.  This is why previous studies have shown that the yes-no scales (or other dichotomous scales) tend to be more reliable in test-retest situations and are also more clear (Bruna, 2002).  This is where the main merit of a dichotomous scale comes from.Finally, another advantage of using a yes-no scale, is that the students are able to judge their skills in more of a factual way. Either they can do something, or they cannot. This is done in an attempt to make the criteria on which they judge their skills on the subjective test as uniform and objective as possible. 

Bruna, K. R. (2002). Screening and placement of students into appropriate Spanish assessment levels: Self-assessment, cognitive screening, and survey data. Paper presented at the American Educational Research Association, New Orleans, LA, April 1-5, 2002. 

Papanastasiou, E. C. & Simoes, A. R. M. (2002, June). Self-assessment for Brazilian Portuguese and its relationship to adaptive testing. Paper to be presented as poster at the International Conference on Computer-Based Testing and the Internet, Winchester, England, June 12-15, 2002. 

Shrauger, J. S. & Osberg, T. M. (1981). The relative accuracy of self-predictions and judgements by others of psychological assessment. Psychological Bulletin, 90, 322-351.