An Investigating into the Differences between Advanced and Intermediate Proficiency Levels in EFL Writing Skill Test

Abstract

In this study, it is aimed to determine the most measured writing constructs based on confirmatory factor analysis to evaluate the significant differences in the specifications of the EFL writing skills test between Iranian advanced and intermediate proficiency level learners. For this purpose, sequential combination method was performed to provide comprehensive answers to research questions. In the present study, 60 out of 114 Iranian language learners from Boroujerd Branch Azad University were selected as participants. Then, the quantitative stage was followed by the qualitative stage. The research tools included Oxford Placement Test (OPT), structured survey questionnaire, semi-structured interview, and writing tasks. The participants were divided into advanced and intermediate groups based on the students' writing skills scores. Moreover, independent t-tests as well as principal component analysis (PCA), were used to compare the advanced and intermediate groups. The results showed that there is a significant difference between Iranian learners with high and medium skill level in the most measured structures in the specifications of the Iranian EFL writing skills test. Most of the measured structures included mode, evaluation and mechanics, respectively.

Keywords: Iranian EFLs, Writing skills, Advanced, Intermediate, Proficiency.

Resumen

En este estudio, el objetivo es determinar los constructos de escritura más medidos en función del análisis factorial confirmatorio y también evaluar las diferencias significativas en las especificaciones de la prueba de habilidades de escritura EFL entre los estudiantes iraníes de nivel de competencia avanzado e intermedio. Para ello, se realizó el método de combinación secuencial para dar respuestas integrales a las preguntas de investigación. De modo que a la etapa cuantitativa le siguió la etapa cualitativa. Las herramientas de investigación incluyeron la prueba de ubicación de Oxford (OPT), el cuestionario de encuesta estructurada, la entrevista semiestructurada y las tareas de escritura. Según los puntajes de las habilidades de escritura de los estudiantes, se dividieron en dos grupos: avanzado e intermedio. Además del análisis de componentes principales (PCA), se utilizaron pruebas t independientes para comparar los grupos avanzado e intermedio. Los resultados mostraron que existe una diferencia significativa entre los estudiantes iraníes con un nivel de habilidad alto y medio en las estructuras más medidas en las especificaciones de la prueba de habilidades de escritura EFL iraní. La mayoría de las estructuras medidas incluyeron modo, evaluación y mecánica, respectivamente.

INTRODUCTION

Students have difficulty learning English as a as a Foreign Language (EFL) in communicating their ideas effectively because of the lack of creativity and sufficient knowledge in English writing skills (Quvanch, 2020). EFL writing is a complex social activity that involves many capabilities like selecting the right topics for a specific audience, creating logical and clear concepts, constructing valuable, appropriate content, and showing a perfect language (Xiaoxiao and Yan, 2010).

Since, the ability to write skillfully becomes more and more important, and writing skills play an increasing role in teaching and learning a second language (Chelli, 2006). It is one of the most important factors for students' academic achievement. How learners self-regulate the combination of texts and the strategies they use to initiate and control their writing activities are the key factors in their academic success. Students use a variety of techniques to manage their writing activities (Harris et al, 2010). Accordingly, effective writing skill also requires some multiple skills such as researching, reading complex contexts, comprehending concepts, synthesizing, analyzing, and responding critically to new and related information (Al-Zubaidi, 2012).

On the other hand, the traditional teaching of writing has been the predominant approach in many schools around the world. Foreign language (L2) students’ writing attempts have been usually assessed by means of a single final exam as the main criterion representative of their writing ability. Recently, however, there has been a shift from the dominant past paradigm to using portfolios as a possible means of language learning and assessment (Roohani, 2015).

Due to the indispensable role that writing as a language skill plays in foreign/second
language (L2) learning and teaching, the way it is taught or assessed is of utmost importance. Writing used to be conceived of as a product-oriented approach; L2 teachers taught their students the principles and techniques of writing in a foreign language and they would focus on only the final product which was a completed piece of writing (Nunan, 2003).

However, until now less attention is paid to writing as a whole (including the process of planning, generating and organizations). In schools and universities, they mainly focus on English vocabulary, grammar and structure. In a situation where teachers are generally unaware of English writing, it has caused students to be less motivated and interested in writing skills. Although they can learn a large number of new words, there are still many problems in the process of transitioning from language input to output (Li, 2014).

Complaints about poor productive writing skills of Iranian EFL learners have been heard from the public, teachers and even students all over the country. The reason is claimed to be either the teachers’ lack of competence in L2 teaching process or the learners who are not motivated to produce the language. As Sattari (2012) maintains, Iran’s education system should also take the responsibility for the EFL learners’ deficiencies in productive skills as it is perceived not to present an effective educational system for L2 pedagogy (Pouladian et al, 2017).

Undoubtedly, with this shift of thought on the nature of writing, new methods are needed to assess students’ writing ability. These methods have been named as alternative or authentic methods of writing assessment (Tabatabaei & Assefi, 2012).

There is also a close relationship between language teachings and testing and certainly the other cannot be provided without considering one. Therefore, evaluating the progress of writing also becomes a concern for language experts (Farhadi et al, 2006).

The objectives of this study were to determine the most measured writing constructs based on confirmatory factor analysis and also to evaluate the significant difference (in most measurements) in the specifications of the EFL writing skills test through principal component analysis (PCA) between Iranian advanced and intermediate proficiency level learners.

Literature Review

A number of researchers have measured writing using a levels of language approach that considers writing at three levels: discourse, sentence, and word (Nelson & Van Meter, 2007). Discourse level features are often scored using holistic scoring methods that emphasize the organization, ideas and content, sentence fluency, and conventions of the writing sample. Sentence level measures include sentence length, counts of connectives, number of grammatical errors, clause density, and number of T-units. Word level features include vocabulary diversity, measured as number of different words, as well as letter formation elements and spelling (Nelson & Van Meter, 2007).

Research on second language writing is varied not only on the issues explored but also on the methodologies used. The growth in numbers of students undertaking undergraduate and graduate level degrees in English-speaking countries has triggered researchers’ interest in understanding the writing processes of second language learners and the challenges these learners face. One of the issues that tends to be the focus of attention is the identification of effective practices that impact or improve students’ writing but tend to rely on single aspects of writing that do not consider holistic views of writing; but even when holistic views are considered, they become problematic as they are sometimes not specific enough (Haswell, 2000). Causarano (2011) conducted a study to explore how students in an English as a Second Language (ESL) class in an American university acquire academic writing through Vygotsky’s theoretical and methodological framework. Causarano particularly focused on the learning environment, and how it impacted this acquisition. This acquisition of academic writing was analyzed through the writers’ interaction with the writing tasks, instructors, other second language writers, and environment-writing situations because for this study L2 writers learn and use academic writing “in systematic interaction with the environment in which different constituents played a major role” (Causarano, 2011). Samraj (2013) focused on the discussion sections of master’s theses and research articles in biology in her study of the use of citations. Findings show that there are more rhetorical functions of the citations in these texts than reported in previous studies. Samraj asserted that “the relationship constructed between the text and previous ones is complex and multi-faceted” (p. 308) where writers not only use citations with attribution or background functions but also engage in creating higher order connections with the information from sources.

Researchers of L2 writing have also used quantitative methodologies to explore the changes of writing processes of L2 writers according to their level of writing ability by looking at their writing fluency, complexity of their texts, pausing behaviors, and strategy use (Sasaki, 2000).

Other studies have focused on factors that influence the development of L2 writing. Chae (2011) examined how Korean college students’ knowledge, self-efficacy, strategies, and interest impacted their writing performance while participating in a writing course in three Korean colleges. The study gathered data during 1 semester in three different points from 187 students with language tests, self-efficacy, interest, strategy, performance measures, and interviews. Findings of this study pointed out that students’ interest and self-efficacy do not determine students’ writing performance over time, but they relate to students’ performance at the beginning of the semester.

DATA AND METHODS

This study has a combined method to provide comprehensive answers to research questions. The design of the present study was a sequential combined explanatory method that used qualitative results to explain and interpret the findings of a quantitative study. For the qualitative stage, the researcher designed a semi-structured interview and a structured survey questionnaire based on the methods discussed above.

This study also examined the level of language proficiency significant differences in the most measured structures in the specifications of the Iranian EFL writing proficiency test through principal component analysis (PCA). PCA is a way to identify patterns in data and express data in a way that highlights their similarities and differences. The research tools were:

Oxford Placement Test (OPT)

In order to measure and determine the participants' general English language proficiency and ensure their homogeneity, they were required to take the Standard Oxford Placement Test (OPT). This test was applied primarily to measure and determine the general English language proficiency of the participants and to ensure their homogeneity.

Structured Survey Questionnaire

An identified structured survey questionnaire was designed to measure latent features and relationships between writing components. The questionnaire examined writing methods, assessment methods, writing mechanics, coherence and coherence in writing, perspective, choice of words in students' writing, writing style, genre, and supporting materials in writing. To ensure its validity.

Semi-structured Interview

The additional data collection tool was a special protocol designed by the researcher in the form of a semi-structured open interview with about 20 questions.

Writing Tasks

The fourth instrument consisted of two writing tasks. Each task included some composition topics (e.g. friendship, family, background, education). Participants were asked to write an argumentative paragraph for each topic.

A homogeneous sample of participants among students were selected who scored with a standard deviation (+1SD) above and (-1SD) below the Oxford Test (OPT). As a result, by performing an OPT 60 out of 114 Iranian language learners from Boroujerd Branch Azad University were selected as participants in the present study. They were between 22 and 26 years old and the learners' gender was also considered as a moderating variable in this study. Then, they were equally divided into two male and female groups. Based on their writing skill scores from OPT, they were divided also into advanced and intermediate groups. It was assumed that almost all participants had a similar foreign language learning experience.

According to the importance of "piloting", the researcher designed an experimental study in which participants had similar characteristics to the participants in the main study. Then, all research tools were implemented and the participants were asked to answer all questions. Finally, the cases were reviewed as a pilot by expert judges. The main study was characterized by expert judgment and review of the research literature with ten writing components (mode, assessment, mechanics, coherence, cohesion, point of view, word choice, style, genre, and supporting materials) in EFL writing skills.

Principal component analysis was performed using orthogonal rotation method (Varimax with Kaiser Normalization) and oblique rotation method (Oblimin with Kaiser Normalization). Three methods were also considered for decision making: the Kaiser criterion, the scree test, and retaining of many factors that make up at least 70 of the variance. Moreover, conceptual considerations in addition to three criteria were considered in selecting the four-component model. Also, factor loading was used to interpret the extracted components.

RESULTS and DISCUSSION

Participants' responses to the structured survey questionnaire were coded. The structural survey questionnaire was also analyzed using existing principal component analysis (PCA). All data were analyzed using SPSS 22 software. In according to Table 1, the mean and standard deviation of each writing component is illustrated. In addition, the Analysis N shows the number of valid cases. Here, there are no missing values because the entire sample included 100 participants.

Table 1. Descriptive Statistics of Participants' responses

	Mean	Std. D	Analysis N
Mode	.53	.502	100
Assessment	.62	1.071	100
Mechanics	.5200	.50212	100
Coherence	.6200	1.07101	100
Cohesion	.5300	.50161	100
Point of View	.6500	1.06719	100
Word choice	.5400	.50091	100
Style	.5000	.50252	100
Genre	.5200	.50212	100
Supporting materials	.5500	.50000	100

As seen in Table 2, based on the initial eigenvalues, the first three components are meaningful as they have Eigenvalues > 1. Components 1, 2 and 3 explain 51.08%, 23.01%, and 16.37% of the variance, respectively – a cumulative total of 90.46% (total acceptable).

Table 2. Total Variance Explained

Component	Initial Eigenvalues			Rotation Sums of Squared Loadings a
	Total	% of Variance	Cumulative %	Total
1	4.086	51.08	20.516	1.975
2	1.841	23.010	34.539	1.308
3	1.309	16.365	90.455	1.354
4	1.160	11.603	58.817	1.368
5	0.991	9.907	68.725
6	0.877	8.773	77.498
7	0.831	8.310	85.808
8	0.615	6.149	91.957
9	0.564	5.643	97.600
10	0.240	2.400	100.000

Extraction Method: Principal Component Analysis.

1. When components are correlated, sums of squared loadings cannot be added to obtain a total variance.

In according to figure 1, the plot shows that there are three relatively high (components 1, 2, and 3) eigenvalues. Retain components that are above the ‘bend’, the point at which the curve of decreasing eigenvalues changed from a steep line to a flat gradual slope.

Figure 1. Screen Plot for the Writing Component

In order to achieve significance among Iranian learners at the level of advanced and intermediate skills (in the most measured structures) and based on the specifications of the Iranian EFL writing skills test, t-test of other independent samples was performed.

Table 3. Independent Samples T-test

	Levene's Test for Equality of Variances		t-test for Equality of Means
	F	Sig.	T	Df	Sig. (2-tailed)	Mean Difference	Std. Error Difference	95% Confidence Interval of the Difference
								Lower	Upper
Equal variances assumed	1.309	0.255	-2.73	98	.007	-2.800	1.024	-4.828	-0.772
Equal variances not assumed			-2.73	117.61	.007	-2.800	1.024	-4.828	-0.772

According to Table 3, since Levene's Test is not significant, equal variances assumed is examined. In t-test for Equality of Means, p=0.007 is less than significance level α=0.05, thus, it is concluded that the mean of scores for the advanced and intermediate learners is significantly different. Thus, the related null hypothesis is rejected.

Advanced writing skills are one of the basic components of students' academic skills. Among the four English language skills, writing seems to be the most difficult skill to master. This is especially true for L2 students who are not fluent in the target language and whose only source of L2 exposure is found in the ESL/EFL class.

An appropriate assessment of EFL writing should be based on a thorough knowledge of its scientific foundations. EFL writing skills educators who do not have sufficient assessment knowledge may fall into the trap of turning this activity into a psychometric and statistical process, which omits important aspects of language learning and does not provide any direct feedback for teaching.

In L2 writing, the process of generating ideas and using long-term memory is more complex. Students are confused between long-term memory information (ideas) about the subject and language of expression Scott (1996). He believes that this confusion prevents the creation of ideas. Therefore, it is assumed that learners face this type of problem due to lack of L2 skills. The fact that they have the lowest level of academic experience in higher education also means that they do not have sufficient writing skills.

Some studies to address the inadequacies in writing skills have shown that the use of a contrasting lexical approach has had a significant positive effect on the writing skills of Iranian EFL learners. Teaching through a contrasting lexical approach gives students the opportunity to understand their skillful writing skills, which requires the proper use of different forms of structures and phrases. This approach, in turn, can sensitize them to learn more about linguistic features (Ebrahimi, 2021).

Abedi et al. (2019) also investigated the effect of reverse classroom instruction on the writing skills of Iranian EFL learners through the implementation of the Oxford Rapid Deployment Test (OQPT). The results showed that there was a significant difference between the performance of the experimental group and the control group.

Itani (2021) also investigated the effect of mentor texts on EFL undergraduates' writing proficiency. A quantitative experimental design was used to determine the effect of handbooks on the writing skills of EFL undergraduate students. Comparing the scores of the three components of the articles, which are word choice, sentence structure, and organization, it was concluded that the guide texts can be an effective strategy for teaching these three writing elements to undergraduate students studying at a private university.

CONCLUSION

In this study, independent t-tests as well as principal component analysis (PCA) were used to study the difference between Iranian learners with advanced and intermediate proficiency levels. It is found that there is a significant difference between Iranian learners with high and medium skill level in the most measured structures in the specifications of the Iranian EFL writing skills test. The results show that the mean scores for advanced and intermediate learners were significantly different.

The specifications of the L2 writing skills test are also related to the writing performance and should be developed according to the level of language skills of the learners in L2. The specifications of the writing skills test alone do not appear to be sufficient to develop writing performance. Furthermore, it is a synergy between L2 skills and writing constructs that makes a good writer. Finally, the findings of this study show that L2 skill is an important factor in L2 writing performance and plays a mediating role in the learner's capacity to use writing skill effectively. Although other factors may contribute to differences in writing performance, L2 skill seems to be the most predictable variable.

BIBLIOGRAPHIC REFERENCES

Abedi Parisa, Ehsan Namaziandost & Samira Akbari, 2019. "The Impact of Flipped Classroom Instruction on Iranian Upper-Intermediate EFL Learners' Writing Skill," English Literature and Language Review, Academic Research Publishing Group, vol. 5(9), pages 164-172, 09-2019.

Alamri Hayat Rasheed, Promoting Critical Thinking Skill in the 21st Century: The Role of Saudi Female EFL Teachers in the Writing Classroom, Journal of Applied Linguistics and Language Research, Issue, Volume 7, Issue 4, 2020, pp. 46-69.

Allen, J., & Widdowson, H. G. (1979). Teachingthe communicative use of English. In C. Brumfit & K. Johnson (Eds.), the communicative approach to language teaching (pp. 124–142). Oxford: Oxford University Press.

Al-Zubaidi, K. O. (2012). The academic writing of Arab postgraduate students: Discussing the main language issues. Procedia - Social and Behavioral Sciences, 66, 46–52. https://doi.org/10.1016/j.sbspro.2012.11.246

Birjandi, P., & Sayyari, M. (2010). Self-assessment and peer-assessment: A comparative study of their effects on writing performance and rating accuracy. Iranian Journal of Applied Linguistics, 13(1), 23-45.

Chae, S. E. (2011). Contributions of prior knowledge, motivation, and strategies to Korean college students' L2 writing development. Unpublished doctoral dissertation, University of Maryland, College Park. In ProQuest Dissertations and Theses.

Chelli, S. (2006). Errors in grammar as an aspect of learners’ incompetence: The case of first year students. (Unpublished master's thesis). Available from Mohamed Kheider University, Biskra.

Ebrahimi Fatemeh, Ehsan Namaziandost, the Effect of Teaching Formulaic Expressions through Contrastive Lexical Approach on Iranian Pre-intermediate EFL Learners' Writing Skill, April 2021Journal of Psycholinguistic Research, DOI: 10.1007/s10936-021-09778-z.

Farhady, H., Jafarpour, A., & Birjandi, P. (2006). Testing language skills. Tehran: SAMT publications.

Harris, K. R., Santangelo, T., & Graham, S. (2010). Meta-cognition and strategies instruction in writing. In H. S. Waters & W. Schneider (Eds.), Metacognition, strategy use, and instruction (pp. 226–256). New York, NY: The Guilford Press.

Haswell, R. (2000). Documenting improvement in college writing: A longitudinal approach. Written Communication, 17, 307–352.

Li, Q. (2014). An empirical study on the application of lexical chunk to college English writing. Journal of Language Teaching and Research, 5(3), 682–688.

Nelson, N. W., & Van Meter, A. M. (2007). Measuring written language ability in narrative samples. Reading and Writing Quarterly, 23, 287-309.

Nunan, D. (2003). Second language teaching and learning. Boston, Massachusetts: Heinle &Heinle Publishers.

Mickwitz, Åsa and Suojala, Marja. "Learner autonomy, self-regulation skills and self-efficacy beliefs – How can students’ academic writing skills be supported?: " Language Learning in Higher Education, vol. 10, no. 2, 2020, pp. 381-402.

Quvanch Ziauddin , Kew Si Na, A Review on Impact of Blended Learning on the English Writing Skills, Innovative Teaching and Learning Journal, (ITLJ), 2020, 4 (1), 41-50.

Pouladian Nima, Mohammad Sadegh Bagheri, Firooz Sadighi, An Analysis of Errors in Writing Skill of Adult Iranian EFL Learners Preparing for the IELTS, International Journal of English Linguistics; Vol. 7, No. 3; 2017.

Roohani Ali, Farzaneh Taheri, The Effect of Portfolio Assessment on EFL Learners’ Expository Writing Ability, Iranian Journal of Language Testing, Vol. 5, No. 1, March 2015.

Tabatabaei, O., & Assefi, F. (2012).The effect of portfolio assessment technique on writing performance of EFL learners. English Language Teaching, 5(5), 138-147.

Xiaoxiao, L., & Yan, L. (2010). A case study of dynamic assessment in EFL process writing. Chinese Journal of Applied Linguistics.

Samraj, B. (2013). Form and function of citations in discussion sections of master's theses and research articles. Journal of English for Academic Purposes, 12(4), 299-310.