All research, including educational research, is informed by certain philosophies and assumptions. According to Crotty (1998), these assumptions are about reality, knowledge and the paths that we follow in order to achieve this reality. Certainly, to become acquainted with such assumptions, we need to understand the process of research itself.
In keeping with this view, this paper aims to provide a critical comparative analysis of two published articles that used two different approaches. As a starting point, a brief summary of each article is presented. Next, a critical and in-depth comparative analysis of the nature of the ontological, epistemological and methodological assumptions underpinning each article is provided. Moreover, this paper aims to evaluate and investigate the strengths and weaknesses of the two articles in relation to how the ethical and socio-political issues are dealt with. In addition, it shows the relationship between the theory and practice of the articles under discussion. Finally, I conclude this paper by discussing the lessons to be learned from such a comparison.
Summary of the two articles
The first article
The first article is titled: "m-learning: An experiment in using SMS to support learning new English language words" and is by Carvus & Ibrahim, published in the British Journal of Education Technology, Volume 40, Number 1 (2009). The purpose of the article is to find out the potential of using mobile phones in teaching English language words to 45 1st-year undergraduate students. To investigate this, the authors developed a system called the mobile learning tool (MOLT). By means of pre- and post tests to measure the students' knowledge before and after the experiment, the results showed that the students enjoyed using the tool and learned new words with the help of their mobile phones. This led the researchers to conclude that using the MOLT as an educational tool will contribute to students' success.
The second article
The article "Pupils, the forgotten partners in education action zones (EAZs)" was published by Whitehead & Clough in the Journal of Education Policy, Vol. 19, No. 2 (March 2004). The aim of the research was to review literature on student voices and describe interviews with 139 year 8 pupils in two inner city zone schools purposefully selected to gain an insight into their perceptions about their learning. By means of an interview and a semi-structured questionnaire, the results showed the need to include and encourage pupils as stakeholders to become part of the solution to the difficulties EAZs are charged to address.
Theoretical and philosophical assumptions
The first article can be said to be based on a realist stance, wherein reality exists independently of human thoughts and perceptions (Okasha, 2002). The researchers wanted to see whether the use of mobile technology would be useful in learning new English words. The students' role was to practice activities set by the researchers, and at the end of the experiment, the student were tested to see whether there was an improvement or not. Moreover, the researchers discussed the possibility of generalizing the results to different institutions and subjects, which indicates that they believe in the existence of a single reality (Okasha, 2002). Thus, one can safely argue that the ontological assumption that underpins the research is a realist one, in which the reality that the researchers were seeking (the usefulness of m-learning on the students' learning) is not constructed, but rather exists 'out there' with no intervention from the researcher or the participants (Crotty, 1998).
On the other hand, I believe that the ontological assumption on which the second article is based takes a relativist stance. It can be easily felt that the researchers were not seeking generalization (Guba, 1990). Moreover, words such as "exploring", "perceptions" and "experiences" were used many times throughout the article, which suggests a relativist stance (Creswell, 2007). To illustrate, ontology is relativist "where 'reality' (or multiple realities) is (or are) totally created or constructed through the negotiation of meanings" (Pring, 2000:55). These multiple realities are socially constructed, and so the researchers of the article attempted to explore the pupils' feelings in order to devise a general understanding about their perceptions about their learning. The researchers tried to reach as many realities as the number of the pupils by seeking information from different perspectives (Okasha, 2002). Therefore, reality does not exist separately from the mind, but rather, is totally constructed by people.
Nevertheless, the reader should keep in mind that the above must not be considered as anything more than assumptions I set according to my own belief, as the authors of the two articles did not discuss their philosophical assumptions. Therefore, I would insist that the researchers would have strengthened their papers by taking a clearer philosophical stance.
In other words, in spite of the fact that the vast majority of researchers do not mention their ontological assumptions, I prefer a study in which some hints about these assumptions are given to the reader. That is not to say that a researcher has to write, "I am a relativist. Therefore, I will explore ..." (Crotty, 1998, p.13), but some indications of the philosophical assumptions are required, at least from my point of view. Starting with the methodology without mentioning the assumptions that stand behind it is a flaw researchers must take into account, at least when rationalising the methodology of their research.
As we move from a higher level to a lower one in the hierarchical classification of the philosophical assumptions, the case becomes progressively clearer. Having examined the title, purpose and questions addressed by the first article, it became easier to consider the epistemological assumptions that stand behind the methodology as positivism (Della & Keating, 2008).
The researchers attempted to measure the effectiveness of an m-learning system and set a hypothesis to verify the relationship between "cause and effect", which leads us to believe that their epistemology is positivism. That is to say, meaning is imposed on the object (pupils) by the subject (researchers), where the researchers wanted to know the impact of mobile technology on the learning of new English words by using a new m-learning system. To illustrate, the researchers did not intervene in reaching the final result; rather, they introduced an m-learning system (treatment) and at the end of the experiment they tested the students to measure the improvement in their English words. Therefore, they were objectivist (positivist), using scientific methods combined with statistical analysis to reach reality (Crotty, 2002).
On the other hand, the authors of the second article attempted to explore perspectives that exist in multiple minds (realities), which implies that different people may construct meaning in different ways (Creswell, 2007). This notion suggests an interpretivist stance. The epistemological stance is identified as Interpretivism (sometimes called constructionism) because the researchers attempt to construct meaning by interpreting the pupils' perspectives using different methods such as semi-structured questionnaires and interviews. In other words, reaching the meaningful reality through these interpretations comes as a result of the engagement of the researchers (the subject) with the pupils (the object) in the two schools. Therefore, the researchers believe that reality/meaning exists in the pupils' minds and needs to be constructed by the researchers. It is worth mentioning, however, that the researchers did use some numbers and percentages, but they were very simple and descriptive in their nature and the researchers were not seeking generalization through them. This means that the reader can consider those numbers, at most, as a triangulation on the level of methods, and must NOT consider this as a positivistic aspect of the study, because this consideration would contradict the interpretive nature of the research.
As mentioned above, I would recommend mentioning the study's epistemological assumptions to the reader. Starting by discussing the methodology without mentioning the assumptions that stand behind it is a flaw I would have taken into account in order to rationalise the methodology of my research, irrespective of the fact that many researchers would disagree with this view.
Methodology - Study 1
It has been claimed by the authors of the first article that they adopted an experimental methodology, clearly hinting many times at a True Experimental Design. I would argue that it was undoubtedly a Weak experimental study. Conducting an experiment using a treatment combined with pre- and post-tests does not guarantee that a true experiment is being conducted, as the main condition that has to be available to claim a true experiment is Random Assignment (Druckman, 2005). The latter condition cannot have been achieved, simply because the design did not include a control group as the whole sample made up the experimental group. Therefore, I can safely say that the design was the One-Group Pretest-Post-test Design, as follows (Morad, 2006):
Pre-test >>>> Treatment >>>> Post-test
In spite of the fact that it would have been sound to use either a Weak or a True experimental design to achieve the purpose of the present study, the researchers made no mention in their article as to whether they considered the study to be a Weak or a True experiment.
On the other hand, questionnaires were filled in by the students to express their opinions about the MOLT system, which can be considered as a triangulation through which rich findings can be obtained. Furthermore, the sampling process, time, space and the circumstances surrounding the experiment were elaborated in detail.
Methodology - Study 2
Although neither the methodology nor the purpose of the second study was clearly discussed, I was able to infer the general aim of the study, which results in my understanding of its methodology. I believe that the methodology of the second study falls within the category of Case Study research, as most of the features of this type of research are evident in the article. According to Creswell (2007, P.73), "case study research involves the study of an issue explored through one or more cases within a bounded system" which are EAZs in this article. In addition, words such as "describe", "explore", "experiences" and "meaning" are indicative of a Case Study, and the latter words were common visitors throughout the study (Creswell, 2007).
Furthermore, it seems that the methodology could have been distorted by other parties, as it was not purely set by the researchers themselves. The researchers discussed several compromises that had to be made as they were setting their methodology with the zone director and with the head teachers of the two schools, as follows (Whitehead & Clough, 2004, P.218):
"The research methodology was designed by the research team with EAZ partners and discussed with and agreed by the zone director and the head teachers of the two schools ... involving zone staff, unfamiliar with research design and data collection, could increase problems of reliability and validity"
However, I will not elaborate this point in this section, as I will discuss it in the socio-political section of this paper.
Nonetheless, certain fundamental remarks are worth stating: besides the interviews, questionnaires were filled in by the pupils, which can be considered as a form of triangulation that allows the collection of more valid and reliable data. Moreover, the sampling, time, space and circumstances surrounding the data collection procedure and the qualifications of the data collectors were clearly discussed by the researchers.
In my opinion, some amendments to the research design are recommended. For example, the methodology should have been clear and straightforward, and should have followed one of the five established approaches given by Creswell (2007) as much as possible, and I would clearly discuss the changes in the methodology made by the different socio-political parties.
Ethical issues - Study 1
Although the authors of the first article did not discuss many of the ethical issues that arose, there are some issues that need to be addressed. According to the British Educational Research Association's (BERA) ethical guidelines, researchers must take into account three types of responsibilities (BERA, 2004); the first type is responsibilities to participants. With regard to this, the researchers did not clearly discuss issues of confidentiality and anonymity (Pring, 2004). For example, the researchers did not notify the reader as to whether the questionnaires were confidential. Moreover, the researchers did not discuss participants' right to withdraw from the experiment. On the other hand, the participants had been asked for their agreement before conducting the experiment. It is worth mentioning that the students did not have to bear the cost of sending SMS messages, as the local mobile phone company provided financial help by not charging for those messages.
The second type of ethical responsibilities, which is devoted to sponsors of research, cannot be discussed, as the researchers did not state who the sponsor of the study was, and this can be considered a flaw that the researchers should have taken into account.
The third type of ethical responsibilities is dedicated to the community of educational researchers. As implied above, the research was, to some extent, well conducted. For example, the purpose of the study is clear and consistent with the methodology. Furthermore, both the validity and reliability of the methods were appropriately manipulated, which, to some extent, allows the possibility of generalizing the findings. Inevitably, any well-conducted research maintains the integrity and reputation of the educational research community.
Ethical issues - Study 2
According to Butler (2002), discrimination based on ethnicity must be justified, which means that the division of participants into subordinate independent variables on the basis of ethnicity must be rationalised. I would argue that the researchers divided their sample into white students and black ones haphazardly and unjustifiably. Having reviewed the relevant educational literature, I found that black students were usually both in the minority in research samples and under-achievers, which might make sense of this discrimination if the reason for such a division is to probe the reasons beyond their low achievement levels and then offer solutions. We all know that the target group of this division is black students, but the fact is that no evidence was given in terms of the differences between white and black students with regard to educational and economic levels. On the contrary, in this case, black students formed the majority (48%) among the other groups at the two schools and were not under-achievers, as the results of the study showed.
In my opinion, this discrimination might have had a negative impact on the black students, as no obvious reasons were given. In spite of the fact that I am not a black person, I would be offended regardless of whether if I were black or white. If I were black, I would ask myself about the reason for such discrimination. On the other hand, if I were white I would be offended by the belittling of my classmates.
Moreover, the reader was not given any information concerning the source of funding for the research. However, I will assume for the purpose of this critique that it was sponsored by the Ministry of Education. According to BERA, the second type of ethical responsibilities is devoted to sponsors of research. Limitations were found throughout the paper and some of them were quite essential, such as the ambiguity of purpose and the interference from various parties in setting the research methodology. Axiomatically, this would have a negative effect on the sponsor's right to obtain research of a high quality (McNamee & Bridges, 2004). I will assume that the ministry was seeking valid findings on which decisions would be based. In this case the ministry's officials would definitely ask themselves the following question: was the research worth the money spent on it? Given that the research covered two schools for three weeks. I believe that I can answer the question on their behalf, as I am sure the answer was "no, it was not worth it". In order to justify this answer, let us assume that we were the officials and we needed clear findings to make decisions: what were these findings? They were stated as follows: "the results showed the need to include and encourage pupils as stakeholders to become part of the solution to the difficulties EAZs are charged to address" (Whitehead & Clough, 2004, P.215). Here I leave this question to the reader: how could an official employee at the Ministry of Education make use of these findings? In short, I believe that the answer "Yes" is suitable for the following question: does such research badly affect the reputation of the educational research? On the other hand, the principals of right to withdraw, confidentiality and anonymity seem to have been adhered to.
The socio-political context - Study 1
Although nothing was mentioned concerning the source of funding for the research, I have a feeling that the funder was the Near East University where the researchers work. My intuition might be due to two factors: first, the study was applied research and not a basic piece, and the governmental institutions tend to choose applied research rather than basic research (Hammersley, 2003). Second, there were few signs that indicate kind of facilities offered by the university. For example, the researchers stated the following: "the survey was administered at the university by asking the students to fill the survey questionnaire during a lecture session, where the students were given 20 minutes to complete the questionnaire" (Carvus & Ibrahim, 2009, P.84). I believe that I can draw two scenarios on the basis of the latter hypothesis.
In the first scenario, I will presume that the researchers themselves demanded the funds to facilitate the conducting of their research. In this case, the role of the university was solely to provide some researchers with money to conduct their research: the university might not have been interested in its findings, but funded it as a grant to encourage scientific research. As a result, the main audience of the research was us, by whom I mean academics, researchers, teachers and students. It should be kept in mind that publishing research in a journal does not necessarily mean that we are the target audience; on the basis of this fact, I will draw the second scenario.
I will assume that the university was seeking valid findings on which decisions would be based. In this case, the university officials would definitely ask themselves the following question: was the research worth the money spent on it? I believe that I can answer the question on their behalf, as I am sure the answer was "yes, it was worth it". In order to justify this answer, let us assume that we were the officials and we needed clear findings to make decisions: what were these findings? Were they valid and reliable enough to be generalized to other departments? The answers are that the findings were clear, valid and reliable. Therefore, I believe that decisions can be made on the basis of the findings provided that other carefully conducted studies support these findings.
It is worth mentioning that the researchers implicitly encouraged policy-makers to generalize the findings. For instance, they mentioned that, "most teachers kept an eye on the research throughout its duration ... they also discussed the possibility of using similar tools in their own lectures" (Carvus & Ibrahim, 2009, P.89).
On the other hand, there is an important question that needs to be answered: are there individuals who might be negatively affected if the findings are generalized? Using technology sometimes leads to the exclusion of employees. For example, there might have been teaching assistants who felt threatened by the findings. It is impossible, however, to elaborate the latter point, as there are no indications in the article to help me do so.
The socio-political context - Study 2
Regarding the second study, the researchers exceeded the domestic policy of their country to the global one and stated the following (Whitehead & Clough, 2004, P.216):
"There has been a move towards greater involvement by pupils in decisions about aspects of school life, and such a stance has been given legitimacy through the United Nations Convention on the Rights of the Child. In reality though this has remained far from common practice ..."
Inevitably, such a statement would not satisfy the politicians in the researchers' country, simply because such statements may lead to foreign interference in the country's domestic affairs, or at least cause embarrassment by parties who had failed to adhere to the convention they had signed.
Within the country, as mentioned above, the research methodology seems to have been changed to satisfy the officials. However, the researchers themselves acknowledged the possibility of losing the reliability and validity of their research, as follows (Whitehead & Clough, 2004, P.218):
"Involving zone staff, unfamiliar with research design and data collection, could increase problems of reliability and validity."
It is worth mentioning here that the officials were able to compromise with the researchers in terms of methodology, and this serves as a vivid example of the degree to which a researcher might counter the political power of officials.
At a lower level, the researchers foresaw the difficulty they might have to face from the teachers and accepted that, "the findings were unpredictable and might be seen as threatening by some teachers" (Whitehead & Clough, 2004, P.218). Questions regarding the latter concern were left unanswered, such as what happened during and after the research? This allows us to weave our own scenarios. To me, it appears that the research had been aborted even before it obtained its findings.
Furthermore, the researchers and their team seem to have disagreed on the issue of whether to record the interviews or not. According to the researchers, the reason for this disagreement was that some members of the research team believed that the students and their parents would be less willing to co-operate because the local families seemed to have had negative experiences of recorded interviews (Whitehead & Clough, 2004). Eventually, they decided not to record the interviews, as there was resistance by the majority of the team; hence, the research lost, for me at least, part of its reliability. The latter argument serves as an example of how the sociological dimension can shape or re-shape the structure of any given educational research.
Nonetheless, this study can be looked at as a scientific struggle to capture valid findings, but is badly affected by the socio-political dimension. This research gives us a lesson to learn about how to predict and tackle this dimension even before setting out to conduct a study. Educational researchers are now facing a bitter fact: alongside the need to improve the quality of educational research (Gretler, 2007), there are always different ideologies and agendas within any country. Researchers, during their professional life, are bound to collide with some of these ideologies and agendas, but this fact must not prevent us from seeking valid knowledge. This cannot happen unless the researchers tackle those different parties wisely so that we can achieve our studies' aims, as long as they are ethical.
The relationship between theory and practice
Biesta and Burbules (2003, p.105) argue that, "theory no longer comes before practice, but emerges from and feeds back into practice". Pring (2000:129) states that "no practice stands outside a theoretical framework''. Although I concur with the latter perspectives, I will not base this section on the question of which is being informed by the other - theory or practice. Indeed, I believe more that it is about time educational research offered solutions to the problems that practitioners are encountering. Therefore, I will discuss the relationship between theory and practice on the basis of "what works" (Biesta, 2007), as I adopt this point of view.
In the first article, the research followed a positivist approach in which the researcher tried to find out whether m-learning significantly improved students' learning of new English words. The researchers held the theory that technology could be used in teaching English words. This theory assumes that when students use m-learning, their achievements in memorizing new words will improve. When the researchers put this theory into practice, their hypothesis was supported. Therefore, the relationship between theory and practice is clear in this article. To illustrate, this article is offering an evidence-based solution to a problem that informs teachers of "what works" to reach desirable "outcomes". As a result, I can safely say that in this research, there is virtually no gap between theory and practice. Furthermore, taking into account both the ontological and the epistemological stance of the present study, it is possible to generalize the findings so that different teachers in different countries might find the findings useful in practical terms, as a desirable end is given and the reader is provided with the means through which the end is reached.
On the other hand, in the second article, the researchers aimed to review literature on student voice and describe interviews with 139 year 8 pupils in two purposefully selected inner city schools to gain their perceptions about their learning. As the sample was purposefully selected, it is obvious that the researchers did not want to generalize their findings. Therefore, any practical findings would be difficult to apply to a different reality, simply because the study itself, as an interpretive one, implies the uniqueness of the reality to which it was applied. However, no practical findings were given, even for the local practitioners, which allows me to say that the gap here between theory and practice is considerable. In other words, let us suppose that we were the teachers from the two inner schools and we needed an evidence-based solution to a problem: axiomatically, we would ask ourselves, "what were the findings and how could we make use of them?" The findings were that there is a need to include and encourage pupils as stakeholders to become part of the solution to the difficulties EAZs are charged to address. I leave the second question to the reader, as I could not answer it.
The main strengths and weaknesses in the two articles
In the first article, the main strength is the triangulation of the methods and analysis of the data. The researcher used pre- and post-tests, as well as administering questionnaires to see whether the students enjoyed the MOLT system or not. Moreover, statistical validity and reliability were achieved: for example, the internal consistency of the questionnaire using Cronbach alpha was found to be 0.94, which is rather a high score. Another strong point is that most of ethical principles were adhered to: for example, the students did not have to bear the cost of sending SMS messages, as the local mobile phone company provided financial help by not charging for those messages. In addition, the findings offer an evidence-based means that helps English language teachers to achieve their educational aims.
On the other hand, one of the weaknesses of the research is the frequent hints at a true experimental design, when in fact it was a weak one. Another drawback of the research is that the researchers seem to have forgotten to discuss the principle of confidentiality, which is an important ethical issue.
In my opinion, one of the strengths in the second article is the triangulation of methods to collect the data: using interviews as well as questionnaires to probe for additional data can be regarded as an attempt to overcome any weaknesses in subjectivity. Moreover, the circumstances surrounding data collection were clearly discussed by the researchers, such as time, space, the qualifications of the data collectors as well as the impact of the socio-political dimension of the study.
On the other hand, I believe that one of the weakest points in this research is that the researchers did not use observation as a method. To justify this, I believe that using observation would have revealed more valid and reliable information. As mentioned above, the methodology of this research was badly affected by the socio-political dimension and observation as a method might overcome some of the latter effects because the pupils would have been observed in their natural setting. In addition, there is a huge gap between theory and practice in this research, as educational research is facing global political pressures, which are to some extent justifiable. I believe that educational researchers must focus more on finding evidence-based solutions to the problems teachers are encountering in the field, at least at this stage.
This assignment and the module have given me a valuable opportunity to understand the basic elements of educational research. Now, I am able to identify the philosophical assumptions behind any study, which will help me to critique it and identify its strengths and weaknesses. One of the most beneficial aspects that I gained from this module is that my way of thinking has developed to be more critical rather than being passive and descriptive.
Moreover, prior to this module, I was not aware of the importance of ethical principles in educational research. Now, I am aware that I need to pay more attention to participants' rights. It is understood that such an exercise is crucial for any prospective researcher. I feel now that I am able to apply agreed ethical principles, such as those set by BERA, to my future research.
There are always different ideologies and agendas within any country. As a researcher, during my professional life, I am bound to collide with some of these ideologies and agendas, but this fact must not prevent me from seeking valid knowledge. This cannot happen unless I tackle those different parties wisely so that I can achieve my studies' aims, as long as they are ethical.
In terms of the relationship between theory and practice, I believe that educational research is being faced by socio-political pressures all over the world, which is, to some extent, due to the huge gap between theory and practice in the field of educational research. As I am convinced that those pressures are justifiable, I declare, today, that one Aristotelian researcher can be added to the number of quantitative researchers.
List of References
- BERA (2004). Revised Ethical Guidelines for Educational Research. Retrieved Nov. 5, 2009 from: http://www.bera.ac.uk/publications/pdfs/ETHICA1.PDF
- Biesta, G. (2007). Why "What Works" Won't Work: Evidence-Based Practice and the Democratic Deficit in Educational Research. Educational Research, 57 (1), 1-22.
- Biesta, G. J. J. & Burbules, N. C. (2003). Pragmatism and Educational Research. Boulder, Co.: Roman & Littlefield.
- Butler, I. (2002). A Code of Ethics for Social Work and Social Care Research. British Journal of Social Work, 32,.239-248.
- Carvus, N. & Ibrahim, D., (2009). m-learning: An experiment in using SMS to support learning new English language words. British Journal of Education Technology, 40 (1), 78-91.
- Creswell, J.W. (2007). Qualitative Inquiry and Research Design: Choosing among five approaches. London: SAGE
- Crotty, M. (2003). The Foundations of Social Research. Meaning and Perspective in The Research Process. London: SAGE
- Donatella D. and Michael, K. (2008) Approaches and Methodologies in The Social Sciences: A pluralist perspective. Cambridge: Cambridge University Press.
- Druckman D. (2005) Doing Research: Methods of inquiry for conflict analysis. London: SAGE.
- Gretler, A. (2007). The International Social Organization of Educational Research in Europe: Reviewing the European Educational Research Association as an example -facts and questions. European Educational Research Journal, 6 (2), 174-189.
- Guba, E. (1990). The Paradigm Dialog. London, SAGE Publications.
- Hammersley, M. (2003) Qualitative Social Work: Social research today: Some dilemmas and distinctions. London: SAGE.
- McNamee, M. & Bridges, D. (2004) The Ethics of Educational Research. Oxford: Blackwell Publishing.
- Morad, S. (2006) Experimental and Quasi experimental designs for research. Cairo, Dar Alkitab.
- Okasha, S. (2002). Philosophy of Science: A very short introduction. Oxford: Oxford University Press.
- Pring, R. (2000). Philosophy of Educational Research. London/New York: Continuum.
- Samuel B. and Neil, J. (2004) Using SPSS for Windows and Macintosh: Analyzing and Understanding Data. New Jersey: Pearson Education.
- Whitehead, J. and Clough, N. (2004). Pupils, the forgotten partners in education action zones, Journal of Education Policy, 19 (2), 215-227.