Chapter 3: Methodology
This chapter is concerned with the design of the research methodology conducted. Busha and Harter (1980), have described research methods as the "procedures and methods by means of which the identified problem" has been studied. A combined approach using both quantitative and qualitative methods has been the research methodology chosen for this research, as this would allow for the necessary flexibility and level of details of study required. This research employs a mix research methodology (quantitative and qualitative) through primary and secondary data collections including survey and gathering information from publications such as reports, journals and magazines.
Combining Qualitative and Quantitative Method Approach
Research is all about contributing to knowledge. Epistemology refers to the assumption about how knowledge can be obtained (Hirschheim, 1992). Any research is based on some underlying assumptions about what methods are valid and appropriate. As Chua (1986) states, researchers share "a constellation of beliefs, values and techniques" and they "circumscribe definitions of what are worthwhile problems and acceptable scientific evidence".
Qualitative method and quantitative method are two general approaches for research methodology that widely recognized. Qualitative method can be define as "a process of inquiry with the goal of understanding a social or human problem for multiple perspectives; conducted in natural setting with a goal of building a complex and holistic picture of the phenomenon of interest" (Monash University Information Technology, 2009). By contrast, the definition for quantitative method is defined by Monash University Information Technology (2009) as "an inquiry into an identified problem, based on testing a theory composed of variables, measured with numbers, and analyzed using statistical techniques; the goal is to determine whether the predictive generalization of a theory hold true". The selection of which research approach is appropriate in a given study should be based upon the problem of interest, resources available, the skill and the training of the researcher, and the audience for the research. Although some research may incorporate both qualitative and quantitative methods, in their 'pure' form there are significant differences in the assumptions underlying these approaches, as well as the data collection and analysis procedures used.
While the quantitative approach is still the most widely applied research method, it is being more often critiqued when used on its own. Exclusive reliance on statistical or experimental testing of hypotheses has been criticized widely, where some have referred to its effects as "disastrous" (Kaplan and Duchon, 1988). Kaplan and Maxwell (2005), argue that when the research goal is to understand a phenomenon from the participant's points of view and contexts, the meanings and insights are largely lost when trying to quantify textual information; hence they argue the importance of qualitative research. The growing recognition of the value of qualitative research methods is emphasized widely. However, these too have been criticized for their lack of generalisability and the inherent weaknesses of different qualitative methods (Benbasat, Goldstein and Mead, 1987). Thus, the value of combining research methods in information systems research has received significant applauds. Nevertheless, multi-methods approaches are still not the dominant call (Gable, 1994). Failure to discuss how multi-methods approaches can be combined with one another (Kaplan and Duchoh, 1988), practical concerns over possible contradictory results from multi-methods, doubts over the legitimacy and feasibility of combining different research approaches and potential vulnerability that researchers may feel when moving away from their value systems or commonly practiced methodology (Gable, 1994) are all contributing elements to the slow adoption of multi-methods research designs in information systems.
The advantage of using multi-methods research design has been clearly discussed (Gable, 1994; Kaplan and Duchon, 1988; Lee, 1991) and is no longer negligible.
"Collecting different kinds of data by different methods from different sources, provides a wider range of coverage that may result in a fuller picture of the research problem"..."it provides a richer, contextual basis for interpreting and validating results" (Kaplan and Duchon, 1988).
This evidence encourages researchers to seriously consider a multi-methods design within their new studies.
Overall Research Design
Data collection is considers as the crucial stage in gathering all required information from the fundamental in achieving main objectives of the study. This research employs a mix research methodology (quantitative and qualitative) through primary and secondary data collections including survey and gathering information from publications such as reports, journals and magazines. This would allow for the necessary flexibility and level of detail of study required. The relative newness of the field of study and lack of written evidence in the literature has determined this combined research approach in order to find out as much information as possible. Integrating quantitative and qualitative methods lends depth and clarity to this study. This combination of approaches is necessary because of the wide range of data needed to develop the prototype system.
For this study, the primary data were collected from questionnaires survey distributed to MLIS students of FCSIT, University of Malaya. The questionnaire was created to examine the aspects identified in the literature and meant to fulfill the objectives pre-determined. Secondary data are collected from literature review on books, journals, articles, magazines and websites which emphasize on the DL and DLE areas. These data are important to generate and design the questions for the questionnaires of primary data collection. The secondary data reviewed prepared the platform for the formulating of questionnaire. Questionnaires can be designed to determine what people know, what they think, how they act or plan to act. It also allows data to be collect quickly in a standard manner. It allows both quantitative data (numeral values) and quantitative data (comments from open ended questions) to be collected. In this study, a questionnaire was developed aimed at determining the problems or difficulties face by students when study DLs course and to identify how DL course should be conducted.
For the survey questionnaire, the questions themes were determined by carrying out an initial literature review looking at the issues relating to student's learning of DL course for DLE. This enabled a better understanding of the subject and therefore the questions could be tailored to provide only information in areas of interest. This method allowed relevant primary data to be obtained. During the questionnaire's preparation stage, the questionnaire was designed and is aimed to enable a more objective responses rather than superfluous view that contained to concrete substance or justification. The questionnaire has been designed in order to obtain a wider range of views towards the study in such a way that simple and straightforward. Respondents only need to select or choose the responding answer. In some of the question, the respondents have to contribute their views and opinion in writing to fulfill the objectives of the study.
An overall research design used within this study is depicted in a flow diagram in Figure 3.1 which graphically depicts how the outcomes of the literature review lead into the survey phase in an attempt to integrate the two research method (qualitative and quantitative). The different stages of the research are represented by ovals, information flows and their directions are depicted by arrows, where the rectangles depict the outputs derived from the different stages. The first stage of the research is to define the objectives and questions for this study then based on these, the previous works related to DLs and DLE in academic setting were studied through literature review. From this review, the instrument for data collection was defined. Then, data collection and requirement analysis was conducted to investigate the problems or difficulties face by students when studying DLs course and the implication if the course is conducted using DLs environment. The findings were transferred into statistical analysis for further analytical work. The next stage was designing the system framework and develops the prototype system. The prototype system would be developed using DSpaceTM, a DL open source platform. During this stage, the core modules of the prototype system were developed and the functionality of the system was tested.
Survey Design for Data Collection
The survey method enables a researcher to collect data from large or small populations (Colorado State University, 2009). The researcher selects a sample of respondents from a population and administers a standardized questionnaire to them. This questionnaire (or 'survey') can take many forms. It can be a written document, an online questionnaire, a face-to-face interview, or a telephone interview that will be completed by the data gathered from the person being surveyed (Fink and Kosecoff, 1985; Pinsonneault and Kraemer, 1993). As for this study, the written document form of survey has been selected and distributed among MLIS students of FCSIT, University of Malaya, as survey population.
Characteristics of Survey
Surveys recapitulate one of the most common types of quantitative research methodologies. Scheuren (2004) suggests that the survey method's proliferation is in response to the present society's demand for prompt and accurate flow of information and preferences, needs, and behavior. Furthermore, surveys are also recognized for their ability to complement other forms of methods (Gable, 1994; Newsted et al., 1998). In such multi-method research designs, surveys serve as a way to add to one's knowledge through "triangulation". Thus, while surveys are typically used in quantitative research context, they can assist qualitative studies as well (Newsted et al., 1998).
Survey research is known to be the most widely used method in the MIS field (Kraemer and Dutton, 1991). Sedera et al. (2003) summarize four main reasons for the method's proliferation in the IS field, they include the method's ability to:
- Document the norm accurately identifying extreme outcomes and delineating associations between variables in the sample.
- Analyze data both at aggregate and at individual levels
- Facilitate more rigorous hypothesis testing and generalization by giving more cases (samples) and more systematic data than other methods.
- Potentially add to the inventory of previously well-developed research instruments. Such an inventory of items allows the IS field to evolve as a discipline where research activities can be expedited without re-inventing instruments, once instruments are sufficiently validated.
Pinsonneault and Kraemer (1993) identify three distinct characteristics of the survey method:
"It is a quantitative method, which requires standardized information form and/or about the subjects being studied"
"The main way of collecting information is by asking people structured and predefined questions...which might refer to themselves or to some other unit of analysis"
"Information is generally collected from a fraction of the population, but it is collected in such a way as to be able to generalize the findings to be population"
Survey Instrument Design
Questionnaires should be self-explanatory, so they can be 'self-administered', filled out in privacy without supervision. They must consist of general directions on the overall survey and specific instructions (when deemed relevant) for different sections within the instrument (Fink and Kosecoff, 1985). Mandatory and optional questions must also be clearly indicated.
"The selection, wording and ordering of questions and answers requires careful thought and a reasonable command of language." (Fink and Kosecoff, 1985).
A through concern was given to this study to abide by these recommendations, and clear instructions were provided upfront at all instances (see appendix for a copy of the paper-based questionnaire instrument used in this study).
In this study, the survey has been conducted to identify the problems or difficulties face by students when study DLs course and to identify how DLs course should be conducted. Fink and Kosecoff (1985) suggest that many human attitudes and feelings are subject to a range of definitions. Even when one "is not measuring theoretical ideas... You must define your terms... It is best to adopt a respected point of view... and when possible, an already existing and tested survey form" (Fink and Kosecoff, 1985). Hence, the definitions of DL and DLE were provided at the beginning of the questionnaire for this study to give understanding about the questionnaire and assist the respondents to answer the questions. These definitions were derived after reviewing definitions that were identified from the literature.
The survey questionnaire for this study is divided in two sections. The first section was targeted at respondents that have taken DL course, whereas the second section was for respondents that have not taken DL course. By splitting the questions, it were imposes certain restrictions on the questions to the respondents so that both of the respondent types were directed to the appropriate questions as well as to facilitate data analysis. The survey consists of questions that required respondents to provide information on their demographic characteristic, with the remaining questions being designed to identify the respondents' views about DL course.
As discussed earlier, this study's survey phase had an objective to identify a way to conduct DLs course effectively. A few descriptive questions, capturing demographic details of the respondent were included at the beginning of the questionnaire. Table 3.2 illustrates the final descriptive questions that were included in the survey, together with their intended purposes. Whenever applicable, these descriptive questions were made mandatory to assist in gaining a comprehensive snapshot of the desire characteristics of the sample. However, some are optional, and this was either due to the respondent's assumed lack of knowledge to respond or instrument design issues.
It was important to get the design of the questionnaire tailored to the situation that it was to be used in.
"The sampling procedures used in survey research will depend up on such factors as the nature of the study, available financial support, time constraints and the accessibility of respondents." (Busha and Harter, 1980).
Questionnaires are used to collect data from people. Thus, the sample from whom this information is collected and the number of respondents becomes a critical design aspect as it will significantly influence the wordings and focus of the items used (Fink and Kosecoff, 1985).
A. Who to Collect Data From
Past success related studies have shown the importance of properly identifying the correct 'view' to collect data from (Seddon et al., 1999). It was important that the sample population of the questionnaire is relevant to the area of study.
"Investigators should consult with authorities in the topical area of research so that first-hand, additional insights, suggestions and information can be obtained from these experts", (Busha and Harter, 1980).
Therefore, MLIS students of FCSIT, University of Malaya were chosen to be sent the questionnaire. This study targeted MLIS students of FCSIT, university of Malaya as they are might be familiar with the term 'Digital Library'. Besides, these students are offered DL subject as elective subject in their study programmes. Also, given the unit of analysis of this study (using DL for education purposes); it was important that the sample group targeted had exposure to the term 'Digital Library'. Thus, MLIS students seemed the most appropriate single targeted respondent group, as they are knowledgeable on DL environment and basic IT, and are also able to respond on the best way to conduct and learn DL course.
B. Number of Respondents in the Sample
A common goal of survey research is to collect data representative of a population as the data gathered is used to generalize observations back to the population (Bartlett et al., 2001). Thus, planning for an appropriate sample size is important in a strong survey design. Determining the sample size can be largely influences by the goals of the survey and the tests that are planned to be conducted on the data, as different test types have different sampling size heuristics. In general, larger sample is much appreciated in survey research as larger samples aid in more accurate validity and reliability testing. Formal sample size calculations are used with different sampling techniques to identify the minimum required sample size for testing a given set of items.
The response rate tends to be a subjective concept. Generally, the rule is "the higher the better" (Fink and Kosecoff, 1985). In this study, 45 survey questionnaires were distributed to 45 MLIS students in FCSIT UM. From this distributed survey, 33 survey questionnaires were answered and returned. This means that the response rate for the survey is approximately 75%.
Usability Evaluation Method
This chapter first discussed the epistemological foundation of this study, classifying this research as a mix-research methodology. The chapter continued its discussion with an overview of applying a multi-method approach. The generic advantages and pitfalls are discussed here, with an introduction to the qualitative method and quantitative method integration that were used within this study. Next, the detailed research design was unfolded, describing the different process stages of this study and how the different outputs feed in as inputs at the different phases.
The primary data were collected from questionnaire survey delegated to MLIS students at FCSIT, University of Malaya. Secondary data were collected from literature review on books, reports, journals, articles, websites and magazines, which emphasize on the DLs and DLE. The secondary data reviewed prepared the platform for the formulating of questionnaire. Questionnaires can be designed to determine what people know, what they think, or how they act or plan to act. It also allows data to be collected quickly in a standard manner. It allows both quantitative data (numeral values) and qualitative data (comments from open-ended questions) to be collected. In this study, the survey questionnaire was developed aimed at determining the problems or difficulties face by students when study DLs course and to identify how DL course should be conducted. This chapter also described the entire design of the survey phase in detail. It first briefly introduced the survey research method, and then described the specific details that were considered when designing the overall layout and mode of survey distribution. The next chapter (Chapter 4) of this thesis will present the results of the analysis conducted on the data collected through this effort.
To summarize the overall research design for this study, the first stage is to study the previous works related to DLs and DLE in academic setting, as reported in chapter 2. Then, data collection and requirement analysis was conducted to investigate the problems or difficulties face by student when studying DLs course and the implication if the course is conducted using DLs environment. The findings were transferred into statistical analysis for further analytical work. The next stage was designing the system's framework and develops prototype system. The prototype system would be developed using DSpaceTM, an open source DL platform. During this stage, the core modules of the prototype system were developed and its functionalities were tested.
- Bartlett, J. E., Kotrlik, J. W. and Higgins, C. C., 2001. Organizational Research: Determinig Appropriate Sample Size in Survey Research. Information Technology, Learning, and Performance Journal, 19(1), pp. 43-50.
- Benbasat, I., Goldstein, D. and Mead, M., 1987. The Case Research Strategy in Studies of Information Systems. MIS Quarterly, 11(3), pp.369-385.
- Busha, C. H. and Harter, S. P., 1980. Research Methods in Librarianship: Techniques and Interpretation. Academic Press, New York.
- Colorado State University, 2009. Writing@CSU: Writing Guide: Advantages and Disadvantages of the Survey Method. Available from: http://writing.colostate.edu/guides/research/survey/com2d1.cfm [Accessed 20 July 2009].
- Fink, A. and Kosecoff, J. B., 1985. How to Conduct Surveys: A Step-by-Step Guide. Sage Publications, Beverly Hills.
- Gable, G. G. 1994. Integrating Case Study and Survey Research Methods: An Example in Information Systems. European Journal of Information Systems, 3(2), pp.112-126.
- Hirschheim, R. 1992. Information Systems Epistemology: An Historical Perspective. In R. Galliers (Ed.), Information System Research: Issues, Methods and Practical Guidelines. pp. 28-60. Oxford: Blackwell Scientific Publications.
- Kaplan, B. and Duchon, D., 1988. Combining Qualitative and Quantitative Methods in Information System Research: A Case Study. MIS Quarterly, 12(4), pp. 571-587.
- Kaplan, B. and Maxwell, J. A., 2005. Qualitative Research Methods for Evaluating Computer Information Systems. In Anderson, J.G. and Aydin, C., eds. Evaluating the Organizational Impact of Health Care Information Systems, 2nd ed. Springer New York. pp. 30-35.
- Kraemer, K. L., and Dutton, W., 1991. Survey Research in the Study of Management Information Systems. In Kraemer (Ed.), The Information Systems Research Challenge: Survey Research Method. Boston: Harvard Business School. 3, pp. 3-57.
- Lang, M., April 2002. The Use of Web-based International Surveys in Information System Research. European Conference on Research Methodology for Business and Management Studies (ECRM). Reading, England.
- Lee, A. S., 1991. Integrating Positivist and Interpretive Approaches to Organizational Research. Organizational Science, 2(4), pp.342.
- Monash University Information Technology, October 2009. Select a Research Methodology. Available from: http://www.infotech.monash.edu.au/resources/student/research/hdr/select-a-research-methodology.html [Accessed 18 September 2009].
- Newsted, P. R., Huff, S. L. and Munro, M. C., 1998. Survey Instruments in IS. MIS Quarterly, 22(4), pp. 553.
- Pinsonneault, A. and Kraemer, K. L., 1993. Survey Research Methodology in Management Information Systems: An Assessment. Journal of Management Information Systems, 10(2), pp. 75-105.
- Scheuren, F., 2004. Introduction. The American Statistician, 58(4), pp. 290.
- Seddon, P. B., Staples, S., Patnayakuni, R. and Bowtell, M., 1999. Dimensions of Information Systems Success. Communications of the Association for Information Systems, 2(3), pp.5.
- Sedera, W., Rosemann, M., and Doebeli, G., June 2003. A Process Modelling Success model: Insights from a Case Study. 11th European Conference on Information Systems (ECIS), Naples, Italy.