Assessing psychometric properties of scales

Chapter 5. Data Analysis

We used Partial Least Square (PLS) to assess the psychometric properties of the scales and to test the research model and hypotheses. PLS is a latent structural equation modeling technique that utilizes a component-based approach to estimation (Jöreskog and Sörbom 1993). Because of this, it places minimal demands on sample size and residual distributions (e.g., Chin 1998a, 1998b; Fornell and Bookstein 1982; Lohmoller 1989).

According to Chin et al (2003), PLS is perfectly suited for testing complicated relationships by avoiding inadmissible solutions and factor indeterminacy, and its capability in exploring complex relationships has been proven in many other studies(e.g., Fornell and Bookstein, 1982; Fornell, Lorangem and Roos, 1990). PLS has been gaining interest and used in MIS, IS and KM (Compeau, 1995; Wasko, 2005).

In general, a PLS analysis consists of the stages:

  1. Calculate a PLS model using a high number of factors (more than is likely to be required);
  2. Determine the number of factors to include in a fitted model by either:
  3. Fit the model with the determined number of factors by calculating parameter estimates of the linear regression;
  4. Given a set of predictors and responses used to fit a PLS model, and a suitable number of factors to use to calculate parameter estimates, estimate response values to new predictor data.

Based on previous lectures, we conclude that there are two main stages of PLS analysis: the assessment of the reliability and validity of measurement; and the analysis of the structure model.

Both of the two stages are introduced below:

Assessment of the measurement model

To validate our measurement model, we assessed the reliability and validity, including internal consistency, convergent validity and discriminant validity. A good level of reliability and validity are main characteristics to a good survey.

In PLS, Reliability was assessed using internal consistency scores. And the internal consistency of a given block of indicators can be calculated using the composite reliability (CR) (Wasko, 2005). The composite reliability should greater than 0.7 which is considered adequate and which was interpreted by Cronbach's alpha coefficient. Cronbach's alpha, the most common test methodology in Likert Scale, is between zero and one. When the value is close to one, it means the high reliability of this test. In the table 7, it is demonstrated that almost al the Cronbach's alpha values fall between 0.7413 ~ 0.9292, which is in line with the threshold (alpha should be > 0.7) proposed by Nunnally (1978). In general, however, results are indicative of adequate reliability.

Besides the reliability, we also assessed the convergent validity and discriminant validity. In this study, we measured the factor loading per their representative variables in first-order construct, and continue with factor analysis and extraction. Because it's suggested to trim those indicators with loading less than an absolute value of 0.5 (Ron. 2006) and then form the variable with these qualified factors. Table 8 shows some unqualified indicators (with a loading of less than 0.5) andthen eliminated. After all the disqualified factors are trimmed, we repeat again the factor analysis and the result are illustrated in Table 9. Each of these factors is incorporated as reflective indicators into its principal construct of relation.

As for the convergent validity, it should be assured especially when the multiple indicators are used to validate one single construct, can be examined by average variance extracted (AVE), which reflects the variance captured by indicators. The AVE value should be greater than the generally recognized 0.5 cut-off (Fornell and Larcker, 1981). In the test of convergent validity, all the AVE values are meeting the recommended threshold 0.5.

Discriminant validity is the degree to which a construct is different from other construct, and was assessed using the latent variable correlations matrix, where the square roots of the values of the AVE calculated for each of the constructs along the diagonal is reported. Therefore, we assessed the discriminant validity and found that all the square roots of AVE are greater than the off-diagonal elements in the corresponding rows and columns, we also concluded that the discriminant validity is satisfied (See Table 10).

Analysis of the structural model

For the reason that we've obtained convincing results from the testing of reliability and validity in the previous sections, we shall move on with the testing for our proposed hypotheses. We assessed the structural model with the bootstrapping technique in PLS. In the analysis of regression, PLS Bootstrap with size re-sampling to five-hundred is applied.

The explanatory power of the structural model is evaluated by R2 value. Besides, in order to examine if each hypothesis is established, we assessed t-statistic for the standardized path coefficient. All the path coefficients and explained variances for the model using a bootstrapping procedure are shown in Figure 5.

The whole message

We discuss the results in the following sequence: Technology-Task compatibility, Technology-People compatibility, Technology-Organization compatibility part, and then usage to performance part and compatibility to performance part.

First of all, when we design the model, we proposed the link from technology-task compatibility to KMS usage. And we found that the effect of technology-task compatibility to KMS usage is positive and quite strong (β=0.5505 , p<0.001).Additionally, the relationship between technology-people compatibility and KMS usage was statistically significant (β=0.444,p<0.001).Thirdly, the path from technology-organization compatibility to KMS usage(β=0.3453 , p<0.05) was also positive and significant. But it has the least impact on KMS usage.

Finally, the link between KMS usage and performance, while in the direction predicted by the model, was significant as well(β=0.149 , p<0.001). Furthermore, the path from technology-people compatibility to individual performance is positive and significant (β=0.2959 , p<0.01). But the links between technology-task compatibility and individual performance, and the links between technology-organization compatibility and individual performance, while in the direction predicted by the model, were not significant.

As for the part of variance explanation, the three compatibility beliefs together explain 41 percent of the variance in self-reported usage, while 19 percent of the variance in individual performance is accounted for by the self-reported usage and the three compatibility beliefs. In sum, we conclude the hypotheses and the results of structural model assessment in table 11.

Chapter 6 Discussion and Implication


This study proposes and tests a model for the factors of compatibility impacts of knowledge management systems on technology use and performance. In general, the empirical results are encouraging and providing support for the main objective of the study. The major objective was related to the development of a fresh perspective on the notion of compatibility both in terms of dimensionality as well as the structure of the construct. Based on the theoretical definition of compatibility, we proposed three distinct aspects of compatibility: Technology-Task compatibility, Technology-People compatibility, Technology-Organization compatibility.

Following of this study was to find empirical support for the theoretical consequents of these compatibilities. With no exception, the posited theoretical relationships between the three kinds of compatibility and usage (both scope and intensity) were all supported. The Technology-Task compatibility has the most impact on usage and its path coefficient is higher than the Technology-People compatibility. Out potential explanation for this circumstance may be that people is adaptable and always take advantage of learning to improve personal capability. Even if the KMS incompatible with user's prior experience, compatible with task needs, they may willingly use it to handle the hard task. In addition, the Technology-Organization compatibility has the least impact on usage. The reason for this result possibly that the organizational compatibility is a contextual factor for personal KMS use, so it indicates less influence than else.

In our theoretical development, we hypothesize that Technology-Task compatibility would positively relate to usage (H1). Our results support this hypothesized relationship and indicated that the fit between the task and the available knowledge management technology is associated with the usage. The result is different from the initial evidence of the causal link between TTF and utilization, which was ambiguous (Goodhue and Thompson, 1995).Our explanations of this outcome are from two view points:

First, from the view of task characteristic, most users of KMS are Knowledge workers. A creative knowledge worker can contribute to face the problems that need new kinds of resolution, the situations that demand various knowledge resources. They also apply the KMS to achieve a work-related task, to deal with the semi-structure issue and to cooperate or communicate with fellow workers. As the degree to which a job requires different activities and skills in carrying out the work, user has the demand to acquire the explicit knowledge for his/her job, and the KMS should have the capabilities to combine new sources of knowledge and just in time learning (Alavi & Leidner 2002), provide enough to meet user's business needs.

Second, unlike the original TTF model (Goodhue and Thompson, 1995), which was focusing on the general Information System (IS), herein our object is Knowledge Management System (KMS). Through the integration of different systems that were individually designed and developed into an infrastructure (Cantin, 2004), KMS aim to enhance the capability of worker to manage and generate knowledge faster and more accurate in proposing an overall solution of task, and then they will count on KMS frequently.

Following we will discuss the impact of technology-person compatibility on the usage. As postulated by hypotheses 2, the direct relationship between this compatibility and usage is significant in the current. This means that higher compatibility of KMS with one's experience or belief, in turn, results in more usage. And KMS usage would be expected to be less as technology-person compatibility decreases.

By far the majority of research on compatibility has looked at its impact on adoption decision (Rogers 1983, Tornatzky and Klein 1982) and attitudes (Agarwal and Karahanna 1998). Previous researches (Karahanna et al. 2006, Venkatesh et al. 2003) noted that as the compatibility of IT increase, attitude towards information system usage should more positive. This model directs our attention to another aspect of system usage.

Moreover, factors affecting information system usage are best investigated at the individual user's level (DeLone and McLean 1992, 2002). For example, greater consistency with prior experiences should facilitate the learning process and improve KMS use. Thus, it may be that individuals who have used similar systems before or who have used systems that embodied similar types of functionality won't need to spend extra time on adapting and additional learning than individuals who encounter such a system for the first time.

Finally, we also examined that relationship between technology-organization compatibility and usage (H3). Our result show that the more compatibility of the KMS was with organization's culture and work practices the more usage it was revealed. Hence, organizational compatibility is as important as others compatibilities, to affect the usage of KMS. The point is consistent with Schultz and Slevin's findings of the importance of organizational and technical validity for ensuring success in implementation. This outcome also supports Rogers's argument that innovations are more likely to be adopted when they are compatible with an organization's values, previously introduced ideas, and needs (Rogers, 1995).

Training can enhance use of KMS through building people's competence and confidence to try and to adopt it. When organization provides more training in system use, therefore, it will help end-users overcome the fear of complexity of the KMS and improve the computer self-efficacy of users is likely to be effective in gaining user acceptance (Amoako-Gyampah and Salam 2004).

Moreover, extrinsic rewards are important motivators for knowledge sharing in organizations (Huber 1991; Pan and Scarbrough 1998). Organizations provide incentives or tie rewards (promotion and bonuses) to contributions as well as reuse (Davenport et al. 1998), then will increase the frequency of KMS usage.

In the performance part, the result provides that the use of KMS improves the performance of employee indeed. This finding seems coincidence with the prior argument that KMS can improve individuals' performance and productivity in terms of time and speed of the knowledge sharing process (Maier 2002). Furthermore, the technology-people compatibility has deeper effect on performance than other compatibilities. But there is something worth to be noticed. The proportion of variance explained by the relationship between usage and performance is relative low. There may be other variables that need to be included to better explain performance.

Theoretical Contributions

In this paper, we argued for the need to revisit the compatibility construct. Empirical results supported a new view of compatibility which consists of three distinct constructs as well as linkage to KMS usage. In reviewing the literature, this research found that the major constructs in this study have been addressed separately by various researchers, and most of the prior related researches were concentrated only on single viewpoint of compatibility. More specifically, the expected contribution of this research is 4-fold:

First, this research is one of the first to apply the contingency theory to analyze the KMS usage. By understanding the fit between contingency variables, we can know about their effects on KMS usage more. Furthermore, we suggest future research can use the theory in other fields based on our study.

Second, we adopted the "Diamond model" which help use to disaggregates the content of compatibility into three distinct and separable constructs. There were no research integrate these constructs of compatibility into a cohesive model to explore the system usage. And from the research results, we can see all of the assumptions we proposed in every element are supported. Therefore, we can know when people use KMS, they put emphasize on different constructs of compatibility, and the future research should take a closer look at these significant constructs which may be the important factors of other technologies usage.

Third, there was few research focused on organizational compatibility to explore KMS usage. Since KMS is more complicated than traditional IS, users require more organizational support to apply it to the job. Compare with other constructs in our research, although the organizational compatibility has least influence on usage, it still worth discussing and the future research can study related issue from this aspect.

Fourth, the theoretical and empirical nature of this study will lead to further research in KM to strengthen its theoretical and may be able to reveal many additional new insight into the ?Compatibility?and the use of KMS in organizations. In others words, we apply the notion of compatibility on KMS domain, and the future research can base on our study and use the concept in different information system.

Implication for Practitioner

Knowledge is a powerful intangible resource that enables individuals and organizations to improve learning, decision-making processes, and consequently achieve a competitive advantage in the knowledge-based economy. KMS is a class of information system developed to manage (store, search/retrieve, transfer and distribute) knowledge throughout the organization (Bock & Qian, 2005).

Nowadays, in practice many managers tend to emphasize on how to encourage to exchange and utilize knowledge through KMS usage. However, if the technology doesn't compatible with employees' task demand, this may cause them resist to use and waste resource. The task-technology compatibility contributes most to KMS usage.

From our results, compatibility is really a major factor and necessary to take it into consideration for increasing the intensity and scope of KMS use. In addition to users' compatibility, organizational compatibility is also significant, and further, the sequence of the three compatibilities' impact on usage gives momentous priority to managers.

We also provide managers some means to raise the compatibility. An enterprise should accord with all aspects of the way employees typically conduct their task to provide compatible KMS, easy to find out useful knowledge on a given subject and available when task need. Since technology - person compatibility beliefs are instrumental in shaping beliefs about usefulness and ease of use (Agarwal and Karahanna 1998), further, our results reveal that it is crucial to the KMS usage, managers responsible for implementing new KM technologies need to pay careful attention to their formation.

Positive beliefs about the compatibility of KM technologies can be developed in many ways: by highlighting the similarities between workflow enabled by the KMS and the individual's preferred work styles, by underscoring how the KMS embodies prevalent values, and by emphasizing the fit between the technology and the mental models created through prior experiences.

Furthermore, when an organization wants to enhance the usage of KMS, it should provide sufficient training to help people overcome their fear of the complexity of the KMS and educate people that the KMS is there to help them do their jobs better, not to give them extra work. In other words, top management should provide active support and persuade people to understand the value of the KMS and thus, use it. Some variables like culture and organizational subjective are needed to deliberate by managers as well. Finally, in today's highly competitive modern business environment, organizations face different decisions regarding whether to use new and innovative knowledge management systems.

The results of this study indicate that KMS utilization is related to improve performance. In practice, the organization must provide compatible KMS for employees to enhance their work performance.

Limitation and Suggestions for Future Study

The findings of this study should be treated with caution due to some inherent limitations and have several implications for future research.

Please be aware that the free essay that you were just reading was not written by us. This essay, and all of the others available to view on the website, were provided to us by students in exchange for services that we offer. This relationship helps our students to get an even better deal while also contributing to the biggest free essay resource in the UK!