Evaluation on the Effectiveness and Usability of the Developed Collaborative Interaction Management System

A Collaborative Interaction Management System (CIMS) is a system that potentially provide some assistance to the teachers in managing the student’s collaborative interaction in forum discussions. The projected main function of the CIMS is to provide tools that automatically gauge the quality of interaction as reflected in student’s individual contributions so as to make it easier for the instructor to make judgments on the effectiveness of the collaborative process. This study aims to assess how effective and usable is the developed CIMS based on the evaluation of faculty and students. The descriptive research design is utilized in this study. The five-point Likert scale is used and the gathered data are statistically interpreted through the utilization of a weighted mean. The sample population of this study is the one hundred twenty participants coming from the different universities in the area where the survey was conducted. The researcher adopted a survey instrument made by some software specialists. The bases of the instrument are some of the characteristics listed on the software quality model of ISO 9126. These are the functionality, usability and the reliability. Each of these software characteristics is described by some indicators to obtain the responses of the participants. The overall evaluation of the participants displayed the weighted mean of 4.0, thus it can be concluded that the functionality, usability and reliability of the software is very good.


Introduction
In the traditional lecture-based method of teaching and direct instruction, learning is primarily perceived as involving the simple transfer of fixed knowledge from the teacher to the students.In this paradigm, the teacher's main responsibility is to cover as much course material and deliver knowledge to their students.In such a setup, the minds of students are viewed as empty vessels in which knowledge can easily be poured into; as such, learning is simplistically presumed to have occurred when students have individually integrated and organized the knowledge they received in their heads.While such methodology and perspective has been tried-and-tested, it nevertheless has numerous disadvantages.Over the years, perspectives on learning have evolved considerably.
Educators nowadays, have increasingly realized the inadequacy of the lecture-based approach and have started to view learning as a process that is based more on student's personal experience.In this new paradigm, learning is defined as being context-sensitive and focused more on selfdiscovery and exploration.Knowledge therefore is viewed as being created within a group process where interaction between individuals and the sharing of experiences in learning activities are the primary motivators.The teacher's responsibility in this scenario is shifted towards getting students to interact with each other in a productive and supportive manner.Learning environments that emphasize the social nature of learning are considered as valuable tools in this new paradigm (Baytak, Tarman & Ayas, 2011;Tarman & Baytak, 2011;Tarman, 2012).Tools like forum discussion that pays particular attention to promoting and encouraging communicative cooperation and collaboration is a good example of this kind of environment.In fact, in the e-learning panacea, both in the academe and the industry, discussion forums are gaining tremendous popularity as a mechanism for increasing learner interaction.The utilization of discussion forum in academic setting is often problematic.The main reason is the enormous amount of time and effort that it imposes to implementing teacher-mediators in monitoring and guiding student interaction.
Another factor is the high level of difficulty faced by the implementing teachers in evaluating the contributions of students in such online interactions.In the field of e-learning, similar problems are being addressed through the development of specialized environments such as the Learning Management System (LMS) and Content Management System (CMS).
After thorough study the researcher was able designed a framework that was used to develop a management system that potentially provide the teachers some assistance in managing student's collaborative interaction in forum discussions.This is the system that is referred to as a Collaborative Interaction Management System (CIMS).The task of the researcher does not end after the development of CIMS.There is a need to evaluate the effectiveness and usability of the developed CIMS; thus, this study was conducted.

Objectives and Rationale of the Study
There are huge number of researches that explore about some areas of information and communication technology; however, it is believed that there are limited studies that explained its effectiveness and usefulness which would guide the developers on how to enhance their newly created system.The developed CIMS could reduce the time and tasks of the teachers when making decisions on the assessment's performance of every student.
Through the results of this study the teachers would be equipped with more knowledge and they will be able to extend more support to the students in managing their collaborative interactions.
Moreover, it would also address the end users' perspective (teachers and students) as regards to the issues of measurement on the quality characteristics of the developed software.
In addition, the outcomes of this research would serve as a tool, to be utilized by the end users namely teachers and students as well as designers of instructional technology, which could expedite progressions in their communication and instructions activities.

Review of Related Literature
Nowadays, in the area of education technology is widely used.This includes the use of Powerpoint presentation, study reading materials, online books, and the like.Technology has become very important for any business whether it is a small medium enterprises or huge corporations.The meetings with employees around the world are now scheduled with the use of the Internet.
Technology changes most of people's lives (Tarman, 2009;Yucel et.al, 2010).The connection between families even in the different parts of the world are maintain through the use of cellphones and electronic mail.In other words, technology is used in various activities and in the different areas; thus, it is expected that improvements will continue over time and it will aid in making the works of average people a lot easier ("What role does technology", 2017).Yet, in the paper of Bocar and Biong ( 2015 ) they recommended that students must be more careful in the utilization of on information and communication technology either it be within or outside the school so that the time and money could be spent wisely so that it will bear good results and goals for success will be achieved.
ISO 9126 is an international standard for the evaluation of software.The ISO 9126 or other quality model transports clearness on the meaning of purpose and operating capability; however, the requirements, compliance and operating environment of software continuously changing; thus, the search to find valuable characteristics that facilitate measurement and control of the software production process continue (McCall, 1977).The important purpose of a product or service connotes functionality.The more functions the products or services have the more complicated it becomes.In the case of software there is specified list of functions.Usability refers to the easiness of use for a given function.It may also refer to the ability to learn on how to use a system.It is considered as a major sub-characteristic of usability.Specifically, reliability is define as the capability of the system to uphold the provision of its service under defined conditions for definite periods of time.It is a software characteristic which include fault tolerance.This means that the system has the ability to recover and continue its functioning after it experienced failure (McCall, 1977).According to the study of Bala and Chana (2012) "fault tolerance is a major concern to guarantee availability and reliability of critical services as well as application execution".It was explained in their study that the impact of failure can be minimized through pre-emptive measures.
Technical people must forecast failures and take proper actions before failures actually happen.This is the meaning of fault tolerance techniques.The researchers above mentioned proposed cloud virtualized system architecture.In the proposed system autonomic fault tolerance was implemented and it was found out that the proposed system can deal with various software faults for server applications in a cloud virtualized environment.
According to Britain and Liber, (2003) since early 1980's the academe have used email to communicate and the influence of technology continues as the growth and explosion of internet came very popular that leads academician to undertake in making their lecture notes and other supplemental reading materials on the web for the students to look through.It is common nowadays to some universities to have web-interfaces.Some institutions used conferencing software to create online group discussions.These are the innovations brought about by the modern technology.It is not surprising when IT researchers would be able to develop software to further make the technology more useful in the educational setting.The developed software could be used as a teaching and learning tools.
There are numbers of software quality characteristics; however, Chua and Dyson (2004) explained that the three software quality characteristics amongst others namely: functionality, reliability and usability are assessable, thus, this encourage the researcher to evaluate the effectiveness and usability of the developed CIMS.An instrument with specific indicators designed from the quality characteristics of the ISO 9126 software quality model by experts (Davis, 1989;Tullis and Stetson, 2004) was adopted to assess the developed CIMS.

The Problem
The implementation of the developed CIMS was not assumed as perfect; thus, assessment needs to be done.This study specifically focus on what is the level of the effectiveness and usability of the developed CIMS based on the evaluation of the (a) teachers, and (b) students?

Scope and Limitations of the Study
This study focuses on the evaluation of the software's quality that is assessed by the Information Technology (IT) teachers and students from various Universities in Metro Manila using evaluation criteria provided in the ISO 9126 software quality assessment tool.The general objective of this study is to determine the observation of the user-participants on the developed CIMS.

Method
This section discussed the process on how this study was conducted.The descriptive research design is utilized in this study.The projected main function of the CIMS is to provide tools that automatically gauge the quality of interaction as reflected in student's individual contributions, thus, it makes it easier for the instructor to make judgments on the effectiveness of the collaborative process.In order to fully investigate the CIMS as a system, which is appropriate for use by faculty and students in the subject area, the characteristics of the CIMS have to be evaluated.The quality characteristics items mentioned in ISO 9126 software quality model was the bases of the adopted evaluation instrument on the effectiveness and usability of the developed CIMS.These are the functionality, usability and reliability.Each of these software characteristics is described by some indicators, from the work of Davis, (1989); Tullis and Stetson, (2006) to obtain the responses of the participants.The sample population is randomly chosen which composed with the thirty teachers and thirty students from the selected universities in the National Capital Region area.One hundred twenty are the total number of participants of this study.

Statistical Tools
The five-point Likert scale was used and the gathered data are statistically interpreted through the utilisation of the weighted mean.For purposes of interpretation, analysis, and discussion the researcher constructed the scale of measurement as follows:

Data Gathering Procedure
Prior to the administration of the instrument the researcher ask permission from the heads of office of the four universities.For confidentiality and ethical reasons they are named as, A University, B University, C University and D University.After the approval of the request the questionnaire was distributed to the participants.The answered questionnaires were collected and the gathered data are tallied, interpreted, and analised.

Results and Discussion
The table below reflects the results of the study.The participants exposed the level of the effectiveness and usability of the developed system.

Functionality
Based on the gathered data the participants revealed the functionality of the developed CIMS.It can be observed in the results as manifested by the participants that out of the six (6) description or indicators to measure the functionality of the developed CIMS four (4) of them were rated very good.Nevertheless, the two (2) of them were rated excellent.The results show that to some extent the functionality of the developed CIMS is effective at most of the time.Moreover, its usefulness would improve job performance and enhance effectiveness always.

Usability
Moreover, with respect to the usability of the developed CIMS, the table above showed split results.As evaluated by the participants, they rated excellent on the three of the six (6) descriptions or indicators.They revealed that they would like to use this system frequently since it was easy to use, and they felt very confident using the system always.However, in the other three (3) descriptions or indicators the participants rated the effectiveness and usability of the developed CIMS at most of the time.This signifies that the developed CIMS is effective and usable but nonetheless the assistance of the technical person is necessary at some extent as it shows the lowest item average (µ=3.42)among other descriptions in this characteristic.This situation is reasonable since every individual has his or her own expertise.As per the definition of American Heritage Dictionary of the English Language (2016) a technical person in the scientific field means "one who is having or demonstrating special skill or practical knowledge relating to information technology".In the present study some of the participants are students which is it presumed that their skills and practical knowledge are yet inadequate.

Reliability
In addition, with regard to the probability that the developed system does not experience many failures in the given time interval the participants demonstrated that they are satisfied on the effectiveness and usability of the developed CIMS as shown in the results on the table above.
However, the participants established that errors preventive messages must be increased in the system to decrease the failures on the part of the users.The overall results displayed the rating of very good which means that the effectiveness and usability of the CIMS is at most of the time.
The results of this present study strengthen the words of Bala and Chana (2012) when they stated in their study that to minimize failure, forecast on the future failures needs to be done and take the appropriate actions before failures actually happen.This signifies the application of fault tolerance techniques.points need further study and enhancement in order to make it more effective and usable.
Specifically, to wit: in the usability characteristic of the developed CIMS the description which says that, the participants need the support of a technical person to be able to use this system must be given greater consideration since this signifies that though they rated the overall usability at very good level they still found some difficulty to some extent.Moreover, it can be noted that the reliability characteristic gathered the lowest factor average among the other two characteristics.
Thus, more improvements or enhancement in this area must be done.