Using computer-based text analysis to integrate qualitative and quantitative methods in research on collaborative learning
I have been reading:
Wegerif, R. and Mercer, N. (1997) ‘Using computer-based text analysis to integrate qualitative and quantitative methods in research on collaborative learning’, Language and Education, vol. 11, no. 4, pp. 271–86.
The main research question was to find out if computer based analysis of transcribed talk which incorporates both quantitative and qualitative research methods could produce a more effective analysis of children’s speech.
The education setting is based in a primary school and it was a series of debate lessons concentrating on explorative talk. A classroom setting which was rather informal, so children could speak freely amongst groups of four.
Computer based text analysis of transcribed talk. They also look at collaborative learning; they discuss the implications of quantitative and qualitative research identifying strengths and weaknesses in both paradigms. They looked at systematic observation of coded categories which can create statistical evidence. Qualitative approaches were: detailed classroom observations, interpretation of transcribed talk, ethnography and discussions between teachers and children. Research Participation Discourse analysis as qualitative discourse analysis using computer-based tools as concordancing software ‘!Kwictex’.
There was one ‘target’ class and one ‘controlled’ class chosen at the beginning and at the end of the intervention programme. The use of psychological tests of reasoning such as ‘Raven’s Standard Progressive Matrices’ which involved matching shapes and patterns. Children were aged between nine and ten years old and work together in groups of three. The target class had nine groups the control class had five groups.
The research found out that using computer-based transcript analysis combining qualitative and quantitative methods during discourse analysis were more effective. The computer-based methods proved to be more practical as certain words of children’s speech could be analysed and then easily compared with the full text of the speech, so ensuring research was more valid and authentic.
The limitations could have involved the perceived choice of children’s phrases to back up or enhance their predictions. The reader could not see the whole transcript in order to fully understand and develop a true sense of the children’s groups and discourse. They already had a preconceived idea of what words to look for regarding discourse analysis and this could have stopped the investigation from being open to other parts of the dialogue which could have shown different findings.
Ethical issues I believe were not addressed are: no consideration any learning difficulties, didn’t take into account the hierarchical structure of the each children’s group. No evidence to prove that Wegerif and Mercer took into consideration that their presence during the research activities could have influenced the findings; this could have affected the children’s discussion. The timing and placement of the group lessons were not stated which could also have affected the way the children interacted with each other.
The implications for future research could entail other activities which explore children’s dialogue concerning other aspects of their learning experience.
In my summary below S = Strength, W = Weakness
S & W from looking at coding methods to analyse talk – Quantitative
-S being able to handle large corpora of data
-S it can produce a set of results that can be directly compared with results from the same test based on varying groups
-S They offer a more objective basis for validity claims
-S shows a statistical relationship between two events in time
-W it does not relate the two events from a statistical relationship
-W ambiguity in meaning of words – makes coding difficult
-W reliability is questioned – results are presented in the paper but it is not explained the basis of the results – we cannot know how the coder would have analysed the results – opinions may differ.
S & W of qualitative analysis methods
-S sensitive to context and the “temporal development of meanings”.
-S effective for generating theories
-W not effective at testing theories rigorously
-W can only test small samples
-W educational research relies upon facts and results – this cannot necessarily be produced
I think the examples in the paper needed to be more extensive. I’m not sure if they’ve done anything further on this
Automated methods still require reseachers to make choices. Analysis of this kind is a time consuming and complex process, requiring quite sophisticated multi method analysis. Although this was 1997 and things have moved on.
For a recent review see:
De Wever, B., Schellens, T., Valcke, M., & Van Keer, H. (2006). Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review. Computers & Education, 46(1), 6-28.
For multimethod analysis see:
De Laat, M., & Lally, V. (2003). Complexity, theory and praxis: Researching collaborative learning and tutoring processes in a networked learning community. Instructional Science, 31(1-2), 7-39.