The Challenge of Evaluation
Evaluation of best practices in any form of education, as experienced practitioners and theoreticians know very well, is a very challenging endeavour. However, even more challenging is the evaluation of collected educational practices in diverse cultural contexts and among – from the point of pedagogical methods used –sometimes totally different projects. This is precisely where the evaluation of best practices in informal media literacy education initiatives fits. There is also obvious issue of small foreign languages – many national projects have only limited, if any, description of a project in the English language.
The EURISPES and SKAMBA teams working on this work package expected these challenges. Therefore, we have developed specific set of evaluation criteria focusing at key competencies in education and key projects´ parametres. We have discussed these criteria further with key experts in the area of education, for example with Dr Monica Bulger from Oxford Internet Institute. In spite of these efforts, the sheer size of sample – over hundred cases collected – needed initial downsizing to a more manageable sample of some thirty cases. An experimental phase of our workpackage revealed that each case study must be evaluated by at least two independent evaluators and in controversial cases (fundamental disagreement over quality/contribution to education) by three or, indeed, four evaluators. In short, methodological aspects are of utmost importance in doing scientific research.
The key general conditions for scientific work must be met too, in order to devote as much time and intellectual capacities to the research proper. Namely, these conditions include clear division of tasks among partners, properly allocated budget, detailed working schedule and smooth research communication. Perhaps it is redundant also to mention such obvious prerequisites as professional competencies and interest in doing research. These conspicuous prerequisites for successful research are not always met within organisations, and sometimes not even among organisations, especially in international cooperation. There are many challenges and mysteries in international cooperation in social sciences research.
written by: Dr Andrej Skolkay, School of Communication and Media