Oppimisanalytiikka – Mahdollisuudet ja edessä olevat haasteet Euroopan koulumaailmalle Aulanko, Finland April 6 2017 Dr. Riina Vuorikari DG JRC – Directorate.

Esitys aiheesta: "Oppimisanalytiikka – Mahdollisuudet ja edessä olevat haasteet Euroopan koulumaailmalle Aulanko, Finland April 6 2017 Dr. Riina Vuorikari DG JRC – Directorate."— Esityksen transkriptio:

1 Oppimisanalytiikka – Mahdollisuudet ja edessä olevat haasteet Euroopan koulumaailmalle Aulanko, Finland April Dr. Riina Vuorikari DG JRC – Directorate Innovation and Growth Unit B4 Human Capital and Employment

2 The Joint Research Centre (JRC)
Yhteinen tutkimuskeskus on komission oma tutkimusyksikkö Sen riippumattomat, näyttöön perustuvat tieteelliset lausunnot tukevat EU:n toimintapolitiikan suunnittelua JRC-IPTS: One of the key knowledge providers for DG EAC Directorate Growth & Innovation Seville

3 ulkosuomalainen Riina Vuorikari Tutkijana Sevillassa vuodesta 2013
European Schoolnet as Senior Research Analyst and Project Manager Tausta: Savonlinnan OKL (MEd) Studying abroad (exchange and postgraduate studies) e.g. hypermedia (DEA) web, use of ICT in education Doctoral 2009 from the Dutch School Information and Knowledge System ulkosuomalainen Ulkomailla Suomesta kysellään aina kaikenlaista, esim. PISA tai revontulista. Hirvimerkki herättää myös hämmennystä ja naurua

4 Current JRC research on Digital Age Learning and 21st Century Skills
DigComp (DG EMPL) DigCompOrg (DG EAC) DigCompConsumers (DG JUST) DigCompOrg4Schools (DG EAC) DigCompEdu (DG EAC) Individuals Organisations OpenEdu (HE) (DG EAC) EntreComp (DG EMPL) MOOCKnowledge (DG EAC) DigPolEdu (DG EAC) Anticipatory studies Policy & society OPTEV (DG JRC) CompuThink (DG JRC) CPDmodels (DG EAC) Learning Analytics (DG JRC) ICTinPISA (DG EAC) MOOCs4 inclusion (DG EAC) Blockchain (DG JRC) Openedu Policies(HE) (DG EAC) Current JRC research on Digital Age Learning and 21st Century Skills

5 Outline: . Osa 1: Oppimisanalytiikka, JRC:n uusin selvitys Osa 2: Mahdollisuudet Osa 3: Edessä olevat haasteet

6 Oppimisanalytiikalla
Tarkoitetaan oppijasta kertyvien tietojen keräämistä, mittaamista, analysointia ja raportointia siten, että tarkoituksena on ymmärtää ja optimoida oppimista ja oppimisympäristöjä. (source) Learning analytics have their roots in many fields of educational and technical research, including assessment, personal learning and social learning, but also in business intelligence and data mining. The field draws on theory and methodologies from disciplines as statistics, artificial intelligence and computer science (Dawson et al., 2014). Source from web-analyics in the early 2000 Even if the field in new, there are long root in existing research such as Adaptive learning; Personalised learning and Intelligent tutoring systems; Recommender systems to support learning, etc Techniques and methods are also borrowed from statistics, artificial intelligence, computer science It is like a catch-all term, an umbrella under which many existing old things have been re-dressed with some new spice Adaptiivinen oppiminen on opetusmenetelmä, jossa hyödynnetään tietokoneita vuorovaikutteisina opetusvälineinä.selvennä Tietokoneet sovittavat (eli adaptoivat) oppisisällön oppijan vastausten perusteella pääteltyjen heikkouksien ja vahvuuksien mukaan. Tarkoitus on, että tietotekniikalla voidaan näin korvata ihmisopettajan tai tuutorin tarjoama vuorovaikutteinen opetus. It’s also a topic with lots of hype around it, so thinking of the mission of the JRC and the demand for evidence driven policy-making in Europe, it was clear that the policy-makers in Europe had a real need for better data and evidence of what is actually happening. People in the Ministries, educational boards at the national and local level do take many decisions…

7 Ferguson, R. , Brasher, A. , Clow, D. , Cooper, A. , Hillaire, G
Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T., Vuorikari, R., Research Evidence on the Use of Learning Analytics and Their Implications for Education Policy. (2016), Joint Research Centre Science for Policy Report.

8 The Study behind the JRC Report
Goal: Provide research evidence on the use of learning analytics and discuss their implications for education policy Study conducted between September 2015-June 2016 Design of the study: the JRC in Seville, Unit of "Human Capital and Employment” Research: The Open University, UK under the contract and supervision of the JRC

9 To access the Inventory
Google “leap inventory learning analytics” Google “learning analytics JRC science hub” To access the Report

10 Mitä raportti sisältää?
Inventaatio tämän hetkisistä käytänteistä, mitä on tarjolla ja miten sitä käytetään: Työkaluja, käytänteitä, koulutusohjelmia (60 esimerkkiä) 5 case studies Tavoite: kriittisesti miettiä oppimisanalytiikan vaikutuksia, potentiaalia ja haasteita koulutuksen alalla “The Action List for Learning Analytics”

11 The Inventory: Tools A template used for describe tools:
Not an exhaustive list! 26 examples with international focus Descriptions available online A template used for describe tools: Inventory type: Design and planning tool, Learner support tool; Analyics for assessment; General analytics tool; Recommendations; "smart system"; learning enviroment tool; Role of analytics – the different uses of analytics: summary and description; visualisation; statistical inference; modelling; alerting; prediction; adaptation Data sources – where the data originate Uses own data; Other (vle; mis; social media; statistical services) Keywords, Context info, Maturity and Evidence, Further info… …

12 Outline: . Osa 1: Oppimisanalytiikka, JRC:n uusin selvitys Osa 2: Mahdollisuudet Osa 3: Edessä olevat haasteet

13 Esimerrki 1: Inventory of Tools
Specific models of domain knowledge (in math) and on the learner responses (cognitive models ) 1. 2. Stand alone application that generates its own data. 3. 4. 5. Adaptiivinen oppiminen on opetusmenetelmä, jossa hyödynnetään tietokoneita vuorovaikutteisina opetusvälineinä.selvennä Tietokoneet sovittavat (eli adaptoivat) oppisisällön oppijan vastausten perusteella pääteltyjen heikkouksien ja vahvuuksien mukaan. Focus only on one area: mathematics and provides personalised learning activities and feedback Only certain courses which are based on specific models of domain knowledge in math and on the learner responses (cognitive models ) Information on student progress and mastery of each achievable skill Teachers get several reports on individual’s engagement and learning of skills, but also on underperformance. Class assessment What does this data tell us about learning? Is it actionable?

14 Esimerkki 2: Inventory of Tools
Data sources include VLE; social media, “card swipes” (e.g. using student card to go to library), libraries, housing 1. 2. 3. 4. 5. 6. Works directly with each institution and makes use of available student data. Is individually tailored to each institution to fit their analytics needs Data sources include VLE; social media, “card swipes” (e.g. using student card to go to library?), libraries, housing - Aggregates student data for analysis and visualisation Historic and predictive data for institutional leaders and student service providers Visualises student performance and success across modules and predicts programme completition Integrates different tools

15 Example 3: Inventory of Practices
Keskeyttämisuhan alla olevien oppilaiden identifiointi: Georgia State University At the university, predictive analytics have been used to tackle the achievement gap for low income and first-generation students. GSU graduation rate rose from 32% in 2003 to 54% in In the process, the university claims to have removed the achievement gap between students from minority backgrounds or lower socioeconomic status and their peers. Tasa-arvo, Oikeuden-mukaisuus

16 Esimerkki 4: Inventory of Tools
2. 3. 4. 1. A learning environment tool that has a module (a feature) to make analytics available Provides analysis and reporting at individual and group level. Tools to support evaluation and improvement of pedagogical practices Data is gathered from different sources: works with several school book publishers whose modules can be used for analytics, but also student surveys and statistical data & data from national texts Also uses adaptive tools, see Knewton in tools inventory for more information

17 The Inventory: Tools (1)
“stand-alone” tools; custom-made solutions; add on to an existing VLE Different target beneficiaries of analytics: Learners, teachers, tutors, advisors, counsellors, school heads/managers, policy-makers,..

18 The Inventory: Tools (2)
Different data sources: Student digital traces from the platform or outside of it, e.g. interaction data, social media, libraries Data from offline sources, e.g. evaluations by the learner, demographic data, nation-wide test data/evaluations Actions on data: scaffold, support, recommend, predict,.. Action based on based: past behaviour, similarity in grades, domain knowledge, right answers, statistics, …

19 Esimerkki 5: Inventory of Tools
1. . A programme that provides content Students can complete lessons, the programme analyses performance and detects gaps Teachers have dedicated tools available Runs easily in a brower HOWEVER 5. Little information about privacy 6. No information about the impact of the tools, whether the use actually guarantees any learning outcomes or effectiveness, etc. 2.

20 Mitä tuloksia ja perustuvatko ne tieteelliseen näyttöön (6)?
Evidence of Impact: The research evidence documented in this study shows that there is little formal validation of tools e.g. whether the tools fulfil their intended purpose such as having a positive impact on learning; encouraging more efficient learning; or more effective learning,.. näyttöön perustuvat tieteelliset lausunnot

21 Do we see real improvements in learning outcomes for learners
Do we see real improvements in learning outcomes for learners? 37 examples Teaching 32 examples

22 Eettiset ja tietoturvaan liittyvät kysymykset (7)
Impact: The research evidence documented in this study shows that currently, most impact of learning analytics in education and training seems takes place around issues, little impact on changing practices yet: Keskustelun teemana ovat eettiset ja tietotrvaan liittyvät kysymykset useimmin kuin esim. oppimisen prosessejen tukeminen ja uusien kehittäminen

23 Examples of Australia Example 6: Inventory
Interesting policy documents include a report by the Australian Government Office for Learning and Teaching on “improving the quality and productivity of the higher education sector” Practices: Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement Tools: Loop, open source analytics tool to connect with Moodle or Blackboard (funded by the same gov. body) Case study: “Learning analytics in the context of a data-intensive strategy” - University of Technology, Sydney

24 Outline: . Osa 1: Oppimisanalytiikka, JRC:n uusin selvitys Osa 2: mahdollisuudet Osa 3: Edessä olevat haasteet

25 Haasteet Euroopan koulumaailmalle
Haaste 1: Luodaan eurooppalainen visio oppimisanalyytikasta, joka pohjautuu meidän arvomaailmaan Luodaan toisenlaista tulevaisuutta, jossa pyritään luovuutta, innovaatiota, sosiaalisia taitoja ja ongelmanratkaisua tähtääviin oppimistuloksiin Entä aktiivinen kansalaisuus ja työllistyvyys? Syrjimättömyys ja kansalaistaitojen edistäminen? Challenge !: What does it mean for educational policy? work is needed to make links between learning analytics, the beliefs and values that underpin the area in Europe – and European priority areas for education and training 2020 See also Vuorikari, R. (2017): Can Learning Analytics help the EU achieve its strategic objectives for education and training by 2020? Learning Analytics and Policy workshop in LAK Challenge 2: - much of the current work on learning analytics concentrates on the supply side – the development of tools, data, models and prototypes. There is considerably less work on the demand side – i.e. on how analytics connect with education and the changes that school administrators, teachers and studentswant these tools to make in order to support their everyday learning, teaching and assessment work. More attention needs to be paid to the demand side - like, for example, the work carried out by Kennisnet in the Netherlands. This sought to help schools articulate what they want from ICT vendors, mediating requirements and exploring possible solutions, thus ensuring that learning analytics products have useful features for their end users.

26 Haasteet Euroopan koulumaailmalle
Haaste 2: Kehittää oppimisanalytiikan työkaluja, jotka oikeasti auttavat opettajia ja oppijoita Nyt on tarjontaa, mutta onko kysyntää? Tuleeko käyttäjille se wow-efekti? Challenge !: What does it mean for educational policy? work is needed to make links between learning analytics, the beliefs and values that underpin the area in Europe – and European priority areas for education and training 2020 See also Vuorikari, R. (2017): Can Learning Analytics help the EU achieve its strategic objectives for education and training by 2020? Learning Analytics and Policy workshop in LAK Challenge 2: - much of the current work on learning analytics concentrates on the supply side – the development of tools, data, models and prototypes. There is considerably less work on the demand side – i.e. on how analytics connect with education and the changes that school administrators, teachers and studentswant these tools to make in order to support their everyday learning, teaching and assessment work. More attention needs to be paid to the demand side - like, for example, the work carried out by Kennisnet in the Netherlands. This sought to help schools articulate what they want from ICT vendors, mediating requirements and exploring possible solutions, thus ensuring that learning analytics products have useful features for their end users.

27 Do Learning Analytics remain the lower hanging fruit of digital technology with dashboards and half-hearted visions of learning ? What about the visions for empowering learning?

28 Thank you! Check the research of our team at the JRC Science Hub: New skills agenda: news/ education-skills- factsheet_en