Learning Analytics Workflow

LAW Module 1: Conceptual Overview

Welcome to Learning Analytics Workflow

• Learning Analytics Workflow (LAW) is designed for those seeking an introductory understanding of learning analytics using basic R programming skills, particularly in the context of STEM education research.

• It consists of consists of four modules. Each module of LAW includes:

• Essential readings

• Conceptual overview slidedeck

• Code a-long slidedeck

• Case study activity that correlates with the Learning Analytics workflow

• Optional badge activity

Module 1 Objectives

By the end of this module:

• Types of Learning Environments:
  • Learners will be able to identify and describe different types of learning environments, explaining their unique features and applications in educational research.

• Characteristics of Data:
  • Learners will gain proficiency in recognizing and categorizing various data formats commonly used in educational research by the end of this section.

Essential readings:

1. Learning Analytics Goes to School, (Ch. 2, pp 16 - 28 ) By Andrew Krumm, Barbara Means, Marie Bienkowski
2. Learning Analytics Goes to School, (Wrangle Ch. 3, pp. 38 - 43) By Andrew Krumm, Barbara Means, Marie Bienkowski
3. Educational data mining and learning analytics: An updated survey, By Romero & Ventura

• Optional Readings on Stakeholders in folder By Gašević et al., 2022 and Sun et al., 2019

Education Data

• Interaction between instructors and students

• Administrative data

• Demographic data

• Student affectivity

Interaction between Instructors and Students: Examining how students and teachers engage through quizzes, exercises, and communication.

Administrative Data: Insights into school and teacher-related information, providing a structural view of the educational environment.

Demographic Data: Analysis of student demographics, such as age and gender, demographic data can be used to understand diverse learning needs.

Student affectivity: Focuses on the emotional and motivational aspects of students, crucial for personalized learning experiences.

Each category plays a vital role in shaping educational strategies and outcomes.

Types of Learning Environments

• Face to Face

• Blended

• Digital (computer aided) Learning Systems

Digital learning environments include any set of technology-based methods that can be applied to support learning and instruction.

According to Krumm et al., Digital Learning Environments have fueled data intensive research in education more than any other data.

The ones we are probably most familiar with are….

Face-to-face contact between educators and students organized through lectures, class discussion, small groups, individual seat work, and so on. Traditional education systems are classified in different levels UNESCO. These systems gather information on student attendance, marks, curriculum goals, class, schedule information, and so on. Finally, it is important to note that all these traditional systems can also use computer-based educational systems as a complementary tool to face-to-face sessions.

Digital Learning Systems or Computer use computers in education to provide direction, to instruct or to manage instructions given to the student. The global use of Internet and the application of artificial intelligence (AI) techniques have led to today’s plethora of new web-based intelligent educational systems. Gibson and Frietas (2016) suggest the function of technology as an active participant in data generation is significant, as it is involved in the intricate process of data creation. This process stems from the dynamic interplay between the learner and the digital learning environment, as well as from the collaborative data production by the learner, environment, and social context. Such collaborative situations give rise to a wide range of diverse data sources, which exhibit substantial variation. It becomes necessary to harmonize and integrate these distinct data sources together.

Blended (“blended learning,” “hybrid learning,” and “mixed-mode instruction”) combine face-to-face instruction with computer-mediated instruction. Before COVID we saw some use but after COVID we saw an increase in the use of Digital Learning Systems and Blended.

Digital Learning Environments (Computer Aided)

• Adaptive and Intelligent Hypermedia System (AIHS)

• Intelligent Tutoring System (ITS)

• Serious Games and Simulations

• Learning Management System (LMS)

• AI-driven virtual tutors and chatbots

• Massive Open Online Course (MOOC)

• Test and quiz system

• Sensor Wearable learning systems and Virtual

• Augmented reality systems

• Multimodal Computer Aided Learning

Adaptive and Intelligent Hypermedia System (AIHS) - These systems attempt to be more adaptive by building a model of the goals, preferences, and knowledge of each individual student and using this model throughout interaction with the student in order to adapt to the needs of that student. The data recorded by AIHs are similar to ITS data.

ITS - ITSs provide direct customized instruction or feedback to students by modeling student behavior and changing its mode of interaction with each student based on its individual model. Normally, it consists of a domain model, student model, and pedagogical model. ITSs record all student-tutor interaction (mouse clicks, typing, and speech).

Test and quiz system - The main goal of these systems is to measure the students’ level of knowledge with respect to one or more concepts or subjects by using a series of questions/items and other prompts for the purpose of gathering information from respondents. They store a great deal of information about students’ answers, calculated scores, and statistics.

Discussion

• Choose one category (Interaction, Administrative, Demographic, or Student Affectivity).

• Share an example of how data in this category can be collected.

• Discuss the potential impact of this data on educational decision-making.

Characteristics of Data

• Types
• Structured
• Unstructured
• Semi Structured
• Meta

# A tibble: 603 × 30
   student_id course_id     total_points_possible total_points_earned
        <dbl> <chr>                         <dbl>               <dbl>
 1      43146 FrScA-S216-02                  3280                2220
 2      44638 OcnA-S116-01                   3531                2672
 3      47448 FrScA-S216-01                  2870                1897
 4      47979 OcnA-S216-01                   4562                3090
 5      48797 PhysA-S116-01                  2207                1910
 6      51943 FrScA-S216-03                  4208                3596
 7      52326 AnPhA-S216-01                  4325                2255
 8      52446 PhysA-S116-01                  2086                1719
 9      53447 FrScA-S116-01                  4655                3149
10      53475 FrScA-S116-02                  1710                1402
# ℹ 593 more rows
# ℹ 26 more variables: percentage_earned <dbl>, subject <chr>, semester <chr>,
#   section <chr>, Gradebook_Item <chr>, Grade_Category <lgl>,
#   FinalGradeCEMS <dbl>, Points_Possible <dbl>, Points_Earned <dbl>,
#   Gender <chr>, q1 <dbl>, q2 <dbl>, q3 <dbl>, q4 <dbl>, q5 <dbl>, q6 <dbl>,
#   q7 <dbl>, q8 <dbl>, q9 <dbl>, q10 <dbl>, TimeSpent <dbl>,
#   TimeSpent_hours <dbl>, TimeSpent_std <dbl>, int <dbl>, pc <dbl>, uv <dbl>

In data-intensive research we might acquire data from LMS, SIS or another source. Data maybe to big for a typical database software tools to capture, store, manage, and analyze”

Those data are similar but different because 1. the tasks that students are engaged in and 2. how data from those tasks are collected and stored by the technology.

We identify four main types of data structures. - structured - unstructured - semi-structured and - meta-data

STRUCTURED - TAB

Structured data is Quantitative in nature Here we see the rows and columns nice and tidy. - data is tidy when each column is a variable, each row is an observation, and each cell is a single value.

We typically have tidy data in Excel files or SQL databases that we can import using the readr function or here function.

• Tabular - relational data. Examples might include:
• performance data (current and prior)
• demographic data (student’s ethnic and or group or disability)

Each field (variable) is discrete and can be accessed separately or jointly along with data from other fields (variables).

UNSTRUCTURED TAB

Unstructured data is Qualitative in nature and includes: - Non-Relational Data - sensor data, web logs, clickstreams, keystroke capture data, emails, images and videos

Text Heavy relationships between nodes IE: Audio, video, dates SQL database

**SEMI-*STRUCTURED-TAB** Semi-structured data includes Qualitative Triangular Data (uses surveys and interview answers) Semi-structured data is a form of structured data that does not conform with the formal structure of data models associated with relational databases or other forms of data tables.

Semi structured can also include log data.

It is a self-describing structure that contains tags examples: Social media, JSON and XML, Social Networkdata

META-DATA - TAB

Not in the readings but something to think about is Meta-data collection, such as the field dates, times and locations of photos, videos and other structured data about data. - contexts and purposes for why the data was collected

You can read more about the meta data standards on Canada’s heritage information network. The link is in the presenters notes. https://www.canada.ca/en/heritage-information-network/services/collections-documentation-standards/chin-guide-museum-standards/metadata-data-structure.html

Stakeholders

• Direct Beneficiaries
• Educational Practitioners
• Policy and Decision Makers

• Students
• Parents and Guardians

• Teachers
• School Administrators

• District and State Education Officials
• Policy Makers and Government Agencies

More stakeholders

• Data and Technology Providers
• Community and Support Organizations
• Economic Stakeholders

• Educational Technology Providers
• Educational Researchers

• Community Members and Organizations
• Non-Profit Organizations

• Employers and Industry Representatives
• Funding Agencies and Foundations

Direct Beneficiaries Students: Primary beneficiaries whose performance and engagement are directly impacted by data. Parents and Guardians: Receive data about their child’s progress and use it to support their learning.

Educational Practitioners Teachers: Use data to inform instruction and personalize learning, and contribute data through assessments. School Administrators: Oversee data collection and analysis, and use data to make strategic decisions.

Policy and Decision Makers District and State Education Officials: Develop and evaluate policies using aggregated data. Policy Makers and Government Agencies: Develop regulations and allocate funding based on data.

Data and Technology Providers Educational Technology Providers: Offer tools and platforms for data management and analysis. Educational Researchers: Use data for research to improve educational practices.

Community and Support Organizations Community Members and Organizations: Advocate for educational improvements using data insights. Non-Profit Organizations: Use data to advocate for reforms and support specific populations.

Economic Stakeholders Employers and Industry Representatives: Interested in data related to skills and workforce readiness. Funding Agencies and Foundations: Evaluate the impact of programs and provide funding based on data.

Discussion

• What challenges arise from the different needs and priorities of these stakeholder groups when it comes to data use and management?

• How can these challenges be addressed to ensure that data is used ethically and effectively to improve education for all students?

What’s Next?

• Explore the R Basics column of Posit Cloud’s Recipes
• Complete the Prepare and Wrangle part of the case study
• Complete the Foundations of Learning Analytics badge
• Do essential readings for the next module