Communication with Stakeholders

Foundations Module 4: A conceptual Overview

Welcome to Foundations of Learning Analytics

Foundations of Learning Analytics are designed for those seeking an introductory understanding of learning analytics and either basic R programming skills or basic Python skills, particularly in the context of STEM education research.

It consists of consists of four modules. Each module of the Foundation of LA includes:

Essential readings

Conceptual overview slidedeck

Code a-long slidedeck

Case study activity that correlates with the Learning Analytics workflow

Optional badge activity

Module 4 Objectives

Fundamentals of Data Visualization:
Learners will grasp the basic concepts and significance of data visualization in distilling complex educational data into comprehensible and actionable insights.

Variety of Visualization Techniques:
Participants will be able to identify and apply different types of visualizations, such as charts, graphs, and heatmaps, to represent educational data effectively.

Practical Application and Best Practices:
Learners will gain insights into real-world applications of data visualization in education and learn best practices to enhance the clarity and impact of their visual data presentations.

Essential readings:

Optional readings in folder

In this presentation we are going to focus on communicating your data and findings effectively with education stakeholders. We’ll also talk about how to create publication-ready products in R markdown.

For the code along, we will continue on in our Learning Analytics workflow to finish up with Model and Communication. Where we will be converting R markdown files into a variety of formats and how to bring it together in a flexdashboard.

So the pre-reading for this module was chapters 21 and 23 in the R for Data Science book by Hadley Wickham and Garrett Grolemund. Both of these chapters are a really great introduction to this module. The information is presented really clearly and succinctly, and each section of these chapters also has a number of small very manageable exercises that will enable you to practice some of these skills. And I think we may be incorporating a few of these exercises into the code-along, but I just want to reiterate that this is a really high quality resource. So, if you didn’t have time to look at these chapters before today’s session, As mentioned previously, I recommend that you get this book or even bookmark it to have it on your bookshelf to aid you as you continue to practice with data analytics in R.

Discussion

Who are some of the stakeholders that you communicate your findings?
What experiences have you had with communicating the results of data analysis and how did you communicate these findings?
What are some things you are normally asked to communicate? What types of products must you produce?

Communicating Effectively with Stakeholders

Data Storytelling

“Data storytelling is a method of communicating information that pairs data with visualization and narrative tailored to a particular audience” (Anderson, 2020).

Education stakeholders:

Administrators
Teachers
Other practitioners
Students & their families

More Points

Tell a story (characters, setting, conflict, resolution)
Use “signposts”
Explain the data and why it matters
Add graphs and visualizations (but not too much!) to enhance understanding
Make it possible to dig deeper

STORYTELLING TAB

You may have found that you have similar experiences with your stake holders. Which is a nice segway into talking about communicating effectively with stakeholders. In the education setting you are going to have a variety of stakeholders that you’re working with and for. If you have actively been engaging in research and scholarship, you’re probably fairly comfortable at this juncture tailoring a message to fellow scholars. But oftentimes in the field of education we want to craft and explain our message for people who are not necessarily academia - like - people such as; administrators, teachers, other types of practitioners, and of course, students and their families.

One of the most effective ways that we can communicate is by telling a story. We tend to remember stories and we find them relatable. So one of the things I want you to think about as you’re going through this module, and as you are going through the code-along, and later as you try to carry these skills and this knowledge over to your own practice, is, How can I take what I’m working on or what I’m investigating and turn it into a story that will captivate and inform and perhaps persuade people?

MORE POINTS TAB

I also just want to add that in the references at the end of the presentation there is a really nice blog post that I have linked from a site called Storytelling with Data. And the specific post is entitled “visualizing education”. It’s a really nice example of ways in which different participants have crafted visualizations to tell a particular story related to education, and specifically many of these visualizations have to do with equity in education. So I really recommend that.

Okay so continuing in this theme of communicating effectively with stakeholders, we’ve already talked about the importance of telling a story. One of the other references that I’ve linked to at the end of this presentation is from the Harvard Business School online and this is a nice web page that talks about data storytelling as well. They elaborate a little bit on this concept of the story and actually talk about these elements of story that we typically think of more in the context of maybe journalism or creative writing. They emphasize the characters, the setting, the conflict, and the resolution. Your “characters” are your audience, who the message is being delivered to. “Setting” really refers to the context of the larger issue. So let’s just say that maybe your research is about women and girls participating in STEM fields, and so part of your setting or context might be to say that there’s been a significant increase in women’s and girls’ participation in STEM education programs but in the professional realm, especially in technical roles, there’s still few women working in the field. That’s your “setting.” All right, so then there’s the “conflict,” which they describe as the root issue. So this might be the specific research question you posed and investigated. The last element is the “resolution.” This part is particularly meaningful if your research has generated some clear dos or don’ts. Or perhaps if you’re trying to influence policy. What does your research show and what is the solution or the resolution that you are recommending to your audience? For example, maybe continuing with that previous example of women in STEM fields, perhaps you have found that women who participate in internships in their last two years of undergraduate education are more likely to enter technology professions. So that would be kind of your “call to action” or resolution.

Let’s talk about some of these other points. I have gleaned these recommendations from the Anderson, Ball, Cote, and Knaflic sources. Some of these latter points really speak to what your product looks like, so I think this is a nice transition because in just a slide or two we’re going to start talking about using R markdown to actually put some of these concepts into practice.

On the page from the data quality campaign, that’s the Ball citation, they talk about pointing audience members in the right direction. So I’ve rephrased that a little bit to say include signposts. And what I mean by that is that you’re going to include elements in your final product–whether that’s a presentation or a web page or a written document–you’re going to include visual cues that help direct the reader or the viewer to the overall organization of the document as well as major concepts and points. You probably do this intuitively in many ways but what we’re talking about here is using things like headers, bold text, perhaps different sizes or types of font, etc, to alert your audience to major points or transitions to new topics. This means that someone who’s viewing your report can easily and quickly go to the relevant information that they’re seeking.

The next point from the data quality campaign is to help your audience make sense of the data. So I think this is really important because I think sometimes data can be confusing to people. Having someone talk them through it is going to make it much more clear to them and help them understand and also make the topic more approachable. So take a moment, whether you’re communicating your data in person or in written form, to explain the data that you’re presenting. Don’t just present it but also explain what it means and how someone might interpret for example your visualization. And then you also want to explain why it matters. So I’m sharing the data with you, this is what it’s showing, and this is why it’s important to this topic.

Well that’s a nice segue to the next point, which is that we want to add graphs and visualizations to enhance understanding. Some people are going to be very drawn to the words and the text of your message whereas others are going to be drawn to the visualizations, so it’s important to have a blend of both. And each should enhance the other. I think that’s an important point, your graphs and your tables and your visualizations are not just there to look pretty, they’re actually there to enhance and better convey the message. An additional point that was brought up on the Storytelling with Data website is also to just be careful about the number of visualizations you have, and the amount of information you have in general. Everything that is there should be serving a specific purpose, it should be absolutely necessary, and there really shouldn’t be any more information than what is absolutely necessary. So kind of a “if the house was burning down what would you take” sort of mentality. This allows your audience to understand and process the major points of your report and makes it manageable for them to actually consume the information that you’re giving them.

A really important caveat though is that you are going to also make a plan for and provide a method for enabling your audience members to dig deeper should they wish to. So that could be accomplished by maybe including links in your presentation or site, including an additional list of relevant readings, or even providing other opportunities to speak with you or hear presentations about the topics in greater depth. So ideally you want to provide opportunities for people to delve as deeply into the data as they wish to, but you also want to avoid the “fire hose”, and so for that initial presentation should include a digestible amount of information that clearly conveys your data story.

Communicating Your Message with R Markdown

RMarkdown
Basics
Yaml
Table

Supports many output formats:

PDF
Html
Word
Slideshows…and more

Designed for communicating with:

Decision makers
Other data scientists
Future you

Plain text file with extension .Rmd
Code and its output appear together in the file/report*
Contains 3 types of content:
Optional YAML header
Chunks of R code
Text with simple formatting text
“Knit” to produce a complete report

Designed for communicating with:

Controls “whole document” settings
Here, we’ll focus on just two types of settings:
- output:
- bibliography:

Default table formatting in R Markdown is same as what appears in console. Add formatting with knitr::kable function For additional customization, see help within IDE (?knitr::kable).

Additional packages allow for further customization:

xtable
stargazer
pander
tables
asci

RMARKDOWN TAB

According to Wickam et al. “R Markdown provides a unified authoring framework for data science, combining your code, its results, and your prose commentary”p.423

All right so we’re going to step away from sort of the conceptual piece here and now talk about how you would actually accomplish some of these things in R markdown. So I started out here with a quote from Wickham and Grolemund from the R for Data Science book, it says “R markdown provides a unified authoring framework for data science, combining your code, its results, and your prose commentary.” So R markdown has some really nice features, and one of them is that you can use it to create reports in a variety of formats, including HTML, PDF, word, slideshows, websites, book formats, and more. And R markdown is designed for communicating with lots of different types of audience members, so it’s really a good fit for communicating data to education stakeholders.

BASICS TAB

As a reminder on the RMarkdown basics from Foundation lab 2 -

When you go into the R IDE, you will have different options for the type of file that you can open or create and R Markdown files are files with the extension.rmd. An R markdown file allows you to combine your code and the output of that code together in a single file or report. By default that your code and the code’s output are going to show up together in your report, however there are ways to adjust that. For example you can suppress the code, you can suppress the output, you can include things that you don’t necessarily want to run every time, you can even set it so that the code is visible but only if your viewer clicks a button to sort of unfold or show that code. So there are some options there.

The R markdown file contains three basic types of content: your YAML header, your chunks of our code, and interspersed with these chunks of code you have your text with some simple formatting. When you are done composing your report you are then going to knit it, which will convert your report to the format you choose, using a package called Pandoc. When you knit the file, you’ll have the opportunity to choose the format of your output and we’ll talk more about that in a bit.

YAML: TAB

So let’s talk about the yaml header first. YAML stands for yet another markdown language. The YAML header allows you to create settings that affect your whole document. And there are a number of settings and parameters that you can set in the YAML header but for today we’re really just going to talk about two primary ones, output and bibliography, and then a brief mention for citation styles as well. So off to the right here I just have a quick example, and this is from the Wickham and Grolemund text, of a very brief YAML header. And you can see in this header there is a title for the file, the date of creation, and then it specifies how you would like this report to be knitted. So in this case, the final product of this report once we knit it is going to be an HTML document.

TABLE TAB

So when we are creating reports, or articles for journals, it’s common to have tables to convey data concisely and in one place. R markdown’s basic format for tables looks pretty much identical to what is going to appear in the console, and this may not be the most visually appealing output. Luckily there are lots of different packages that can help you to create tables that are much more visually appealing and easier to read. So the formatting option that Wickham and Grolemunc recommend is to use the kable function which is part of the knitR package. And you can always learn more about that package in the IDE help function and I’ve just given you the command for that here. They also recommend several other packages that can help with even further customization of tables and I’ve listed these here on the slide.

Two ways to set output of a document
Permanently, by modifying the YAML header
Transiently, by calling rmarkdown::render()
Useful if you want to produce multiple types of output
When you knit, R will automatically put output in format listed in YAML header
- You can choose something different from the dropdown menu beside the knit button

Designed for communicating with:

Word: word_document
OpenDocument text: odt_document
Rich Text Format: rtf_document
Markdown: md_document
GitHub: github_document
PDF: pdf_document

Must install LaTeX package first

Built in formats
- HTML presentation with ioslides ioslides_presentation
- HTML presentation with W3C Slidy slidy_presentation
- HTML presentation xarnigan_presentation
- PDF presentation with LaTeX Beamer beamer_presentation ]

New slides begin at each first (#) or second (##)
- A horizontal rule (***) used to create slide without a header

Specify a bibliography file. ie: bibliography: rmarkdown.bib
use @ and the citation identifier Facts from articles [@smith04; @doe99]
Change style with citation style language (CSL)
csl: apa.csl

Books
- bookdown package
Themed documents
- prettydoc package
Articles formatted for specific journals
- rticles package

FORMAT TAB Once you have completed your R markdown document and you are ready to share it with your audience, you want to decide how that output is going to appear. So remember that R markdown enables multiple different types of outputs, including everything from an HTML document, to a word document, to a PDF document, and more. There are two ways to set the output of your document. The first is by modifying the YAML header, and we talked about that earlier in the presentation, where you set the output in the YAML header to the type of output that you want. And on the next slide I’ve given you the code for those different types of outputs. You can also also set the output transiently by calling this function here: rmarkdown::render() and that is useful if you want to produce multiple different types of output.

When you knit your file R will automatically produce the output of the format listed in the yaml header. But you can override this, by choosing a different option from the drop-down menu beside the knit button, so this is something that you’ll practice doing in your code along today.

DOCUMENTS TAB Okay so here I’ve given you just some different options for how you can set the output to a variety of different document formats. And off to the left is an example of what that sort of piece of code would look like in your YAML header. So in this particular example the output is set to be an HTML document, But it’s also going to produce a PDF document. So there’s going to be two different types of outputs. The PDF document is going to be the default output. But if you use that drop-down menu to select an HTML document output it will do that and it will also include a table of contents, which is indicated by the TOC = true.

Presentations TAB There are also a number of ways to create presentations in R markdown and I’ve shared just a few options here.and again I’m going to refer you to the Wickham and Grolemund text because they list some other packages for presentations in R. Just as kind of background information, if you’re using some of these packages listed here to create presentations there’s some basic formatting that provides structure to the presentation. So for example, new slides begin at hashes, and then you can add a horizontal rule (by adding the three asterisks) and that’s going to create a slide without a header.

Citations & Bibliography TAB Another really relevant form of output that you might want to create is a bibliography. That’s very necessary if you are producing scholarship, and so we wanted to talk briefly about how you would include a bibliography and citations. So Pandoc is actually the package that helps translate your R markdown file into your final report. And so Pandoc can generate citations and a bibliography for you.

The way that you’re going to do that is you’re going to specify a bibliography file in your file’s YAML header. And you want to make sure to include the path from the directory of your R markdown file to the directory of your bibliography file, if they’re not the same. One thing that they note in the R for Data Science book is that this command will print your bibliography but it typically does not print any sort of header, so one of the last things that you should include in your Rmarkdown file is a header, so for example # (because the hashtag remember is going to indicate that it’s a header), and then the word “references” or “bibliography”.

Typically you’ll also need to add citations within the text document, or in-text citations. And the way that you’re going to do that is to use the @ symbol and then the citation identifier, and I’ve given you an example here on the slide.

Lastly, you can change the style of your citations and your bibliography with a citation style language or CSL command in the YAML header. So here I’ve given you an example for how you would set that citation style to APA. And I’ve given you some other references about citations and linking different citation managers with R a bit later in this presentation.

OTHER TAB

Given our time constraints today we don’t have time to talk about everything that R markdown can do. Not by any means. But I did just want to let you know that there are other output formats that R markdown is capable of and I’ve listed them here. And you can feel free to do investigation on your own to learn more if you’re interested. The one that I wanted to mention specifically though, was the rticles package because if you are engaged in scholarship and you’re trying to get your scholarship published this could be very helpful. The rticles package allows you to format articles for specific journals. So, if you are fairly advanced in your R markdown skills, and some of the information presented in this module is old hat for you, one of your afternoon activities today could be exploring the rticles package a little bit more.

Ethical Considerations in Learning Analytics

Data Privacy
Bias in Data Analysis
Student Consent
Responsible Data Usage
Inclusive Practices
Transparent Communication
Continuous Evaluation

Data Privacy: Challenge: Protecting sensitive student data from unauthorized access or misuse. Initiative: Adoption of robust data encryption and secure storage protocols to safeguard privacy.

Bias in Data Analysis: Challenge: Unintentional bias in algorithms leading to unfair outcomes. Initiative: Implementing algorithmic transparency and regular audits to identify and correct biases.

Student Consent: Challenge: Ensuring students are fully informed and consent to data collection and analysis. Initiative: Implementing clear consent processes and providing students with opt-out options.

Responsible Data Usage: Challenge: Balancing data-driven insights with ethical responsibilities. Initiative: Establishing ethical review boards to oversee learning analytics projects and decision-making.

Inclusive Practices: Challenge: Ensuring learning analytics benefit all students, regardless of background. Initiative: Encouraging inclusive data collection and analysis practices to address disparities.

Transparent Communication: Challenge: Effectively communicating learning analytics goals and methods to all stakeholders. Initiative: Promoting transparency through clear communication and accessible documentation.

Continuous Evaluation:

Challenge: Keeping up with evolving ethical standards and challenges. Initiative: Engaging in ongoing evaluation and adaptation of ethical guidelines.

Discussion

Think about the following questions and then discuss at your table:

What do you think are currently import ethical considerations for LA?
How can educational institutions balance the need for data-driven decision-making with the ethical considerations of data privacy?
How can educators ensure that student data is collected, stored, and used responsibly while still enabling personalized learning experiences?

Important Ethical Considerations for Learning Analytics

Privacy and Consent: Ensuring that data collection processes are transparent and that students (or their guardians) understand what data is being collected, how it will be used, and have given their informed consent.
Bias and Fairness: Identifying and mitigating biases in data collection, analysis, and the algorithms used in LA. This is essential to prevent discrimination and ensure fair treatment of all students.
Data Security: Protecting sensitive student information from unauthorized access and data breaches.
Use of Data: Establishing clear boundaries on who can access student data and for what purposes. Ensuring that the data is used to benefit students’ learning experiences and not for punitive measures or profiling.

Balancing Data-Driven Decision-Making with Data Privacy

Data Minimization: Collect only the data that is necessary for the defined educational purposes. Avoid excessive data collection that can lead to privacy concerns.
Robust Data Governance: Develop and enforce strict data governance policies that detail how data is to be handled, who has access, how it is protected, and how long it is retained.
Transparency and Communication: Maintain open communication with all stakeholders about the data being collected and its uses. - Transparency can build trust and ensure that the community understands and supports data-driven initiatives.

Ensuring Responsible Data Use while Enabling Personalized Learning

Data Handling Policies: Implement policies that specify how data is to be collected, stored, analyzed, and deleted. These policies should comply with legal standards such as GDPR in the EU or FERPA in the USA.
Secure and Selective Technology: Utilize technology solutions that offer strong security measures to protect data and ensure that these technologies allow for the controlled use of data to prevent misuse.
Personalized Learning Frameworks: Design personalized learning experiences that use data responsibly. This could include systems that allow students to view their own data and understand how it is being used to tailor their learning experiences, thereby enhancing transparency and control.

What’s next?

Complete the Communication section part of the Case Study
Complete the Badge requirement foundationlab4_badge- Data Visualization