Showing posts with label learning analytics. Show all posts
Showing posts with label learning analytics. Show all posts

Wednesday, January 9, 2013

Inspirations from Eric Mazur

Yesterday afternoon I was watching a really nice talk given by Eric Mazur, professor at Harvard University, in the Physics department. I knew about it thanks to my colleague Mar's twitter (@marimar_ps), but I was pointed to Eric's work before in September, by Al Essa (@malpaso) at the ECTEL'12 conference.  Here is the video: let's watch it and then discuss about it.


At the beginning of the video, Eric talk about something I've already noticed in academia: some excellent researchers forget about scientific method in their teaching, and they are not consequent with data facts in the classroom. As Eric wisely paraphrases: "the plural of anecdote is not data"; in other words, as teachers we have to look at the data that arises after our interventions in the class and should make sense of it in order to improve our teaching, make future interventions, etc. Eric is selling the learning analytics product with a different wrapping.

After this nicely done introduction, he explains 3 examples of his usage of data in the class, examples that result quite different from a first intuition. The first one is about the gender knowledge gap for the current instruction methodologies, and that he claims it could be flattened by means of collaborative methodologies. I really don't like much this example since the assessment methodology used to measure students knowledge (standardized tests) could also affect the results in a different way for both genders.

The second example is about demos: Eric demonstrates the inefficacy of just doing a demo in class without previous thinking about the expected results and discussion with colleagues. Really neat, and something I will take into account in the future. And another nice advice in this part: when a teacher is grading a student exercise/exam he should not only indicate what was good/bad but also try to think how the students reached to that conclusion in order to avoid this kind of failures in the future. At this point, someone in the audience pointed out the importance of formative feedback.

Finally, the last topic and my favorite one was confusion. Was confusion bad for students? If there were no confusion maybe means that there was no learning at all. Not always non-confused students obtained the best results. In fact, students that did bad in a exam were not aware of their deficiencies: they thought that they were not confused at all because they didn't even have started to understand the subject, and they didn't pose the right questions to themselves. Therefore, this awareness information is a very powerful tool that should be given as fast as possible to the students for them to understand their deficiencies and be able to conduct their learning path. And if this information is backed up with strong, reliable data... the better.

Thursday, August 23, 2012

GambientFication

A few weeks after my last post about designing a use case using ambient displays, I'm back to report about the experiment I carried out (started last Tuesday). I have developed several artefacts to show in the ambient display (an iPad at the end) as a digital frame (like it was part of the furniture): a couple of charts using highcharts, a couple of games and an activity stream (really linked to it, not developed it), all of them integrated in a single app. As I told you before, the data I use comes form tinyarm, that tool for researchers to report read/skimmed papers, suggest to each other and create a list of papers to read. Therefore, my research this time fits better in CSCW (computer supported collaborative work) than in the e-learning topic, but I hope I could extrapolate some of the results (and the created artefacts) to that field too.



As I described, my intention is to provide awareness to the users (in this case researchers) about their activity (read papers), and engage them in the process of reading and suggesting papers each other. We'll see if the results of the evaluation of this research point into that direction.

In this post, I'm going to focus on the games as the title denotes. The idea is that the ambient displays could be used to enrich the ambient of the user with digital data, and I'm mixing this concept with gamification (the usage of game dynamics and mechanics to increase the engagement of the users). That's why I coined the term of GambientFication, or the use of ambient displays to show the users some parts of a game, that is itself the result of gamifying another process (in my case, the activity of reading/suggesting papers by researchers).

Regarding my own experiment, I ended up developing a couple of games: "The Hangman" and "Risk" (both of them modifications of popular games). Almost everybody know The Hangman. Its core mechanics consist of representing the failures you make trying to guess letters in a word by means of a man hanged: the most letters you fail, the more parts of the hangman are drawn. If you succeed to guess the word without being hanged, you're dead. Annotating the number of failures in another ways (just with numbers or little sticks) would not be the same: you don't want to be hanged, man! Why did I choose to link the hangman mechanics to research activities? I thought quite a bite about it, and I ended up finding a solution: as a researcher, if you don't read at least 2-3 papers per month, you got disengage from the state-of-the-art, you got dead, you got hanged! That's it! So, let's use the metaphor of the hangman to say that to researchers.


On the other hand, I wanted to make another game for several reasons: do something a bit more complex, visually attractive, and to compare the hangman with. So I looked for another metaphor and, although my colleagues didn't like it at the beginning, I ended up implementing that too: a modified version of Risk. Risk is a board game which objective is, in turns, to conquer the world accumulating army and expanding your troops abroad your domains. But I didn't want to foster competition among researchers for "who reads more papers in the group", so I modified the mechanics. The participants cannot conquer other participant's country, so the objective is not only individual (conquer as much countries as possible) but also made up a collective/collaborative one: the group members should conquer a complete continent in a given time period. Therefore, the individual work turns also into a work for the team, expecting that collaboration flourish and out of it the group itself appears strengthened. Anyway, certain level of competition is not bad also, so there's a case when 2 participants want to conquer the same country and only one will succeed.


That goes without saying that I took advantage of the HCI principles and techniques that I learnt here in Leuven (my first idea of the darts game seems so far in time), to develop the games and the other components. And hopefully the results of the experiment could led to a nice publication, as a result of the collaboration with the HCI group in KUL.

Monday, July 23, 2012

Ambient displays and exploring datasets

Ambient displays are artefacts used to arise awareness about certain data not directly observable in the physical world (e.g., learning progress), and integrated in the environment. That is, they are not the main focus of the users activity but display information that could be consulted when needed, that make them aware of something interesting for their activity. In the concrete case of ambient displays for learning, several use cases have been analysed in the awesome Dirk and colleagues' literature review. Some conclusions could be drawn out of their work, for instance, that ambient displays if well used have been demonstrated not to distract the students. And that generally the amount of data displayed is quite low. The most sophisticated displays got to make what they called situational awareness, that is, besides acquiring some knowledge and understanding of the situation, the users could obtain a pinch about the future; that is, the data displayed if correctly interpreted could predict future events.

Therefore, I'm playing around with some datasets to come up with a suitable visualization for an ambient display: my dataset from the classon use case; another dataset from tinyarm, a tool for awareness of researchers; and, in the near future, with a dataset from stepUP. I have to first explore the dataset in order to determine how the visualizations should look like (bottom up approach). But I'm also exploring some tools for creating data visualizations, in order to get to know some novel visualization approaches that could be applied to the data (top down approach). Let's see what could I find in the mid-way, supported by the tools and the data itself.

In this regard I wanted to quickly visualize the tinyarm data, provided to me in JSON. I could not find a suitable tool, so my best chance was to export it into CSV using a JS script (I'm using JS for absolutely any processing I need, executed with node). I was pointed to a couple of tools to explore: on the one hand gephi, but it is more about networks (nodes and relations), not so useful for visualizing time series; and gapminder, that could be used to nicely visualize data about countries in the world, but you cannot upload your own data. Anyway, they point to another solution in the FAQ: to use Google Public Data Explorer, where you could upload your data in Dataset Publishing Language. For using this format, you should have your data in CSV plus a XML file describing your data: concepts (organized in topics) and their relations (slices). I can say that constructing the XML is not so easy (and so funny). Besides, I realized that there's a simpler and more powerful tool that I could use: Google Spreadsheets. So, after processing the data with the JS script I'm uploading it into a spreadsheet and visualizing the data using the built-in charts, very powerful to visualize this dataset (around 1.5k rows).

Regarding the visualization of the datasets for the ambient displays, my co-advisor @rcg0 pointed my to a very interesting blog post about visualization of speedos and tachometers. Roughly, it describes why the "in principle not convenient" circular visualization of the speedo make a lot of sense to detect acceleration. At the end of the post a visualization of the last BMW dashboard is presented, and the circle of the tachometer and speedo are linked by a fuzzy line to indicate that both measures are related, and this relation is important in sport mode (in this mode the link appears). That relation could be used as a metaphor for the relation between effort and outcomes (quality outcomes), since a great effort should imply a higher speed if controlling the gears correctly. Sometimes the students claim to devote a lot of effort in a course and the outcomes are not the ones they expect. By means of this dashboard they (and we as teachers) could detect easily these situations and we'll be able to help them to change to the correct gear.
Image from http://blog.visual.ly/
I have created a simple visualization using the Google chart's gauge. Let's see if it works for the purpose.

Thursday, June 23, 2011

Visualizations in the feedback loop

Second post of the day: I surprise myself!

This evening I attended to the workshop of R given by @jfelipe (Felipe Ortega) corresponding to the Visualizar'11 seminar. I was playing a bit with R in the past, and for me is not much more than the "open Matlab"; but it has nice features like extensibility installing new libraries and the graphs obtained are pretty beautiful... but interactiveless!

But after the workshop I had the great opportunity to talk with @infosthetics (Andrew Vande Moere) about my own research, that is, my PhD, and how I am introducing now the concept of visualization in it. And the 15 minutes we discussed were much inspiring and made me reflect about the directions that my PhD is taking (yeah, it's kind of live creature). Let's try to summarize the main ideas.

One of my previous ideas we discussed was the visualizations of learning events collected in the lab sessions will help teachers to make better decisions: visualizations to make them more intelligent (as stated in my previous post). Teachers will be able to orchestrate the classes in a better way and provide more effective and quick feedback to the students (hopefully!). I confirm that this is a good idea if executed appropriately: cool!

But the majority of the discussion was about the other leg of my topic: introducing visualizations for the students for awareness and reflection. He recommended me three important things these visualizations should comply with to be engaging (I have to measure it!!) for students: they have to be beautiful (esthetically good), fluent (that is, very straightforward and simple interaction, like this example) and that is part of a story. As I understood (if I did at all), this means that the visualization is not the end of the process (of learning) but a part of the story; it has to put into the mind of the student the next action he/she has to take (yeah, this is getting complicated!). Anyway, I am going to present to the student these engaging visualizations with their individual data and the aggregation of the data of all the students in the class. And I am going to collect all the data I could about access to the visuals, discussions between students and with the teacher, etc. to be able to demonstrate engagement.

And then I have to evaluate the tool I am developing, in a real scenario: typical control group and experiment group. I have to think better how to measure engagement (qualitative parameter measured as quantitative as possible?) and learning, decomposed in the attainment of learning outcomes that the learners are achieving during the course. How to measure that? Think, think, think. And, of course, I have to have always in mind the ethical implications of my experiments: one of the groups will have an advantage over the other if testing my tool with them? What if I don't know exactly if it works? I feel the responsibility over my shoulders...

Thanks Andrew for the nice+inspiring+engaging+fascinating conversation! I very very much appreciate it!