Data evolution & revolution

The past

Data has been an undercurrent in my teaching since my first classroom in 2007. Of course, in that year, I struggled to gather data and there was virtually no chance of utilizing much of it to inform and enrich instructional planning. For good or ill, data is not essential to the survival of a first year teacher.

Each year after, I slowly improved, including a variety of experiments like the one shared in the post Student Empowerment | COETAIL final project. I tried different forms, organizers, notebooks, etc, until finally unveiling an integrated digital system last year. I shared it as a presenter at the GAFE Summit 2016 in Kobe, Japan, and used it for the school year to publish students’ ongoing assessment data, and other key information such as website usernames and passwords, directly to them as web pages. After celebrating and discussing the system, I felt that it was terribly unsatisfying.

The present

Inspiration came in the form of media such as Jack Norris’ keynote presentation from Strata + Hadoop World in San Francisco, Let’s Get Real: Acting on Data in Real Time, embedded below.

The concept of ‘data agility’ through converged data and processing appealed to me because what I sought a tool which would organize all assessment data in a way that could be searched, shared, and analyzed. Over the years I had been introduced to many ‘tracking systems’, only to discover that they were utterly unmanageable at scale. Ticking boxes on scope and sequence documents or highlighting learning objectives almost arbitrarily seemed like a show at best. In fact, a colleague who shared such a system with me admitted that at the end of a term, due to a lack of hard data, he would simply choose outcomes to highlight on every student’s document regardless of their actual progress or learning. To quote Mr Norris, I wanted my data to ‘get real’.

While designing my own system, I became somewhat of an amateur data scientist. The implications of the article Putting the science back in data science got me thinking about the flow from data entry to visualization and publishing. A quote from the post Can Small Data Improve K-12 Education? helped to clarify the objective for the project.

‘Small data observes the details or small clues that uncover large trends. The idea is that by honing in on the elements that make up relationships and narratives in schools, education can be enriched.’ The Edvocate

What I wanted to do was bring transparency to the relationships between myself, students, parents, and administrators. Further readings within the big data and data science trends like Data Quality Should Be Everyone’s Job  by Thomas C Redman directed my attention toward the purpose for the data. Before data is collected, it should already have a purpose, and that purpose dictates the design of the collection, publishing, and analysis tools.

 

screenshot-5
Copious data entry (lots of dragging)
The next piece of the design puzzle was my school’s Assessment Handbook. In it were the categories, criteria, and descriptors on top of which my system would function.

 

screenshot-4
Student data visualization via Google Sheets
Utilizing a system of Google Sheets, data is entered and student progress viewed in potentially real time, depending on the efficiency of my data entry. As we began using the system I shared a video, Tour of your data book, embedded below, which illustrates the details of the user experience much better than I can describe in words.

The future

This system has been remarkably effective and unlike last year, I only plan to make minor tweaks, especially to the user interface. Feedback from students and parents revealed, as I expected, that there are too many graphs and that it’s difficult to know which are more or less important.

Another feature I plan to add is a Google Form which mirrors the data entry document which would allow teaching assistants, specialists, and even parents or students themselves to contribute data to the system.

If articles like The Three Ways Teachers Use Data—and What Technology Needs to Do Better by Karen Johnson and 7 Steps to Becoming a Data-Driven School by Eric Crites are any indication of the direction that data utilization is heading in education, I’m glad to be along for the ride.

Impact on learning: Language and engagement

One admirable feature of professional development at KIST is the annual Impact on Learning study. Teachers design a data driven experiment based on a pedagogical approach or strategy and then analyze the data to reflect on the efficacy of that aspect of their teaching.

To start, I formulated a question and answer dialogue:

On which group of students do I want to have the greatest impact?

All of them. Inclusive practices and thoughtfully designed learning experiences which emphasize student choice and voice should provide opportunities for all students to excel.

Which group of students are most difficult to reach with inclusive practices and learning experiences that emphasize student choice and voice?

Students who are reluctant to share their ideas in class or participate actively in learning engagements are the most difficult to reach. 

Why don’t those students participate?

The reasons they don’t participate are as diverse as the people themselves. However, if they don’t participate now, they likely didn’t before either. If not, then their opportunities for practice have been limited, possibly severely.

Often, students (and people in general) with little experience speaking in a group feel shamed by their lack of fluency. Lack of confidence leads them to withdraw more, causing them to practice even less.

I have been tempted in the past to ‘call out’ reluctant students, but Alfie Kohn’s article, ‘Your Hand’s Not Raised? Too Bad: I’m Calling on You Anyway, provides needed perspective into this issue. When done improperly or insensitively, calling on these students might do more harm than good.

Being fairly introverted myself, I sympathize with many people’s preference to remain in the shadows of a crowd, but nine year old introverts, preferences aside, need to practice public speaking in a safe environment.

//platform.twitter.com/widgets.jsArticles like Chapter 1. Why Talk Is Important in Classrooms from Content-Area Conversations by Douglas Fisher, Nancy Frey and Carol Rothenberg, and Talking to Learn by Elizabeth City reinforce the position that Listening and Speaking form the foundation of Reading and Writing.


What I needed was a strategy to encourage the students to grow as courageous Communicators by sharing their ideas with the whole class.

Gathering data

My methodology for gathering data is simple. At various times in class, I propose an open ended question. For example, I might ask for interpretations of an idiom, impressions of an image, or opinions about a famous quote. That there are no correct or incorrect answers is made clear to students, as is the fact that ‘I don’t know’ is an acceptable response. Students may also ‘pass’. Sometimes the provocations are directly connected to our unit of inquiry, sometimes not.

Using a deck of laminated cards with the students’ names written on them, I ensure that every student has an equal opportunity to speak. My response to every contribution is ‘thank you’, and I very rarely paraphrase or ask clarifying questions in this context. To students who ‘pass’, I simply respond with ‘OK’.


Cards are separated into two categories and then data entered into a spreadsheet about who contributed an answer and who did not.

Visualizing data

The raw data is relatively easy to process to produce interesting graphs.

Some students always talk and some never talk by default. Filtering out those students makes the graph more readable, but still not very revealing.


Referring to diagnostic assessment data in Reading from the beginning of the school year, I included only students who score ‘just below expectations’ or ‘below expectations’.




Next, filtered for students who scored ‘Just below expectations’ or ‘Below expectations’ on Aug diagnostic assessment in Writing.


  
Again, this graph doesn’t instantly reveal anything other than a general upward trend in participation.

Next, I wanted to explore a possible correlation between this exercise and improvement. The next graph is students with more than 10% improvement on ongoing informal and formal assessments in Reading from Q1 to Q3.



Next, students with more than 10% improvement on ongoing assessments in Writing from Q1 to Q3.





Interpreting data

This is the first graph which indicates a clear correlation. With only two exceptions, students whose writing has improved are also increasing their participation.

Since the objective is to improve language skills, I tried including only students who consistently achieve below 80% on ongoing assessment in English language.




Compare to consistently strong achievers in language.



Interesting that consistently higher performers seem to have random participation while consistently lower performers are participating progressively more and more.

If only for the purpose of having more data visualization, members of most advanced guided reading group.



And the least advanced guided reading group.




It’s exciting to see that this activity is impacting the exact group of students it was designed to benefit.

When the class completes its end of year diagnostic assessment in Language, I expect to see similar improvements among students who have gained confidence as communicators through this simple activity.

Finally, here is the whole class average.



Unexpected outcome

Rather than analyzing individual students, this graph reveals something I hadn’t expected. If I compare the number of opportunities to speak with the average rate of participation, there is stark correlation.

Number of data points (% participation):

October 4 in one week (61%)
November 16 (61%)
December 3 (41%)
January 7 (60%)
February 7 (74%)
March 7 (71%)

It would seem that the more we do this activity, the more participation there is. Thinking of the student trying to build confidence, it makes perfect sense. If one hesitates, one loses an opportunity. However, missing a chance might be just the motivation one needs to seize the next one. If that next opportunity comes sooner than later, one is more likely to take it.

And so, the data comes full circle from thinking of individual students, back to individual students. 

Peter Gow’s post, The Data Challenge for Schools – What Problem Are You Trying to Solve?, reminds me that the importance of data is not about averages, it’s about outliers. The greatest impact can often be made where there are cracks or gaps in the data. What is important is being intentional when gathering data so that when it is organized and interpreted, it answers the initial question.

It’s also important to remember that while ‘data’ and ‘gut’ are not the same, as Doug Johnson notes in his post, Data or gut?, through investment in time and training, it is possible to align the gut more precisely through data.

GAFE Summit 2016

A few weeks ago, I attended the Edtech Team Summit Featuring Google Apps for Education in Kobe, Japan. It was my second ‘GafeSummit’. The first was in 2013 and was a dramatic turning point in my career as a teacher and my life as a digital citizen.


The one notable difference was that this year, I would be presenting a session on Google Apps for Transparency.

https://www.instagram.com/p/BBbXZe8jSpA/

360º



My first eye opener was Jim Sill’s session, Google Views – Lessons in 360º, in which I was introduced to Cardboard. This is a realist iteration of virtual reality that could be easily integrated into schools. Although I haven’t had other VR experiences, I wonder if Cardboard offers a majority of the sensory experience.

The flow

Overall, I was most inspired by Stephen Taylor’s Formatting the Flow session. As an inquiry teacher, I have always wrestled with the impulse to manage students’ learning. What Stephen showed was how formatted documents can make processes visual and focus students on their learning rather than their presentation and reporting media.

BreakoutEDU

A photo posted by Bart Miller (@botofotos) on Feb 6, 2016 at 8:13pm PST

//platform.instagram.com/en_US/embeds.js
My group was beta testing BreakoutEDU with augmented reality and was not able to open the box like some other groups.

//platform.twitter.com/widgets.js

Transparency

Finally, it was time for my presentation, Google Apps for Transparency.

As a form of modeling, I shared a Transparency notes Google Doc with all participants for public note taking and documentation.

//platform.twitter.com/widgets.js

I began with a brief introduction to the concept of transparency as it is viewed in practice in government, business, and education. Then, following a generally ‘less to more’ transparent framework according to the slides embedded below, I shared the tools that I use to make planning, teaching, and assessment in my classroom as transparent as possible.



Included in the demonstrations were my weekly planners. I use a template in Google Sheets that allows me to plan to five minutes of accuracy include relevant details including differentiation. These documents are published as a webpage and the link is shared on our class Moodle site.

Having the plans published via a link allows easy access from any internet connected device. A classroom computer at the front of our classroom is dedicated to our projector, but it also has all of our links saved as bookmarks in the web browser. Throughout the day, students check these links. This increases the amount of time that I can devote to learning by minimizing questions like ‘what are we doing next?’ or ‘what’s after lunch?’.

https://docs.google.com/spreadsheets/d/1v7jufrMw8HRPBCEA6CfuhsDhZKSbkts7IKBNN6cYOhc/pubhtml
Click image to view as webpage.


A teacher in the workshop asked if there was added stress from publishing all of my planning. I replied with that this level of transparency adds a component of accountability that is its own reward.

Using the publishing capability of Google Apps, I also publish slides of our daily warm ups and home learning assignments. They are embedded on our class Moodle and require no additional maintenance. They update automatically when new slides are added. If a parent or other member of our learning community uses them even once to have a conversation with their child or keep up to date on home learning, it’s worth the minimal effort to set up.


Finally, I shared my data workbook. This is a system of spreadsheets that provides me with real time data from assessments and then publishes the same data to individual pages, published as websites, for students and families.


This works extremely well for parents to keep up to date on their child’s learning and for sharing web addresses, usernames, and passwords.

All materials for the workshop are shared in a public Google Drive folder, Transparency | GAFE Summit Kobe 2016.

Strangely, as soon as my session ended, I felt the urge to develop a new data management system that could provide more possibilities for data visualization and analysis. I’ve already begun sketching ideas and look forward to designing and programming this summer.

Reflection

I’ve completed tons of online professional development, and nothing compares to the invigorating social and interactive experience of a face to face conference. Ironically, this can be especially true in technology where digitally isn’t necessarily the best way to learn something new.

The tools which I have put to work immediately are Quizizz and SafeShare. Since introducing Quizizz, my students constantly ask when we will be taking the next quiz.

Reflecting on my own presentation, I feel that I probably learned more than my participants! It is easy to feel that the time and energy spent preparing to conduct a conference or workshop session is wasted, but I found the opposite. By deeply analyzing and presenting my approaches to technology in the classroom, I deepened my understanding. Being inspired to expand my strategies was an unexpected surprise!

If you’re curious to explore the conference, follow this link to view the full schedule.

I’ve already been contacted by Google related colleagues about organizing an event in Tokyo, so I look forward to putting some of that inspiration into action.