Above/Beyond: Data Analysis

Whenever possible over the next three weeks, I’m going to draw your attention to opportunities to generate evidence for those top profiles. In this case, we’re looking at the idea of doing “more than just what is required,” which is critical to a GAP 8:

Click to see a legal-size PDF of the profile poster in Room 210.

Nothing in this post is required. There won’t be a Google Classroom assignment with a formal assignment. Instead, I’m asking you to think differently about the work you do and the feedback we generate. It’s not just about our different form of feedback, though; it’s about the sort of collegiality that sits at the top of our set of skills and traits:

Click for a copy of the skills/traits poster from Room 210.

You should work together to make sense of these data, starting with…


[Onomatopoeia] Quiz: 9/19


Here is a PDF of the response summaries Google Forms gives us:

[pdfjs-viewer url=”http%3A%2F%2Fsisypheanhigh.com%2Fmalachite%2Fwp-content%2Fuploads%2F2017%2F09%2Fdd-quiz-091917-pdf-1.pdf” viewer_width=100% viewer_height=500px fullscreen=true download=true print=true]

 

You don’t need to read every response to get a sense of the sort of thinking and understanding we’re dealing with, but every response is here. What do you notice?

I see an unwillingness to leave a question blank, despite a lack of specific knowledge that would answer the question. That’s worth unpacking. It suggests that you are trained to try to earn every point possible in academics, even if you don’t stand a chance of actually doing so. You would never willingly leave an answer blank, even in a grade-abated course, because you latch onto any non-zero chance of earning points. But leaving answers blank is precisely what some of you should have done here.

That’s a general observation that’s worth some writing and discussion from you. Why not leave an answer blank if you know you don’t know it? Obviously, some of it is a lack of reading directions. The context for this diagnostic gives you permission to leave things blank. And that’s a secondary observation: Many students fail to read instructional posts first.

What you could do, if you wanted to generate evidence of “more than just what is required,” is to consider the specific questions from the quiz. What patterns do you see in student responses? How do most people interpret the GAP tiers? Is there a common denominator in what’s missing, poorly understood, etc? What information is missing?

Analyzing this set of data will teach you what wasn’t clear on 9/19, which is the point. If you want evidence of an 8 or 9, you’re going to have to get better at the core skills, traits, and knowledge of the course.

You might also look at…


The Summer Reading Responses


Below is an image gallery of screenshots taken from the summary responses to your summer reading assignment. You might be able to see these by loading the Google Form, but this makes it easier.

You should also read this PDF of responses to one of the questions, which gives you more information to consider.

All of this is an even further step beyond “just what is required.” The data from the quiz on 9/19 will help you adjust to the course as it functions all year, so you have a real motivation to spend time thinking about your work, the work of your peers, etc. This stuff, like the writing you did about summer reading, is functionally very different.

Which is why I’ll leave the prompt even more open-ended. What do you notice about those charts and graphs? What about the data might be meaningful? Why should we care about what you and your peers have indicated in these responses?

Ask questions below about all this, and please let me know if there are any errors loading or reading the data.

Bookmark the permalink.

22 Comments

  1. Christiana Santucci

    How do we as students detach ourselves from the thinking that putting an answer down for a question even if we have no idea what we’re are talking about is better then leaving no response at all?

    • Mr. Eure stated that one solution might be to simply read the directions, and while I’m sure that was the issue with some students, I don’t think that was the case for every student who lacked the knowledge but felt the need to respond. Some students read the part that gave us permission to leave questions blank, yet could not find it in themselves to do so. Just like the Ken Robinson video described, we have been trained since preschool with standardized testing courses to gain as many points as possible. I think that one way to break free from that concept for this class is to simply understand how the course works. This can be very circular because the quiz was testing us on our understanding of the course, yet people put meaningless answers because they never read the material. Students will always respond because they feel that there will be consequences affecting their grade. If they understand how the course works, then they would know that doing well on the quiz is not the only thing that the teacher is looking at. It is our in-class focus and online work that will determine what profile we fall under. If a student did not know the answer to one of the questions because they were confused by the readings and videos, then they should have formed a feedback loop with the teacher or their peers by asking questions. However, I’m sure that most students did not know the answer to any of the questions because they simply haven’t done ANY work. I think that it needs to be made clear that it is worse to answer without evidence than to not answer at all. More students would be inclined to do so and admit that they truly didn’t know the answer if there is an opportunity after the quiz to go over what they don’t know. There was no actual number grade at the end of the quiz, so maybe it shouldn’t be called a quiz. Maybe it should be called a learning checkpoint to make students less uneasy, and more excited to actually learn.

      • You’re right about the need to call this something other than a quiz. The linked article on Medium explains how this would normally be called a pop quiz, but since it doesn’t really suit that name, Mr. Eure wants to describe the quiz using another form of onomatopoeia. But that’s an interesting point to not call it a quiz at all.

    • Someone like you needs to voice that idea repeatedly, invoking the teacher and course as having given permission. But it takes more than that. I think we have to realize that certainty feels different, even if it’s couched in an awareness that we don’t know everything about a subject. I’ll think about it more and get back to you all.

    • I’d agree with this question. This will be very hard for us to achieve because after years of students in our level class being successful, we are wired to do anything to succeed. In our minds leaving a question blank is the equivalent to getting it wrong or not getting the points we desire. How do we stop our minds from thinking like this after having it be trained to do so for so long?

      • This reminds me of something really interesting and kind of ironic. Lots of standardized tests used to deduct points for a wrong answer, but not for a blank answer. But recently, they have changed and wrong answers no longer carry any penalty, so the tests are actually encouraging guessing and not leaving anything blank. On school tests too, students are so used to guessing whenever they don’t know the answer. That is probably a reason why the idea of not leaving anything blank is so ingrained in our brains.

  2. I agree that we all try to leave a response on these sorts of forms because we are sort of trained to get the maximum points possible. However I also think that some students may answer questions that they don’t really understand or know the answer to because they think that a teacher may see it, know that they need some help, and finally see that they are trying but just do not get it. Leaving an answer blank makes it seem that the student is not even trying to answer the question.

    • Please bring this up the next time we’re all together. It’s an aspect I hadn’t considered, and I’m glad you noticed it: Some people struggle to ask for help, and that can play havoc with them in a course like this. There might always be a fear of seeming weak, even in the most self-aware person.

  3. After taking the time to go over the results of the (Onomatopoeia) Quiz, I made a handful of observations. First, (this was most glaringly obvious to me) many students used poor grammar or incomplete sentences or (a surprisingly large number of) incorrectly-spelled words. This was surprising to me because whenever I post something (even right now), I take the time to reread it and consider what I was saying, and I think it is discouraging that a lot of students don’t take that much time to record some meaningful thoughts and edit them down for clarity. If these students can’t take the time to check their own 2-3 sentences for spelling mistakes, are they really going to dive into instructional posts on Sisyphean High?

    Second, I noticed that a lot of students used their resources to write their responses, and so instead of documenting in their own thoughts the essence of the question, they instead paraphrased the language of the essential course materials that Mr. Eure printed for us. This ultimately serves no purpose since the student will not internalize this information and no one else learns anything from this student’s perspective. Mr. Eure will not be impressed by our ability to reword and type information from documents he provides us with, so the copying of this information isn’t even making the student look good. So the question has to be asked (not to piggyback off of Mr. Eure): why didn’t the students admit that they didn’t know the answer? I agree with some of Mr. Eure’s ideas in the post (about the slim chance students cling to that they will receive partial credit), but I’d like to add that I think a lot of us haven’t grasped the underlying idea that the process of learning is often the learning itself, that the final product matters less than our process. Therefore, some students see the best way to demonstrate success is by having the right answers, but what would truly be meaningful is if they internalized that knowledge through critical thinking and discussion and metacognition and engaging with the feedback loop (that’s how we measure success here, by analyzing the process by which the student learned something), which would ultimately enable them to record their thoughts in the form without having to copy part of a previous post or comment.

    Finally, I want to add that while the length of the students’ responses isn’t indicative of their understanding, it seemed to be true that students who were substantially more brief than their peers demonstrated less of an understanding of the concepts. The concepts in here are complex, so one has to delve into the intricacies of the course philosophy and all of our feedback; it takes time to figure everything out, internalize it and rewrite what it means on a form, so we have to learn to take our time and not regurgitate everything just to get it done.

    All that being said, I think it is clear that there is a population of students who understand the feedback mechanisms and who have spent the time getting to know the profiles and the available resources. This means that we can learn from each other and keep refining our skills as we move through first quarter, using the knowledge we gain from analyzing the results of surveys like this one.

    • First of all, I have to disagree with your first comment about the importance of grammar, spelling, and complete sentences on this quiz. Don’t get me wrong, those are the essential building blocks of english that every student should have mastered by now. However, if you look at the context of these responses, you would understand that students weren’t worried about editing what they wrote. It was obvious that the quiz was meant to test you on what you have learned, so students were more worried about the accuracy of their content. When it comes to essays or responses that Mr. Eure has us write on google classroom, I think it is necessary for students to be perfecting these assignments. However, due to the time limit and how informal the assignment was, it is understandable that some students did not have the time or felt the need to look back at their work. If you did not finish by the end of the class period, you might not have had any extra time spend on your responses. I believe that until every student understands the concepts of what the questions are asking, we should not be focused on simple syntax and spelling errors.
      However, I strongly agree with your next point. I found that especially with the question about the difference between third and fourth tier students and the cutting the stone post, many students simply copied and pasted. The first question about how feedback works was so broad and generalized that it was much harder for students that haven’t done the readings to answer. They couldn’t pick out one sentence in any of the posts that directly answered the question, because many different posts answered the question altogether. Many posts repeated key ideas that would help you answer the question if you understood them. I think that on future quizzes, there should be more general questions that students need to fully understand major concepts to answer correctly. Also, I found it interesting that there are many varying answers to the first question that can be considered correct.

      • I’d like to invite you and any other interested students to write the questions for the next one of these quizzes. What general questions do you think would work best to help you express what you know? I love the idea of you all designing that kind of diagnostic.

    • First, I would like to say that one of the most central themes of this course is feedback and interaction between students. Instead of calling them out, perhaps you could help teach them what you know about grammar and spelling. If you have such a deep understanding of the course and other students do not, shouldn’t you take some time to teach them instead of keeping it to yourself? Maybe you need to reread the course syllabus in order to internalize this information better.

    • I agree with most of what you had to say here except I disagree with the first part.

      Poor grammar isn’t really as much of an issue as the lack of knowledge about the course. The problem here is a lack of people who feel okay putting a blank answer, which is understandable. Maybe it’s just me, but I’ve had teachers in other classes literally say “never leave a blank answer! if you don’t know just guess!” And some literally refuse to take homework or a test with one blank response.

      However, if someone has a great understanding of the course, and isn’t so great with grammar and/or spelling, that’s not even nearly as much as an issue as someone who doesn’t understand the course at all. Someone with weak grammar skills who understands grade abatement would probably be a
      higher profile than someone who doesn’t understand the class but has good skills in grammar. And yes there are people who lack both an understanding of the course, and good spelling and grammar. But those people should be way more considered about learning the fundamentals of this course than learning the difference between a colon and a semicolon.

      You’re also assuming that those without perfect grammar didn’t check their responses like you did. For all you know they did check, maybe they fixed some mistakes and just didn’t catch them all. Maybe they aren’t as knowledgeable about the rules of correct grammar as you are. Does that mean they didn’t work as hard as you though? No, it doesn’t.

      Learn the class first, especially if you haven’t already. Worry about grammar and spelling after. And lastly, empathy is the most important skill in this class. You could have perfect grammar and spelling abilities but if you don’t practice empathy, none of that matters.

      • Just wanted to add that I appreciate you all commenting on my thoughts. I never meant to make the spelling thing about what students know, more about how much time they take to reflect on their work. Because I’ve seen errors that we can all fix- we all know what an incomplete sentence is and we all have decent enough spelling skills by eleventh grade to get through a quick Google form. I see what you are all saying about practicing empathy, taking the time to work with each other and understanding the course before criticizing others. Just know that this was never my purpose- I only meant to raise the point that it’s important to set aside a few moments to verify that you did the best work you could, because that is a part of the work ethic that makes anything possible.

  4. I like how Maria pointed out how there are lots of different but correct answers to the feedback question. I think this is because there are a lot of benefits to restructuring the feedback loop in this course, and students are all taking note of the benefits that will be more helpful to them. Some students might appreciate that the feedback in this class has more meaning than a number score; others might appreciate how students lead the feedback process. All in all, the system really does work for everyone.

    However, I wanted to comment on what you said about time pressure in relation to the quiz because I think that’s an interesting point. How long should the quiz have taken? Should students have been able to answer the questions quickly if they truly internalized the material? In my opinion, there is a kind of curve in the relationship between the knowledge students have and the time they take to respond. Students who don’t have in-depth knowledge will answer quickly. Students who know a lot will need to take a minute to sift through the information and construct their response because they can identify the complexities of some concepts. The students who take the longest to answer understand that there are a lot of moving parts in the system but haven’t processed it all yet, so it takes them a while to connect the dots in their answers to the form. I’d be interested in hearing everyone’s thoughts on this… how long should this quiz have taken and what kind of students take a short time or a long time?

    • That’s a smart observation about answering quickly. It has to do with the Dunning-Kruger effect, I think, and our tendency to rush most when we know least. But the more important aspect of what you’ve written here is that, yes, we probably need to rethink this kind of “quiz” as a timed artifact. I’m still stuck sometimes in the idea that assessments are only valid if they are controlled for honesty. Sending you all home for 24 hours to work on this would have been fine. We’ll do that next time.

    • I’d have prefered that the quiz stayed open until (at least) the end of the day, because all classes were taking it and a lot of thought should have been put into each question. It’s very obvious that many people either rushed the quiz to get it over with or paraphrased answers. I think that using resources is fine as long as the material is understood, but giving more time might allow someone to create a more structured response.
      I greatly dislike most tests and quizzes, especially in high-stress timed situations, because I usually have trouble completing the responses on time. I like to think and re-think my responses. Fast responders could also be confident in their answer, or, like you said, they may have rushed it. Slow responders, I think, put a lot of thought into their answers, and may be doing anything from overanalyzing the situation to trying to piece everything together to trying to remember the name of that one person who led the Greeks and conquered half of the known world. It would be nice to have low-stress quizzes that I can easily write well-thought answers on.

  5. I chuckled when I noticed that some people selected “Option 3.” On another note, I’m surprised at how honest most people were in terms of how far they read into their summer reading. More people claimed that they responded to the text than completed their texts. Also, reading the responses for feedback gives me a better idea as to what it truly is in this class — I had the idea, but I focused on the wrong concept. I guess I was too stuck up on the part that bothered me the most. I like being able to see how other people think of something without actually having to talk to them, but it does take some of the effort out of it.

  6. I don’t know what just went on here…. but after reading the text, I agree that students are trained to earn every point possible on every assignment no matter what it takes — whether it be cheating, lying, etc — and willingly (with a lack of guilt) do so. This especially takes place in the higher level/AP classes because these students are possessed with the idea that grades are everything, and grades will determine how they end up after high school. So, basically they will determine what college they go to. I think if external pressures weren’t so fierce, or someone of experience came and talked to students, reiterating points brought up by the course syllabus and Mr. Eure (learning life skills to be successful in the real world), we’d break the traditional bad habits of high school students and become less grade driven, but more willing to succeed through learning.

  7. I agree with most of what you had to say here except I disagree with the first part.

    Poor grammar isn’t really as much of an issue as the lack of knowledge about the course. The problem here is a lack of people who feel okay putting a blank answer, which is understandable. Maybe it’s just me, but I’ve had teachers in other classes literally say “never leave a blank answer! if you don’t know just guess!” And some literally refuse to take homework or a test with one blank response.

    However, if someone has a great understanding of the course, and isn’t so great with grammar and/or spelling, that’s not even nearly as much as an issue as someone who doesn’t understand the course at all. Someone with weak grammar skills who understands grade abatement would probably be a
    higher profile than someone who doesn’t understand the class but has good skills in grammar. And yes there are people who lack both an understanding of the course, and good spelling and grammar. But those people should be way more considered about learning the fundamentals of this course than learning the difference between a colon and a semicolon.

    You’re also assuming that those without perfect grammar didn’t check their responses like you did. For all you know they did check, maybe they fixed some mistakes and just didn’t catch them all. Maybe they aren’t as knowledgeable about the rules of correct grammar as you are. Does that mean they didn’t work as hard as you though? No, it doesn’t.

    Learn the class first, especially if you haven’t already. Worry about grammar and spelling after. And lastly, empathy is the most important skill in this class. You could have perfect grammar and spelling abilities but if you don’t practice empathy, none of that matters.

  8. However, I feel it is nearly impossible to reform this way of thinking by this generation of high school students. There were certain standards, precedents, and expectations of “a good student” that were set a long time ago and are now hard to unfollow or breakaway from by many of us. The students who understand that there is no “good” college (and that anyone could be the right one for them) or that it is more important to learn and truly understand concepts will be more successful and free flowing than those who rely on grades, positive feedback (only), and someone to tell them what to do and/or how to do it.

Start a discussion: