Update: New way for skills to count as "Practiced"

Looking good! On Firefox, it doesn't seem to show up at all (I got all 7 questions right):

As for Chrome, it looks totally fine:

EDIT:

This occured on an older Firefox version.

I love the look of these exercises, the backgrounds are quite appealing and visual data displayed in little animations is always fun!

As for the new way skills level up to practiced, I personally preferred N-in-a-row so I could really work hard to get those N right. Now it's way less pressure if I miss one out of N, I think there should be an option or perhaps make the system recognize if you need N in a row or 70% right.

Personally, as parent/coach, I hate this change. The do N in a row was a great way for my students to self-check their understanding of a particular task. Things they got right away they only needed to do a couple of problems on; things that were harder took more problems. Not having to slog through X problems just to meet someone else's arbitrary threshold was a huge selling point. '

And, the many times they got the Nth question wrong and had to do a whole new set taught them to slow down and be extra careful in a way that all my reminders could not.

I also prefer the N in a row so I would like an option for that.

@Caroline and Elise: we haven't changed the requirement to move passed Practiced to additional mastery levels (via those skills which have corresponding Missions).  It would be really helpful to hear more about how this change in the Practice experience is negatively impacting your students.  (If this distinction is confusing at all, don't worry you're not the only one :-p)

Caroline, I know you already explained a lot of your use case, but I would love to hear more about how you use the "Practice" mechanic in such a way that having the currently mastery requirements isn't an adequate way for students to demonstrate their understanding.  Thank you so much for the feedback - it really is helpful for the team.

Hi -- I'm trying out this new feature in different ways and found what might be a bug -- I intentionally got four in a row wrong, but the skill changed to "Practiced" anyway:

I think it's great.  My students felt a lot of stress with the "n in a row" model.  This new model relieves a lot of that stress.  It would be great if teachers could change the percentage and also change the total number of problems in a practice set.  I noticed also that on the students' assignment lists, it now shows their percentage for each assignment and gives them an option to "redo for a better score".  I think this is fantastic - good job with that!

It's good to know this change only affects skills that "need practice" and are under the practiced level, a little of both are mixed together and that's really nice.

In reference to this excerpt from:I am not a coach, nor do I have one, but I agree with it and would encourage Khan Academy to do so. It is hard to use the same grading form for everyone, since everyone is different. But coaches know their individual students, and can therefore adjust the grading system to help their students obtain the maximum learning possible

Love the color and look and the new scoring system.

I really dislike this new update. The N in a row system encouraged precision, focus, hard work, and an overall solid understanding of the topic. Now, from the point of view as a student, I feel as though I will be able to level up in a topic without having a proper understanding of the topic, which is not a good thing. Please change the system back to the way it was. The N in a row system was much better.

Incase anyone missed it, this change will only affect how users level up from "not practiced" to "practiced". The rest of the mastery system still requires the same precision, concentration, and 100% mastery of the skill to level up to higher levels or "mastered". This is a good thing; changing the whole system to "do-N" is definitely not what we would do, levels above "practiced" will still use "do-N-in-a-row".

Jett Burns, and Laurie, you have both said that this change only affects leveling up from "needs practice" to "practiced," and that higher levels still need N in a row.  

But the higher levels only come from mastery challenges, which are never multiple problems in a row; it's a pretty linear system of they either get one problem right and move up a level or get it wrong and move down. So, it might take them several mastery challenges to move up a level, but in each one, they're only doing one problem in that topic at a time mixed in with other things. I'm not knocking that part of the system at all, it's a great way to test their mastery!

But it doesn't help the current problem. Which is that when they are presented with tasks to practice in their dashboard, they have to work on N problems regardless of whether it's a task they find easy or difficult. It takes away the self-checking mechanism built into N in a row. My kids were always really proud of those "persistence" badges they got when they finally finished a task that might have taken them 30 or more problems before they finally got it.

Higher level tasks aren't on their dashboard, and if we manually add one back, it's still "Do N" regardless of what level task it is.

In my opinion this is a change for the worse. In terms of the gamification KA applies to its exercises, this servers only to frustrate dedicated users while at the same time making others care even less about exercise progress. Let me explain why.

If a student cares about achieving "Practiced" status on the exercise, and they fail one of the four attempts, then this mechanic makes it so the student should skip the rest of the exercises and start over, where previously they should just keep going.

In other words this change wastes the time of the most dedicated students, while encouraging others to not care about the exercises they fail. What's the supposed upside again?

Why should I waste my time doing a bunch of exercises that don't count toward my progress?

I just wanted to add my feedback I agree with Jett , that being said I see others feedback unfortunately from students within Khan Academy often times I would hear frustrations of the old system which this new system it makes it feel a little more laid-back and provides less frustration for some learner's. Having feedback is quite helpful on both sides with more feedback the system can be improved for everyone I am sure the team appreciates​ it.

Here are my perspectives:

As a student: I use KA to review and to practice attention to detail by having to do all the practice problems in a row. I use to reinforce self-discipline. 

As a professor/tutor/parent/coach:  This is partially effective only if the student cares about doing the Mastery Challenges (which is one type of each problem) or they practice the lesson again. There is no way to enforce these options unless you assign grades per lesson, which does not work for tutoring or audit classes.

Also, the option to "recommend" lessons with n-in-a-row requirements is so inefficient as to be completely unrealistic use of a teachers time.

"The only way to learn mathematics is to do mathematics." 
Paul Halmos

I've used Khan Academy extensively this year for summer school.  One class of failed Math 1 students (9th grade) and one class of Pre-AP Calculus summer bridge students.  So these are opposite ends of the spectrum.  Students worked on the site through the transition to the new "practiced" scoring system.  I did not see any negative outcomes.  I did not see any students take it less seriously, as other posts have suggested.  It definitely reduced the stress and frustration of getting 4 right in a row and then worrying so much about getting that last one right.  While I understand some comments about reducing the persistence level, I don't necessarily see that as a bad thing.  Students can get ranked as "practiced" and then move on to another exercise.  They can come back to the previous assignment to better their score if they did not get 100%.  The teacher can see all of this very clearly and grade accordingly.  It's still up to the teacher to decide how to grade this.  In order to get full credit, I can require that students re-practice an assignment until they get 100%.  But they don't have to get bored and frustrated with one assignment because they make a small mistake.  Come back to it later if you only passed with 75%!  This is better for retention anyway.  I would rather my students practice other skills and then circle back to any that they did not pass with 100%.  I love the change, and think it will help both with students' attitudes toward Khan Academy and with overall retention of the material.  My only request is to increase the number of problems in a practice set (I don't think 4 or 5 is enough), or let teachers pick how many problems are in an assignment - that would be even better.  I would rather be able to pick the number of problems than the percentage required for "practiced".  

I really like the old n-in-a-row method.  Many students do get frustrated by it, but I feel like it teaches a really important lesson about practicing something until you understand it well enough to actually do it correctly. However, as long as you don't change the requirements for moving up to mastery I suppose it won't hurt too much.

I feel like this is Khan Academy's version of changing the flavor of the Coca Cola. I just don't like it.

I love this change and would LOVE to be able to adjust the proficiency percentage to differentiate.

Personally, I think this is a terrible change, although allowing coaches to switch the requirement to 100% would allow us to mimic the old behaviour. Personally, I think that the default should be 100% and teachers can shift it down if their students have specific requirements.

I teach years 8-11 using Khan Academy as a skills practice and diagnostic tool. The n-in-a-row was a core part of the original Khan Academy design. The basic idea is that, if you properly understand a skill then you can get every question correct without trouble. If you can't get every question correct then you need to do more practice. It's tough (and, at times, confronting) but that is the point. Dropping the requirement to 70% allows students to rush through without proper understanding:

•  For multiple-choice questions (common within Khan Academy exercises), pure luck can give 70% correct with relatively few random guesses.
• Some exercises have a few different types of question within them. With n-in-a-row, students needed to be able to correctly answer all types in order to achieve practiced level. With 70% requirement, students can completely skip understanding one or two of the question types in an exercise and still "pass"
• n-in-a-row forces students to self-calibrate how fast they can work without making silly errors. It is far better to do this with such a low consequence of failure compared to a formal written test.
• All that will happen with the 70% change is that students will move on to later material without a solid understanding of prerequisite skills and find themselves again frustrated by their inability to achieve the even lower threshold of 70% but with less ability to actually do anything about it. This is the exact problem that Khan Academy was designed to solve.

The mastery challenges are not a relevant substitute for the n-in-a-row requirement in practice exercises:

• Only practice exercises allow deliberate targeting of specific skills. Mastery challenges are weighted towards recommendations (and presumably the new assignments) but this is still a matter of chance. As a result, the only expectation I can have of my students is that they achieve "practiced" level for the required skills.
• Mastery challenges require (relatively low) transfer of skills by mixing up the questions and asking them out of a familiar sequence. This is an extra level of challenge to students who achieve the n-in-a-row level for practiced. For those who barely achieve 70%, the step up will be simply too large.
• Mastery challenges give only a single question at a time from a a skill as it is not intended to provide practice, but confirmation of existing knowledge. This means that students who were only able to achieve 70% in practice will get a higher proportion of mastery challenge questions incorrect without knowing why and without really being able to do anything about it. This is particularly true for skills where students have trouble without only one form of the question.

If students are too stressed by the fear of getting a question wrong in the low-stakes world of Khan Academy then this is a pedagogy problem rather than a problem with the tool. The n-in-a-row requirement has the worst cast scenario for students being that they need to do a few more practice questions. This is annoying for them but hardly life-changing. As professional teachers, it is a large part of our job to help students develop a growth mindset towards mathematics and understand the purpose of practice and mastery.

You have undone the thing that made Khan Academy so great.

When my students complained that they were getting sent back for getting one wrong out of five, I explained to them that it meant there was something they didn't know how to do. That was the signal for them to watch a video or ask a question.

Many of my students were pushing through exercises like playing a slot machine... sooner or later, luck would see me through. That was exactly what I was trying to teach them to avoid.

Getting 70% right means the student is still getting 30% wrong. Originally, the goal was to make sure the student mastered each skill before moving on to the next. Now, you have succumbed to the feel-good-about-yourself mentality that lets kids be satisfied with "trying" instead of "learning."

I am so disappointed by your "new and improved" Khan Academy that I have already started looking for something else to use in my class, after nearly 10 years of being a vocal advocate of this site.

What a waste.

John, I really appreciate your comment.

Dear KA, please stop being so teacher un-friendly. I am willing to learn and adapt to this change if I have options available. The other day I was sitting next to a student who had done several problems (more than the four stated) and finally got three out of the last four correct. The lesson then told the student to get two more correct to finish. Interesting. If more of that type of practice occurs I will be fine with it.

However, with the new teacher made "assignment" options I am very disappointed that I lost the option to require n-in-a-row. KA should allow me to make value-of-content decisions.  We teach in and for different circumstances and should be allowed to be adaptable.  Not all classes are of the public education type.

There seems to be quite a bit of negative reaction to this change, but I think a lot of people are not seeing the flexibility that this adds for teachers.  Just because Khan Academy puts a student at "practiced" for getting 75% right, that doesn't mean it's the end of the story.  It is very easy for the teacher to see all of the percentages and require students to redo assignments until they get 100%.  It actually gives the teacher more control to differentiate for students.  I like this better than n-in-a-row.  With n-in-a-row, once a student got that 5th one right, it was over.  They wouldn't see it again until a mastery challenge.  With the new system, they can get a good feel for the problems (75%), move on to other problems, and then come back (because the teacher requires it) and try to better their score.  The spiraling that this creates is better for retention than forcing the n-in-a-row one time through and then not seeing it again.  For those who seem to be upset over this change, please give it a try with students and see the flexibility that it offers, before making a final judgement.  

Sorry John but legitimate concerns about loss of functionality cannot be simply dismissed as "negative reaction"

The positive spin you suggest is only possible if teachers "require" extra work of their students outside the Khan Academy system. While possible, this is evidence that the system itself is broken and requires work-arounds. It's also no more true now than it was before the changes. A well-functioning system would retain the functionality internally.

The previous recommendation mechanism allowed for the flexibility (teachers could always assign fewer skills in a row if their students needed this) but the assignment mechanic contains no flexibility at all. If anything, the assignment system is more tedious for students because it requires them to do a larger, fixed number of questions regardless of whether or not they are answering correctly.

The idea that the new system encourages "spiralling" has no basis in fact. The new 75% practiced requirement is just as final as the old "n in a row" Teachers always had scope to get their students to 

Your claim that teachers can "easily see the percentages" is technically only true of individual students for individual skills. There are no longer any overall reporting tools that make this data readily available at a class level. With the >200 students that I coach through KA, it's simply not feasible to find and address the % of every skill for every student.

Sorry, but the benefits that you claim simply don't exist (any more than they existed before the changes) and the issues identified by others are real and important.

 i personally like it. i hated that when i got 4 questions in a row right, then i made a careless mistake and ohh...i had to start over.

Isaiah, nobody is suggesting that student don't like it. Most people like success to be easier.

The concern here is that the changes are worse for learning. You might not have liked doing another 5 practice questions but it was undoubtedly good for your learning to do them and ensure that you could reliably perform the skill before moving on.

i think not having to get them n-in-a-row and having to get a certain percentage is good, because it shows you have a grasp and understanding of the concept. it allows careless mistakes to be overlooked while still having to get most of them right.

Thanks for your insights, Chris.  The reporting tools, to me is a separate issue which I completely agree has lost some of its functionality and robustness.  I personally very much miss the grid view and the ability to create and bookmark a filtered set of skills.  You can currently, however, see the results for all students on a particular assignment.  I am only speaking from experience with use of the new system over the summer.  Students who do not achieve 100%, when they look at their assignment list (scroll down to completed assignments) can see each of their scores and are prompted to "repeat the exercise to get a better score".  I very much like this approach.  It encourages students to take ownership of their learning.  Most of my students repeated assignments without any prompting from me, in order to get that 100%.  For me, as a math teacher, part of my philosophy is that the learning should be enjoyable for students.  I've had many students in the past express a lot of frustration with the n-in-a-row method. It turns them off to K.A. and it turns them off to math in general.  To me, that is anti-productive and personally saddening.  Perhaps you have had a totally different experience with your students.

I'm not sure what you mean by students "having to complete work outside the Khan Academy system" in order to achieve the flexibility I was talking about.  Students just look at their list and click on any that they do not have at 100%.  They redo those until they get 100%.  So, yes, it is all contained within the system.  And yes, it creates a beneficial spiraling effect that I have already seen with my own eyes.  Give it a chance.

Again, the analysis tools is a different issue and I agree it needs work.

Hi Everyone,

I love hearing the feedback from both sides I just wanted to say the Khan Academy Zendesk community is always happy to have content and feature requests. Perhaps there are things you would like added to the new coach dashboard,Mastery etc. you can open in a new post and request please make sure to post under Feature Requests and General Site Feedback.