Should I tell students not to do Mastery Challenges?

Hello I am Eddie Whalen I will help you. So mastery challenges as you said are to find the skill level but also to show mastery in a skill. For example if they practice some skill and get a level two in that type of problem then they are give a mastery challenge so they can master it and if they get it wrong they might be given another chance at a mastery challenge.

So no don't tell your students not to do mastery challenges

Use mastery challenges!!! They are an important tool for using your computer time to efficiently address students' needs.

Khan Academy uses the research based practice of distributed learning.  Distributed learning theory says that small amounts of practice over time with increasingly long stretches between practices are a good way to learn. On KA, students have an opportunity to watch the video, get hints, and work on problems until they become proficient--for the moment. They continue to see those same topics over time to insure retention and generalization.

Once they have gotten 5 in a row correct and moved from needs practice to level 1, they will see a question of that type on a mastery challenge very soon. If they get that question right, they move up a level and it will be a bit longer until they see that type of question again.  A wrong answer and they move down a level. Even once a skill is mastered, it will occasionally show up in a mastery challenge to ensure that it is STILL mastered. 

Mastery challenges serve to ensure that students have understood not only how to complete a skill but when to use it. It is one thing to learn a pattern such as cross multiplication and repeat the pattern 5 times in a row. It is another thing entirely to see a problem requiring cross multiplication out of context, remember that's what needs to be done and how to do it. With each success, the time between review is longer until it is considered to be mastered. 

Students, especially when they are new to KA and have practiced very few skills, will also see new content on mastery challenges. In this way, the process of going through all the skills to help level a student and suggest new topics is expedited. If they are given a problem they have not seen on KA out of context and get it correct, it is assumed that is a skill they know already and it will move up, sometimes to mastery but I don't think every time. Remember though, even mastered skills are reviewed through mastery challenges and may move back down.