Conversation AI such as ChatGPT is a disruptive technology which will alter the educational landscape. The present article used a self-study research design to evaluate commonly used curriculum-based assessments (CBAs) by counselor educators in light of ChatGPT and other artificial intelligence-based chatbots. The authors propose that these assessments should be modified to better support student learning in the context of AI technology. Moreover, the authors provide recommendations for counselor educators on how to integrate AI into their teaching practices to improve counselors-in-training classroom engagement and learning outcomes. Authors also discussed the current limitations of ChatGPT and other AI-based chatbots.