Topics
June 16, 2025
June 16, 2025
This article is the first in a two part series, adapted from the webinar “Building Critical Thinking Skills for AI-Driven User Research.”
We all know that AI has the potential to be a real game changer in our research and our work, like a superhero power—but sometimes it can feel like our kryptonite.
Let’s dive into how we can turn AI into our ally by…
Whether you're a designer, data scientist, or manager, this content applies to everyone—regardless of your role. It's all about building critical thinking skills.
Colleen Pate is a Customer and Community Marketing Manager at Dscout.
Ilana Krause is a UX Researcher at Microsoft and Co-Author of Recalculating for Entrepreneurs.
There are a lot of ways people are currently using AI:
Overreliance on AI can lead to unintended consequences—whether it be perceived laziness or misinformation spreading. But it can also provide efficiency and work productivity.
One thing AI can't replace is the nuanced insights and ideas that come from engaged thinking. Recent studies have shown a significant correlation between frequent AI tool usage and a decline in critical thinking abilities. These findings underscore the importance of a balanced AI integration into our lives.
We know that AI can handle repetitive and time consuming tasks, but it's also our responsibility to ensure that we’re actively engaging with AI when it comes to work. There is also some fear and a perception of the potential of AI becoming a Terminator-like character that replaces human roles.
Concerns about AI can't be ignored, but it's important to remember that AI is a tool. The real risk lies in how we choose to use it.
AI has been a transformational tool that helps me perform significantly better as a user researcher. This tool has helped not only from a work efficiency standpoint or an outcome standpoint, but in offering a depth of user insights—which ultimately impacts user experience and delight.
“Concerns about AI can't be ignored, but it's important to remember that AI is a tool. The real risk lies in how we choose to use it.”
Ilana Krause
UX Researcher, Microsoft and Co-Author of Recalculating for Entrepreneurs
Using AI specifically in user research presents significant challenges, such as…
Be strategic about how you're using AI tools to ensure the inputs are high quality. While it’s easy to discuss the challenges of using AI in research, it's important to frame this as an opportunity.
AI can help us…
It's important to strike a balance between leveraging AI to enhance your workflows and processes while also critically validating its output. Being clear about the expectations and intentional with your approach can help boost the efficacy of our efforts—like applying critical thinking skills.
Let’s start with some higher level principles that we should keep in mind when building habits with AI tools, analyzing user data, and informing business decisions.
In order to develop critical thinking skills about your use of AI in research, build awareness of the value that you add to your workstreams through metacognition and self reflection. In other words: regularly reflect on your thought processes when using AI tools.
Recognize the unique value that you bring to the table. Create a habit of asking yourself questions like…
Acknowledge that you do have expertise to help synthesize and contextualize AI-generated outputs.
Using AI tools can help you make informed decisions about what experiments and research you're running, because you understand your strengths and can use AI to outsource and supplement the additional support and consultation needed. By building awareness of your critical value, you can empower yourself to leverage AI more effectively and drive impactful outcomes.
While AI is powerful, it is not perfect. Always double check the outputs generated by AI and make sure what you're using is accurate. It's important to maintain strong analytical skills so you can verify and validate data independently.
AI can be prone to errors and biases, whether inherently or even how we're prompting and interacting with the models. It’s important to not only understand how AI models work, but also feel comfortable conducting some of these processes yourself.
Manually conducting processes and digging into how the models work can help you catch inaccuracies and make more informed decisions. That way, AI enhances your research, but is not the end-all-be-all.
Approach AI as a collaborative partner that enhances your work rather than something to intimidate you. I am betting that those who master AI will rise to the top. Believe in your ability to learn, grow, and evolve within the AI landscape. View challenges as opportunities to grow and learn.
Adopt this growth mindset so that you're better prepared to harness AI in your work to take outcomes to the next level.
Understanding the strengths and weaknesses of AI tools is crucial for effective application. Different types of research inherently need different tools, each with different strengths and limits.
For example, if you're thinking of using qualitative tools like interviews and a focus group, that provides in-depth insights—but then you would use quantitative tools like surveys or A/B testing to identify patterns across larger populations.
Use a similar breakdown in mindset when considering different AI tools. AI-driven analytics like natural language processing or machine learning can quickly process a high volume of data to uncover patterns.
Knowing these strengths and weaknesses help you to make informed decisions about the tools that you're using. Also acknowledge the limitations and the potential biases that come with these tools. Doing so can help you take charge of your research process and make the best critical decisions.
Like all good research, take a balanced approach. Combining different AI tools optimizes the research process and creates more reliable and actionable insights.
At the end of the day, AI is just a tool. It's a powerful one, but its effectiveness depends on how we choose to use it. AI can analyze a ton of data, help identify patterns, and generate insights—but it also requires our judgment and expertise to interpret and ultimately act on these findings.
We're the decision makers guiding AI to help and want to make sure it aligns with our goals, values, and business metrics. By leveraging AI strengths and acknowledging its limitations, we can optimize that process. It's our creativity, intuition, and critical thinking that drive meaningful outcomes.
