Topics
June 25, 2025
June 25, 2025
In the previous installation of this series, we discussed how to build critical thinking skills when using AI tools for research. In particular, understanding AI’s strengths and weaknesses is pivotal to how you use it. Ultimately, AI is another tool in your toolbox.
Once you’ve created the groundwork of building critical thinking skills around AI, it’s time to apply those skills to real-world use cases.
By applying critical thinking to AI tools in research, you can…
Critical thinking will help you navigate the complexities of data, making sure you can better differentiate between meaningful AI insights and noise. It empowers you to ask the right questions of your AI tools, to evaluate outputs critically, and integrate those insights into your decision making process.
Keep reading to learn how to enhance the depth and reliability of your user research and become a more user-centered product maker.
Colleen Pate is a Customer and Community Marketing Manager at Dscout.
Ilana Krause is a UX Researcher at Microsoft and Co-Author of Recalculating for Entrepreneurs.
This framework will help you build competencies needed to fully leverage AI in your user research and beyond.
Making sure you have well-defined objectives is the cornerstone of any successful research project. Start by asking yourself, your team, your stakeholders, your management team: What am I trying to achieve? What do we want to learn? What decisions are we making?
Use metacognition (thinking about how you are thinking for problem solving to critically assess your thought process) and make sure that you can gain an unbiased perspective from the user to answer your research questions. A helpful part of this process is to set specific goals.
Let’s say you want to identify key pain points within a user onboarding process. This clarity will help you focus your AI analysis on the most relevant data. It's something that you can begin prompting different AI tools and models to help gain clarity on analyzing the data and preparing relevant research artifacts.
In defining clear research objectives, it's also important to formulate precise and targeted questions that align with your objectives. Maybe you want to know which steps in the onboarding process see the highest dropout rates, or what feedback are users giving about these steps.
By defining clear research objectives and asking targeted questions, you'll ground yourself. A solid foundation for your research process ensures your efforts are directed—making it easier to leverage AI tools effectively and gain meaningful insights.
Once you’ve defined your research objectives, the next step is to choose the right tools and methods to achieve these objectives. Selecting the right tool is important for effective data gathering analysis.
Let’s say you're thinking about a tool like natural language processing to analyze open-ended survey responses, or behavioral analytics to track user interactions.
It’s important to match the right tool to the task to make sure that you have relevant analysis opportunities. Also consider combining AI with traditional methods for data analysis and using AI to complement it.
If you're using AI to identify common user complaints in a large database, you can then conduct interviews to explore these issues in greater depth. It's an opportunity to thoughtfully choose the right tool and method in order to maximize your effectiveness as a researcher.
Analyze and cross-verify insights to ensure the insights are accurate and representative of your user base. You can achieve this by using AI tools to identify patterns and trends within your data. AI is helpful for looking at a lot of data and identifying overarching themes.
Remember to look beyond raw data by understanding user context and considering external factors that could influence behaviors. AI is a tool. You have the context, the knowledge, and the critical thinking skills to utilize it to maximum capacity. Use triangulation by cross-referencing AI findings with other research methods.
Prior research may have similar or related insights. Use AI to help pinpoint areas of focus or key themes, but don't rely on it solely. These themes may guide you to identify key issues.
Create a balanced approach to make sure you’re getting meaningful insights from your research. You might already have a current flow with how you go through the research process. This is an opportunity to think critically about how you're integrating these AI tools with traditional methods to take your research to the next level.
“AI is a tool. You have the context, the knowledge, and the critical thinking skills to utilize it to maximum capacity.”
Ilana Krause
UX Researcher, Microsoft and Co-Author of Recalculating for Entrepreneurs
Iterating and refining your approach continually enhances the effectiveness of your research and our insights. Apply ongoing refinement based on what you're learning. This allows you to continuously adjust your AI prompts and queries to target the most relevant data, and dive deeper into certain insight areas.
If you feel like initial analysis is too broad, use AI tools to narrow the focus to specific aspects. Conduct more iterative testing by regularly evaluating and refining your hypotheses. If AI identifies a particular pain point within a research study you're running, Product can make changes to address it.
Use AI tools to quickly re-evaluate in a more iterative way and see if the issue persists or improves. This cyclical process of testing and refinement already happens within product making—it's about using AI to intercept and accelerate critical moments of that process.
Thinking intentionally about how you iterate and refine approaches to AI tools helps you…
AI-driven techniques empower researchers to make data-informed decisions and drive innovations. Let’s talk through some real examples about how I'm using AI to transform my user research process.
AI can streamline the creation of research artifacts—whether it be research proposals or discussion guide outlines. You can leverage AI tools to draft initial versions of these documents quickly. Identifying key themes and questions from existing documentation helps accelerate the preparation phase, allowing you to socialize this information with stakeholders, early on with a quick turnaround.
When it comes to analyzing user data, I find that AI tools excel at uncovering bigger picture insight themes. These tools can process large volumes of data to help identify patterns and trends that might be missed through manual analysis.
When analyzing user data with AI tools, I go through my own manual analysis process of the data, then triangulate with different AI tools.
I also use AI a lot with synthesizing and refining my storytelling process. This helps to…
Remember to stay grounded and keep your critical thinking sharp, especially when using these AI tools. Use these grounding questions to critically think about how you're using AI tools for reliable insights:
At the end of the day, AI can help augment your capabilities, but it's your critical thinking that drives impactful research. You make the difference. Your ability to analyze, interpret, and use data thoughtfully leads to successful outcomes. AI can provide us with valuable information as a tool, but it's you who brings the depth, the context, and the action on these insights.
It's important to empower yourself with these critical thinking skills. You are the superhero of your research.
