The recent panel discussion titled “A Genuine Conversation about the Future of UX Research” drew significant attention within the user research community, reflecting the growing debate on the democratization of UX research. The discourse surrounding this topic is both crucial and contentious. Approximately 400 researchers and interested parties attended the panel discussion, with just about one quarter of them engaging in concurrent chat in Zoom messages.

Contextualizing the Debate

The debate on the democratization of UX research has gained momentum, spurred by Teresa Torres’ advocacy for continuous discovery in her book “Continuous Discovery Habits: Discover Products That Create Customer Value and Business Value.” While Torres’ approach has sparked enthusiasm for involving non-researchers in research activities, it has also ignited concerns about the potential risks to research quality and professional roles within the field.

The following is an overview of the panel discussion, my analysis and key takeaways of the major themes that emerged from the discussion panel itself, and the extremely vibrant and active chat thread that took place in Zoom messages during the recording.

Opening and Introductions

Lou Rosenfeld opened the session with a brief note on Rosenfeld Media’s organizational mission to look ahead and advance any area of user experience design. He acknowledged that this is a particularly difficult time for user experience professionals (researchers, designers, managers, etc.) and the importance of charting a path forward together.

Before we go further, for folks who are unclear on the controversy surrounding this topic, I will grossly oversimplify it into two basic positions:

Pro-Democratized Research

Support for non-researchers in digital product and service teams to conduct research to validate product direction and design decisions through training product managers and the potential for their product team members (managers, engineers, and other non-UX professionals) to do continuous discovery work. It should be noted that this trend is and has been happening for some time and doesn’t show signs of going away.  So the question isn’t “do you support this or not?” but how do we support it in a way that becomes a valuable contribution to applied research in digital product teams going forward?

Critics of Democratized Research

Voice concern for the degradation of the practice, calling for accountability and quality in research, citing the risk of insights being lost or misinterpreted in a democratized landscape fraught with confirmation bias and assumptions without the spirit of objectivity and methods for bias mitigation. Chiefly, these critics have also seen their roles eliminated on teams that have taken up the continuous discovery model without needing research professionals. So, the community has a natural animus surrounding this topic and its proponents.

Introductions

Peter Merholz Co-Author Org Design for Design Orgs: Anthropology Degree: “Understanding humans is deeply important to me, and I have long felt that UX research is the foundation upon which literally all of our practices rest.” He expressed his enthusiasm for helping take part in this event while this practice is in flux and in need of reflection.

Dr. Jamaica Burge, “J-Boogie”: Head of Enterprise Product Research at Capital One, leading a team of approximately 25 people including research operations. As an advocate of democratizing research within the organization, Jamaica spoke to the benefit of cross-fertilization of ideas and participation while stating the need for quality control to execute with confidence and skill.

Noam Segal, Ph.D. in Psychology: A self-proclaimed “techno-optimist,” Noam is excited about the possibility of AI enhancing continuous discovery and research. Noam acknowledges that in technology, there is and will always be constant change, but that human nature and all of its complexity remain constant and merit human understanding to build and serve in that context.

Teresa Torres, Consultant “Product Discovery Coach”: She introduced herself by clarifying her core beliefs: “It’s really easy for organizations to give lip service to customer centricity when actions say otherwise. If we’re truly going to create a customer-centric organization, every single person in the organization needs firsthand exposure to customers.” She acknowledged that right or wrong, the democratization of UX research has been happening for 20 years and it is only going to continue. “Even when product teams engage in continuous discovery, there will always be a role for user researchers… we need both.”

Robert Fabricant: Based on Robert’s early career in criminal justice, he pointed out that “The question of understanding people and making experiences work for people go way beyond product.” He went on to point out that what people see today in the market is one of many cycles of buildup that can lead to functional tension. “Where do we get our confidence that what leads us to believe that there will be some integrity across companies, particularly big tech companies, around the fundamental importance of understanding people deeply and of the insights and learning and context that brings to steer both good and ethical decisions”

Quoted from comments in chat (alongside the panel discussion)

“UX Theater is alive and well.”  – Kyle Soucy ( 11 )

Kyle Soucy, a Research Consultant @ Usable Interface who attended the session, posted this comment in the Zoom chat related to her article. “The Problem With Research Democratization The People Aren’t Talking About”

an d image of dispassionate people at a meeting

“Just because non-researchers (i.e. designers, developers, product managers, marketers, etc.) can learn to conduct research, it doesn’t mean they want to.” – Source: Kyle Soucy

Sifting the tea leaves of the discussion

From this point forward, rather than chronicling the blow-by-blow of the panel discussion, I’m going to call out the key themes of the discussion. You’ll be better off watching the full video to dig into the more nuanced discussion points.

Key Opportunity Themes

Show, don’t tell

What you spend your time doing is more important than what your title is. As the practice of UX research matures, it’s not only increasingly important to show the value of the work, it’s about helping partners understand what insights are and what to do with it.

Research of all Stripes

There are many types of UX research, but we need to be more specific about the types of research that can/should/will be democratized when discussing research skills. As we responsibly share the skills, tasks, and research models with the product team, we need to be specific about the types of research best suited for nonresearchers and what quality control and outputs look like.

  • Address concern for quality, integrity, as well as bias and assumption mitigation as a practice for continuous and distributed research in product teams
  • Standards for planning, recruitment, methods, protocols, and reporting need to be set for quality at every stripe of research

The intent of democratized research

  • The opportunity in democratized research as it relates to specialized researchers is to delegate routine tactical usability testing to focus more on strategic longitudinal and strategic research
  • The value of cross-pollination between product, engineering, and UX skill areas is that it elevates the perception of human-centered systems thinking into the product life cycle
  • The word “democratization” is a misnomer as the intent is for a collective evolution of responsibilities for all roles towards a more mature human-centered organization
  • The vision supporting distributed research in digital product teams was intended as a non-zero-sum scenario. The impact of distributing the responsibility of discovery and exposure to research outcomes to more roles in the organization should increase the depth of questions, which, in turn, drives the need to hire more skilled and specialized researchers (win-win – in theory, but the rationale is sound)
    While it remains to be seen if this is true, this has been the experience for at least one panel member in a way that it hasn’t been the case for Product-First “top-down command and control just-build-these-features companies”

Mitigation themes in the future of UX research

Establishing the quality bar

There was general acknowledgment of the need to establish a “quality bar” so digital product and service teams can normalize standards for good research.

  • It was pointed out by a panelist that product performance is the ultimate indicator of whether or research is effective. However, this was rebutted as a lagging indicator vs. the intent of applied research in digital product teams that seek generative leading indicators to drive innovation and strategic direction.
  • Technology research does not go through the same rigor as academic and scientific research with an Institutional Review Board (IRB). While it wasn’t mentioned in the panel – the concurrent community chat discussed the need for an equivalent body to support UXR.
  • Steve Portigal pointed out that the QRCA exists, but it’s more for consultants and more broadly focused than UXR.
  • In the meantime, it would be up to ResearchOps within each org to set standards for practice and quality in research management

Distributed research demands rigor

The framing of democratized research needs to be replaced with a conversation about ResearchOps with an eye on institutionalized UXR Maturity. To support that transformation, proponents for distributed research for continuous discovery brought up the following:

  • Training programs for distributed research need to be orchestrated with guard rails to support democratization and understanding of the value of research maturity, which include but should not be limited to:
    • A holistic overview of UX research diaspora – research operations, methods, practice, and general context and which subset of that arena falls into decentralized research
    • Methods and practices for mitigating bias and assumption confirmation
    • Standards of practice in planning, recruiting, methods and protocols, insight management, and reporting
    • General concern for quality and accountability
  • General research maturity management to provide the proper context for the types of approaches that are realistically distributable and valuable for continuous discovery vs. generative, exploratory, and innovation-oriented strategic research programs
Quoted from comments in chat (alongside the panel discussion)

“We can’t democratize without guardrails, imo”  - Kathy Mirescu

UXR Maturity x Change Leadership

One of the panelists mentioned that context matters and that organizations can be uniquely messy in a way that necessitates a snowflake approach to maturity. This can be true, but there are universal truths in research that can serve as a foundation for maturity framing no matter how individually custom that path looks for an individual org. This is evidenced by over a quarter of a century in applied research for digital products and services that can be practiced from organization to organization, leveraging roughly the same methods, protocols, processes, tools, and intent for analyzing and providing insights for human factors in technology contexts.

All parties agreed that change management has a huge role in supporting research maturity transformation, but they also acknowledged that many UXR teams are not prepared to enable/support the process successfully.

  • Change leadership is a discipline in its own right, and it takes as much same care and needs for focus and attention as user research
  • The question of responsibility for enabling both maturity and skills distribution came up short but presumably falls into ResearchOps management
  • All agreed that the key is to set clear goals for the change you want to realize in your organization and then empower change management necessary to realize the change – think Maria Giudice and Christopher Ireland’s Changemakers.

“Democratizing research feels like a full time teaching position on top of the job of research””  - Kerry Gould (💯 25)

Beware of a Scarcity Mindset in UXR

“If you talk to a customer, then there’s nothing left for me to do”
“I think that’s a fundamental flaw. There is always more work to be done than we will ever be able to do, always 100% of the time.” – Teresa Torres

Chatter during the panel discussion

Roughly 100 out of the 400 attendees contributed to a vibrant Zoom chat during the panel discussion, either through direct comments, replies, or emojis.

Given the current climate in the UXR community, where there was a spectrum of both in-house and consultant researchers who have been negatively impacted by the current economic climate, there was sure to be more charged discussion in chat. Fortunately, the Rosenfeld community code of conduct was in place to help keep things from getting out of hand.

The overall sentiment in the discussion thread was mixed, reflecting a range of emotions from frustration and burnout to optimism and inspiration. Some participants expressed frustration with the current state of UX research and skepticism about AI’s impact on the field, while others appeared to be more optimistic and eager to embrace new approaches.

The key debates in chat centered on the fundamental premise of democratized research and the need for accountability and quality in a world where continuously distributed discovery is a part of research maturity modeling in digital teams.
Key Arguments and Positions:

Democratization of UX Research

  • There was a divide among participants regarding the democratization of UX research. Some argued that democratization is necessary for progress and innovation in the field, while others expressed concern about its potential negative effects on research quality and the professional status of UX researchers.
  • Proponents of democratization emphasized the need to make research more accessible and inclusive, while critics raised questions about maintaining research quality and expertise.

Research Quality and Accountability

  • Participants discussed the importance of research quality and accountability. Some emphasized the need to ensure that research is effectively applied within organizations and that insights are not lost or ignored.
  • There are definite and valid concerns about how research is practiced and applied, with suggestions for improving processes and holding stakeholders accountable for utilizing research insights.

Clarifying the Crisis in UXR

An attendee asked in chat for clarification on the crisis in UX research, and it was found that not all organizations are seeing the same symptoms. But for those who are paying attention and being impacted, here are some of the impacts cited:

  • Massive layoffs, ongoing
  • Entire teams dismissed
  • Lots of soul-searching “Is this a career, a field, an industry anymore?”
  • Disproportionate layoffs of UXRs
  • Thinking about the value of UXR and how we drive business value
  • Too many years of educating leaders on what UXR is and is good for while still bumping up against the same limits in companies.

The community shared that their exposure to lower levels of maturity in research tends to be more the case than not.

Peter Merholz acknowledged the need for higher levels of research maturity to support distributed discovery:

“If you’re in a low maturity research org and you democratize – you’re going to get sh*tty research”  – Peter Merholz

Unsurprisingly, the chat mirrored and followed much of the panel conversation, but without the same limitations of successive dialogue, the chat allowed for more diversity and depth of commentary from the attendees. Several major themes emerged:

  • Democratization of research: Discussions around making research more accessible and involving a wider range of stakeholders.
  • Concerns for quality and accountability: Conversations about maintaining high standards in research practices and ensuring accountability for research outcomes.
  • Standardization and maturity of UX research practice in technology: Dialogue regarding the need for standardized approaches and the evolution of UX research within the technology industry.
  • Change management to support organizational adoption of democratized research: Discussions focusing on the challenges and strategies for implementing democratized research practices within organizations.

Sentiment analysis

There were several topics that inspired significantly more engagement in replies and emojis than others – I ran a sentiment analysis* on the top 3 and here’s what I got:

sentiments on quality of research

Quality of Research: Discussions about the definition and measurement of research quality, as well as challenges and strategies for improving research practices.

  • 60% Positive sentiment: Language expressing support for high-quality research practices, appreciation for efforts to improve research quality, and positive experiences with effective research methods.
  • 25% Negative sentiment: Language indicating frustration with poor research practices, dissatisfaction with the current state of research quality, and challenges in maintaining high standards.
  • 15% Neutral sentiment: Language conveying neutral or factual information about research quality without expressing strong positive or negative opinions.
sentiments about challenges in the field

Challenges in the Field: Conversations regarding the difficulties and frustrations experienced by UX researchers, including issues related to time constraints, bias, and organizational dynamics.

  • 35% Positive sentiment: Language expressing resilience, determination to overcome challenges, and appreciation for opportunities to learn and grow.
  • 50% Negative sentiment: Language expressing frustration, disappointment, or dissatisfaction with the challenges faced in UX research.
  • 15% Neutral sentiment: Language conveys factual information about challenges without expressing strong positive or negative emotions.
sentiments about solutions

Solutions and Suggestions: Dialogue about potential solutions and approaches to address challenges in UX research, such as education, tooling, and community support.

  • 45% Positive sentiment: Language expressing optimism, enthusiasm, and support for proposed solutions and suggestions to address challenges in UX research.
  • 35% Negative sentiment: Language indicating skepticism, criticism, or doubt about the effectiveness or feasibility of proposed solutions.
  • 20% Neutral sentiment: Language conveying factual information about proposed solutions without expressing strong positive or negative opinions.

*= Note: Quantifying sentiment in text analysis involved assigning a numerical score to indicate the sentiment expressed in the text. In cases above, we analyzed the sentiment against the identified topics by looking for positive, negative, or neutral language used in the discussions.

Summary takeaways

My personal take after listening to the panel members talking about the current state of UX research and where it’s heading and the vibrant discussion by all the chat participants can be summarized into the following 5 categories of takeaways:

Democratization of UX Research and Its Impact on the Research Community:

  • There is a growing trend towards democratizing UX research, allowing more team members to engage in research activities.
  • This trend is driven by the need for faster insights and greater cross-functional collaboration.
  • While democratization can increase access to research, there are legitimate concerns about maintaining research quality and ensuring that insights are properly interpreted.
  • Proponents of “democratizing research” or otherwise distributing a portion of discovery tasks to non-research roles in the name of continuous discovery need to dispel the myth that “more non-researchers doing discovery means we can fire all our dedicated research professionals.
  • Overall, the democratization of UX research presents both opportunities and challenges for the research community, requiring careful management and consideration of quality standards.

The Need for Standardization and Quality Assurance in Democratized Research:

  • With the expansion of research activities to non-researchers, there is a need for standardized processes and quality assurance measures.
  • Establishing clear guidelines for conducting research and interpreting findings can help maintain consistency and reliability.
  • Quality assurance mechanisms, such as peer review and validation processes, are essential to ensure that research outputs meet established standards.
  • Standardization efforts should focus on aligning research practices with organizational goals and ensuring that insights drive meaningful action.

The Impact or Need for Research Maturity in Organizations that Want to Democratize Research:

  • Research maturity is crucial for organizations aiming to democratize research effectively.
  • Mature research practices enable teams to conduct high-quality research, interpret findings accurately, and integrate insights into decision-making processes.
  • Developing research maturity involves investing in talent, establishing clear processes, and fostering a culture of research-driven decision-making.
  • Organizations with higher research maturity are better equipped to navigate the challenges associated with democratized research and derive maximum value from their research efforts.

The Need for Change Management in Organizations Attempting to Mature and Democratize Research:

  • Implementing changes to research practices requires effective change management strategies.
  • Change management efforts should focus on aligning stakeholders, addressing resistance to change, and promoting a culture of continuous improvement.
  • Communication, training, and support are essential components of successful change management initiatives.
  • Organizations must actively manage the transition to mature and democratized research practices to ensure adoption and sustainability.

Other Insights about the Outcomes and Future Path for UX Research and Digital Product Teams:

  • There is a recognition of the importance of balancing speed and quality in research activities.
  • The role of AI and automation in research is acknowledged, but there are concerns about its potential impact on the quality of research outcomes.
  • Collaboration between researchers and non-researchers is seen as essential for driving innovation and delivering user-centered products.
  • Continued dialogue and knowledge-sharing within the research community are crucial for advancing research practices and addressing emerging challenges.

Conclusion: Shaping the Future Together

In conclusion, while the democratization of UX research presents opportunities for greater inclusivity and collaboration, it also raises important considerations regarding research quality, standardization, and organizational maturity. As we navigate the evolving landscape of UX research, it is imperative that we engage in open dialogue, share insights, and collaborate to address the challenges and opportunities ahead. By addressing the inherent challenges and implementing effective change management strategies, organizations can unlock the full potential of democratized research and drive meaningful outcomes for their digital product teams. By embracing a collective vision for the future of UX research, we can drive meaningful outcomes and shape a more inclusive and impactful research community.
My net takeaway, acknowledging that there has been a very real and negative impact to research practitioners in recent times, is that the outlook for the future of UX research is strong for the following reasons:

  • More digital product and service teams recognize the need for sound research
  • Digital orgs are looking for ways to scale their research operations
  • Scaling research operations will take professional leadership to mature and service

Will all of this take time, and will you encounter more challenges along the way? Absolutely! Change is undoubtedly hard, and we’re talking about a major cultural change. But shaking fists at the inevitable has never helped move anything forward. Shaping the future of UX research will take all of us looking at the path and charting the roadmap. More importantly than anything else, demonstrating success models shows the world what the future should look like. Nothing is more effective than being the change you want to see.

Resources Shared in Chat:

The following links were shared by participants in the chat. These links provide more nuance and depth to the topics discussed both by the panel participants and the discussion threads in chat. All are worth looking at for a deeper dive into democratized research, design leadership, change management, and methodologies and models.

UX Research and Design Leadership:
Judd Antin’s article on UX research: The UX Research Reckoning is Here
Robert Fabricant’s article on design leadership: The big design freak-out: A generation of design leaders grapple with their future
Noam Segal’s video on specialization: When people become ‘generalists’ no one is really, really good at anything
Peter Merholz’s blog post on design leadership and change management: Design Leadership is Change Management

Democratization of Research:
Carl Pearson’s article on democratizing research: Bad Research Doesn’t Stink
Robin Beers’ article on democratizing research: The right way to democratize UX research
Julie Norvaisas’ article on democratizing research: Smarter Democratization
Michele Ronsen’s perspective on democratization: Is Research Democratization Good? Who Benefits and What Should Be Considered
Kyle Soucy’s article on challenges with research democratization: The problem with research democratization that people aren’t talking about

Research Methodologies and Models:
Steve Portigal’s recommendation of Chris Avore’s design research maturity model: The organization’s design research maturity model
Michele Ronsen’s post on UXR teams articulating and demonstrating strategic value: 28 ideas for UXR teams to articulate and demonstrate its strategic value
Shane Olbourne’s recommendation of Productboard’s blog post on Product Excellence Maturity Model: Product Excellence Maturity Model

Additional Resources:
Angela Craven’s recommendation of a book: Changemakers
Thomas Wilson’s article on sustainable value: The Human Cost of Bad Design