Creating design measures & automation
By addressing the challenges associated with measuring design quality and implementing our data-driven approach, we transformed the way design teams at Capital One evaluate and improve their products, leading to improved user experiences and business outcomes.
Outcomes of my work in this area have garnished attention from several national and international conferences, and has led to a white paper publication with UXDX in 2022.
Problem
The measurement of design quality in today's organizations is plagued by scattered, inconsistent, and subjective methods. The lack of hard numbers often leads to design evaluations being dismissed by product, business, or tech teams. Furthermore, the challenge lies in finding suitable data sources for measuring quality and incorporating them into the product or data tagging at the time of release.
Approach
Recognizing these challenges, I embarked on an extensive research journey to develop an objective usability framework that could revolutionize the way design organizations measure quality at scale. My goal was to introduce quantitative, objective measures throughout the product life cycle. To achieve this, I focused on creating a system capable of leveraging user interaction data to automate the measurement of design quality.
Innovative Work
To bring my vision to life, I quickly assembled a contractor team comprising skilled data analysts who shared my passion for transforming design quality measurement. Our project entailed comprehensive research into the existing data landscape within the enterprise, including data consumption, ingestion, reporting, and analytics capabilities. Additionally, we developed a roadmap for building a design data management capability within the organization.
Results: After an intense 8-month effort, we successfully launched the first pilot program, which involved measuring seven key formulas for objective usability in design. We achieved this by leveraging existing data from user interactions with the product.
Innovative Work
The outcomes of our initiative were as follows:
-
Design Modernization Metrics and Standard Development Outlines: We developed metrics to gauge the modernization of design and established standard development outlines for future projects.
-
Significantly Reduced Access Time to Key Analytics: We reduced the time required to access crucial analytics by a staggering 98%, enabling faster decision-making processes.
-
Increased Demand for Measurement: Four new product teams within the organization expressed their interest in adopting the same type of measurement for design quality.
-
Establishment of Design Data Manager Capability: We successfully introduced the first-ever design data manager capability within Capital One's design organization, fostering efficient management and utilization of design-related data.
-
International Recognition: The underlying framework and case study received recognition through three international presentations, enabling knowledge-sharing and promoting best practices across the industry.
-
Publication: The success of our approach and its impact on design quality measurement culminated in the publication of our case study, disseminating our innovative solution to a wider audience.
-
Enterprise Adoption: Our quantitative approach and progress was adopted by the larger design organization operations as key work to build standardized requirements for the entire company.
