Project

Industry Sponsored - FanDuel

Timeline

Fall 2023

Team

Aaron Gabryluk, Alice Gao, Cassia Tang, Meera Bharat, Viswak Raja

tools

Figma
Notion
Miro
Illustrator
Photoshop

Tags

Research Methods

User Experience Design

FanDuel
Using Generative AI to increase user performance in Fantasy Sports

Overview

FanDuel specializes in mobile sports betting in multiple states across the United States, and have two major products: FanDuel Sportsbook and the Fantasy app. They, partnered with team RGB-GT, and are now looking into how they will be able to utilize generative AI to improve the user experience on the Fantasy app. Our team has been tasked with conducting user research to determine how fantasy players are utilizing AI to improve their experience and how to integrate this into their infrastructure. Furthermore, FanDuel has expressed their interest in setting themselves apart from their competitors by using AI to provide strategic advantage to their players and create a more personalized experience than their competitors can provide.


TL;DR

Too long to read?, I've got you covered
What is it?

An evidence-based design study and exploratory research on the use of AI assistants primarily aiding new players on Fanduel's Fantasy football application.

Why is it needed?

Based on our research we found that amateur users would usually have to their own research on the players they choose for their roster and rely on friends to create and learn how to make decisions while playing Fantasy sports. This would mean the user would have to rely on other sources of information outside the application and evaluate on how these info would effect their decisions

How is it impactiful?

The feature we are adding would aid in the initial steps of user experience and aid the user in getting verified information on the app. Additionally we wanted to provide a quick highlight of the performance and of the football game with insights on how they can improve in their upcoming games. The user can also choose to share their highlights on social media to share their achievements and  encourage their other members in their team

My Role and Responsibility

I was actively involved in user research, conducting interviews to gather insights directly from our target audience. Additionally, I was responsible for the product design, I created wireframes and prototyoes to showcase functionalities, Additionally I created a design system for the final design iterations

Problem Statement

How might we utilize generative AI to improve the user experience on the Fantasy Sports app?

Main Features

01
Game Summary

The Highlight page gives a summary of all the highlights of the user’s most recent game. It also provides AI suggestions for improvement that can help users strategize better for their next fantasy games.

02
Highlights Reel

The highlights can be shared either as a video reel  based on user preference. The user can share these on any social platform or choose to save it on their device.

03
Highlights Report

The highlights can be shared also  as a report. The user can share these on any social platform or choose to save it on their device.

04
Highlights from History

This feature provides a way for the user to get summary of highlights from their past games. The user can select games from the history tab and create a highlight.

05
Edit Highlights

Users can edit the visual style and the content of the highlight video. The user has the option to decide the style from the given options, that changes seasonally and also use AI prompt to describe how they want the video to be generated.

Process

Define
  • Context
  • Target group
  • Existing Issues
Research
  • Semi-structured interview
  • Surveys
  • Website breakdown
  • Competitive analysis
  • Task Analysis
Design
  • Ideation
  • Wireframe
  • Design system
  • Hi-Fi prototype
Evaluate
  • Expert evaluation
  • User-based testing
  • Recommendations

Define

Context

Fanduel

FanDuel has 12 million registered users

In United states, FanDuel’s Fantasy app is operating in 44 states

Utilization of AI

In 2022, AI- powered sports betting platforms boasted an accuracy rate of over 80%

Utilization of AI in the sports industry will reach a value of 19.2 billion dollars by 2030.

Market

The global fantasy sports market size was estimated at USD 20.30 billion in 2022 and is expected to grow at a CAGR of 14.1% from 2023 to 2030.

Despite the 2023 generative AI boom, development has been in progress for decades prior. Generative AI has been used for menial tasks such as spam detection and auto-complete for decades, but has now found itself being able to generate full text, code, art, logos, and even 3D models.

Sources:
https://www.lexology.com/library/detail.aspx?g=d167a21f-57cf-4a9a-b8e7-612c220b2e94,
https://www.alliedmarketresearch.com/artificial-intelligence-in-sports-market-A12905

Target Group

The primary target users are Daily Fantasy Sports(DFS) players. The DFS players can further be characterized as the following:

  • Novice Users
  • Professional Users

Existing Issues

Challenges collecting information:

  • Fantasy sports applications, despite many containing some form of in-app news, lack what users need when making decisions.
  • Users will often go to third-party sources to research, increasing their workload.

No current generative AI integration:

  • None of the major fantasy sports apps currently contain  generative at any point of the user journey.
  • Lack of familiarization and the usage of generative AI by users.

Difficulty for beginners:

  • For newcomers, fantasy sports can be complex and challenging to understand.
  • Applications need to provide better educational resources and a smoother onboarding process for beginners.

Lack of transparency:

  • Users may be frustrated by a perceived lack of transparency regarding how fantasy sports apps operate, including how algorithms work, how contests are structured, and how they determine the winners.

Research

01
Semi-structured Interview

Goal

Analyze how current fantasy sports users use their DFS applications, identify which features they value, and uncover relevant motivations, attitudes and behaviors.

Overview

We covered 4 main sections, each focused on different aspects of the user experience in fantasy sports, especially for app use.

  • Basic information on engagement with DFS
  • Recommendation system
  • Drafting methods
  • Perception and trust on generative AI

Key Findings

We conducted 5 interviews with Fantasy sport players and used Dedoose to open code the transcripts. A few key findings are listed below:

  • “Drafting a League” (103), an outlier among parent code and “Using a Recommendation system” (26)
  • A few strategies, such as “relying on others' suggestions”, “basing decisions on expected points and price”, are most commonly used.
  • The “Choosing Players” step is most common (45) "Gathering Information" (38)
02
Survey - Fantasy Sports

Goal

  • Understand how current fantasy sport users are interacting with their app and what features they would value or not value.
  • Understand how Generative AI could play a role in customization of the app and the drafting process.

Overview

Survey was distributed among people who have experience playing fantasy sports. The structure of the questions were as follows

General Information

  • Type of fantasy sports user play.
  • Platform and contest preferences.

Generative AI for Customization

  • Current customization features.
  • Opinion on using AI for recommendations.

Generative AI for Drafting.

  • Current ways for conducting drafting research.
  • Opinion on using AI to assist drafting.
  • Trust of AI and factors that influence trust .

Key Findings

We received 31 responses and for analysis we used Qualtric’s built-in analysis system for quantitative data and coding for open-ended questions. A few key findings are listed below:

Generative AI to Assist Drafting (two clusters):

  • Small group of individuals  have no trust in AI.
  • Majority of individuals who find AI valuable and have decent amount of trust.

Generative AI for Customization:

  • Most people are okay with sharing historical fantasy sports data with AI
  • People express more concerns such as “privacy”, “want to keep fantasy & life separated”, and “don’t see any benefit from sharing”
03
Survey - Generative AI

Goal

  • Understand how people are interacting with Generative AI and their preferences.
  • Understand how much people rely on and trust information generated from AIs and what factors would influence their trust.

Overview

Survey was distributed among people who have heard of or used Generative AI. The structure of the questions were as follows

For participants who have heard of generative AI and ever used it:

  • Potential obstacles preventing them from using Generative AI.
  • Potential ethical concerns they have.

For Participants Who Have Used Generative AI

  • Purpose and frequency of use
  • How do they interact with Generative AI
  • Comfortableness, reliability and trust
  • Factors that influence trust

Key Findings

We received 22 responses and for analysis we used Qualtric’s built-in analysis system for quantitative data and coding for open-ended questions. A few key findings are listed below:

Communication Preferences:

  • Most people prefer “command-like” and “question-like” prompts.
  • Most people use short and concise sentences when interacting with Gen AI.

Comfortableness and Trust:

  • Most people feel comfortable interacting with Gen AI and trust the responses
  • But significant amount of people are neutral and negative (40%)
  • Participants rated “accuracy of responses” as the most important factor.
04
Website Breakdown

Goal

The goal is to systematically analyze and evaluate the user interface, functionality and identify areas of improvement in the process of creating and playing a Fantasy sports league. The breakdown can help to pinpoint on issues and ensure that the app aligns with the user’s expectations.

Overview

  • The complete journey involved in creating a contest, drafting line ups, and analyzing results during game time was captured to get an idea of daily fantasy sports works.
  • By doing this process, it was easy to understand the game better and provide potential design ideas that can support the scope of the project.
  • Our team created a contest among three members of the team during a game night to gain first hand experience the game-play of daily fantasy sports. Screenshots of each steps were documented to evaluate and analyze every step.

Key Findings

05
Competitive Analysis

Goal

The goal is to identify where FanDuel stands in comparison with its competitors and to also identify potential gaps where the user experience can be improved with respect to the use of generative AI.

Overview

  • We identified and analyzed 5 top competitors in the realm of Fantasy sports.
  • These competitors were selected based on the preferred platform informed from our Interviews and survey.
  • The categories of features were informed from the questions asked in the survey that were relevant to the use of generative AI with respect to personalization and  recommendations.

Key Findings

  • Some competitors target expert players more, hence they don’t have contest recommendations.
  • Some apps have profile customization to support the social media features where users can post updates and chat with friends.
  • Scoring customization helps in engaging a wide array of users based on level of experience.
  • Line-up display is important to make the data easy to understand for users.
  • All apps provide real time statistics on player's performance.
  • In-app news help users be informed about the live sports news updates so that user's don't have to rely on other sources/apps.
  • AI is used only to analyze and represent data, but not used as recommendation system.
06
Task Analysis

Goal

To understand where any pain points may be withing the core process of using Fanduel

Overview

  • We went step by step through the process of participating in a competition using the Fanduel app. We then wrote down each of the steps and visualized them with a flow chart diagram.
  • There were a total of 13 steps with 5 individual total sub-steps.
  • Within the main steps, we included the sub-process of selecting a player, which was a total of 5 steps.

Key Findings

  • An in depth understanding of the competition participation process
  • Understanding where pain points may be for inexperienced players.

User Persona

Based on our insights and observations from the research methods, we created a persona that captured the essence of our users and their characteristics.

Design Implications

Based on our insights and observations from the research methods, we derived design implications that will help us inform our design iteration in the next phase.

Design

Ideation

As a team we brainstormed on different concepts based on our initial design implications. Our purpose was to ideate different methods of introducing generative AI to assist using users in driving decision and improving their experience in the FanDuel DFS app.

Concept 1
Fantasy Friend
  • Practice competing against AI companion.
  • Custom difficulty level based on experience level.
  • Provide feedback to improve the user’s performance for the future contests
Concept 2
Lineup Generation + Customizable AI
  • Choice between prompting or generating a lineup
  • Users can enter keywords and requirements to generate a custom lineup, also provide prompt suggestions.
  • Select the AI’s personality such as “Coach-like”
Concept 3
Player Comparison
  • Compare player stats side by side
  • Recommendations and insights to interpret stats
  • Table view and bar chart view of stats and simulations
Concept 4
Player Recommendation
  • Player recommendations based on personal preferences, statistics, Poplar players, opponent’s strategy
  • This button pops up while users are drafting their lineups as a suggestive AI-tool.
Concept 5
Avatar & Immersive Game View
  • Talk to any player avatars to get direct info and stats.
  • An immersive game view showing real-time alerts of significant moments
  • See your competitor’s and your lineup.
Concept 6 (Selected)
Contest Highlights
  • AI-generated highlights of users performance
  • Includes thumbnails of performance, option to create a video collage for sharing on social media
  • Feedback on drafting (i.e. players, strategies, etc.)

Feedback Sessions

01
Session 1

Goal

Gain insights on our 6 sketched design ideas from a user's perspective and pick 3 best ones.

Sequence

  • Briefing the participants about the activity and the problem statement of the project.
  • Walking through the each of the 6 sketches/concepts and asking feedback questions.
  • Rating of 6 sketches using Likert scale.

Questions

  • How well do these concepts align with your user needs?
  • What modifications or improvements would you suggest?
  • Are there any accessibility issues you foresee?
  • Which features do you find most useful or relevant?
02
Session 2

Goal

To refine our designs, ensure they align with FanDuel’s objectives, and pick the final idea.

Sequence

  • Remind participants about the problem statement.
  • Walking through each of the 3 sketches/concepts and asking feedback questions.
  • Get feedback on which one we should proceed.

Questions

  • Could our designs fit within FanDuel's existing structure?
  • Are there any user needs we have overlooked?
  • What challenges or constraints there might be in implementing these designs?
  • Do you think these designs might impact user engagement on FanDuel?

Information Architecture

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Wireframe

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.

Post contest summary

Edit Highlight

Goal

Feedback Session- Wireframe

Goal

To refine and delve into the intricacies of the final concept wireframe.

Three sessions were performed: 2 focus group sessions of 4 participants each, 1 individual interview.

Sequence

  • Briefing the participants about the activity and the problem statement of the project.
  • Briefing the context of the selected concept.
  • Walking through the wireframe and showcasing different use-case scenarios.
  • Feedback questions and discussion.
  • Rating of design using Likert scale.

Questions

Questions were framed to understand the overall usability and functions of certain elements of the design:

  • What feedback can you provide on the "Highlight Reel" design?
  • What type of content or AI observations would you prefer in the "Highlight Reel"?
  • Are there any accessibility issues within this design?
  • Based on the current design, how likely would you use this feature?

Key Findings

From the notes taken during feedback session, we conducted an interpretation session to synthesize user’s feedback. Then we extracted key findings and specific design implications from their suggestions

High Fidelity Design

Design System

We created a design system based out of Fanduel’s app to make our wireframe more coherent to the existing application

Design Prototype

Evaluation

Heuristic Evaluation

Experts were asked to evaluate our prototype based on 8 metrics. Each metric contained anywhere between 4 and 22 questions associated with it, Each on a 5 point scale.

  • Visibility of system status
  • Match between the system & the real world
  • User control and reversibility
  • Consistency and navigational clarity
  • Error prevention
  • Memory recognition ease
  • Flexibility and efficiency of use
  • Simplicity and information architecture

Scores

User testing

To evaluate this prototype, we conducted a remote un-moderated task-based usability study with Fantasy sport players spanning across different demographics and expertise level. This was followed by a qualitatitve survey that asked participants to describe their thought process while performing the tasks.
‍
Here's a brief overview of the process:

  • Users were asked about their immediate impressions about the app (functions, features, design), without knowing what it does.
  • Explained the features and followed up with questions about their impressions.
  • Users were asked to complete 2 tasks
  • Users were then asked about their experience (things they liked/disliked and advice)
  • Provided them with SUS and word choice questionnaires to gauge their experience.

Test Findings

Post task

+ Positive

  • Users enjoyed the highlights and highlight video.
  • All users found at least some of the base information useful.
  • Design was clean, simple, and straight to the point.

- To Improve

  • Users wanted a more formal and serious form of highlights, providing stats.
  • They want more direct actionable advice for their next game.
  • Users requested adjustments to the editing section.
  • Improve information architecture (tiles and filtering)

Word Choice

+ Positive

  • 13 instances where user’s top 5 words were positive
  • Mostly were descriptive of the design, such as fresh and clean

- Negative

  • 6 instances where user’s top 5 words were negative.
  • Mostly were descriptive of the functionality like ambiguous and confusing

SUS

  • Users expressed on average wanting to use this feature frequently.
  • Users agreed overall that the feature was easy to use.
  • Overall SUS score was a 72.5%

Reflections

This was my first project since I began my Master's in HCI at Georgia Tech. It was a great learning experience working with my fellow classmates, all of whom come from varied backgrounds and complement each other's skill set. Being able to work with an industry partner definitely made the entire thing much more realistic than a traditional 'no-constraints' class project.

Next Steps

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.