UX Design
|
UX Research
Rethinking the Review System
@ Fiverr
Implementing Word Tags and Multi-Stage Feedback for Improved Accuracy and Better Client-Freelancer Matching
THE TEAM
Chloe Jun Chen
Harsheil Arora
Rohan Pant
Me!
MY ROLE
Primary and Secondary Research
User Interviews
Ideating and Sketching
Wireframing and Prototyping
Usability Testing
TIMELINE
September - October 2022
(4 weeks approximately)
DESIGN OUTCOME
Designing and iterating platform features
The Brief
Our everyday work environments have, over the past few years, seemed to become increasingly more accepting of “gig” work. Platforms like 99designs, Fiverr, and Upwork are facilitating and enabling this change in how we understand and approach freelance gig work. One of the most important mechanisms that make this type of gig economy work possible is how the individual freelancer is evaluated and, subsequently, how those collective evaluations are presented to a potential customer.
How might we alter the way freelancers are evaluated, highlight the user journey of working with a freelancer and encourage clients to provide a fair and balanced assessment of their work?
The Brief
Our everyday work environments have, over the past few years, seemed to become increasingly more accepting of “gig” work. Platforms like 99designs, Fiverr, and Upwork are facilitating and enabling this change in how we understand and approach freelance gig work. One of the most important mechanisms that make this type of gig economy work possible is how the individual freelancer is evaluated and, subsequently, how those collective evaluations are presented to a potential customer.
How might we alter the way freelancers are evaluated, highlight the user journey of working with a freelancer and encourage clients to provide a fair and balanced assessment of their work?
The Problem with Current Rating Systems
Freelancers get either quantitative feedback which isn’t actionable or qualitative feedback which is hard to analyze. Either ways freelancers cannot improve.
Clients struggle finding the right freelancer for the job and comparing different freelancers
Clients don’t know what working with a freelancer may be like
5 star reviews are unfair and usually biased and skewed
The Solution
Before I begin - in case you find yourself to be a person who doesn’t fancy reading, feel free to check out this simple presentation video for this project OR just keep scrolling!
Our solution proposes the following qualities to mitigate the problems we came across:
Word tags instead of explicitly qualitative or numerical quantitative data can be used to make reviews constructive for the freelancers and to make ratings more clear to clients scouting for their next big project.
Feedback divided in 3 stages of onboarding, process, and delivery can be implemented to make reviews more accurate and fair as well as to give a bigger picture into the process of working with a freelancer clear.
Matching the clients to the freelancers using the word tags to enable clients to find the freelancers that will be right for them and their project
QUESTIONS! QUESTIONS! so many questions!
We conducted secondary research on perspectives to describe a rating system grabbed our attention.
The first perspective focuses on who should give ratings. Based on this dimension, we researched on topics, including Quantitative Rating Systems, Qualitative Rating Systems and Unilateral and Bilateral Rating Systems.
The second perspective refers to what scale should be used, which discusses qualitative and quantitative feedback. The quantitative feedback system includes the 5 star rating system and the binary rating scale (like-dislike). The qualitative systems include open reviews and word tags.
Competitor Platforms and Identifying Patterns
1
All platforms highlight and prioritize the 5 star rating system
2
Clients are less likely to read through paragraphs when looking for freelancers
3
All platforms showcase the total number of reviews left
4
Most of the Platforms showcase number of projects done by the freelancer
Finding Out What Clients and Freelancers Think
We conducted user interviews with 5 freelancers and 1 ‘client’ who all had experience with one or more of the freelancing platforms and had worked on at least 2 projects. Our questions revolved around how ratings / reviews affect freelancers and client hiring as well as clients’ process for leaving reviews.
Narrowing down the Problem Frame
1
Freelancers don’t get actionable feedback or good qualitative reviews they can use to improve
2
It is hard to differentiate between freelancers with similar ratings.
3
Clients cannot find the right freelancer sometimes because of the amount of emphasis on numerical ratings which steers them away from what it’s actually like working with a freelancer and their portfolios
4
Ratings tend to be skewed because some clients may leave the same star rating for everyone regardless of performance, out of convenience or habit
Narrowing down the Problem Frame
1
Freelancers don’t get actionable feedback or good qualitative reviews they can use to improve
2
It is hard to differentiate between freelancers with similar ratings.
3
Clients cannot find the right freelancer sometimes because of the amount of emphasis on numerical ratings which steers them away from what it’s actually like working with a freelancer and their portfolios
4
Ratings tend to be skewed because some clients may leave the same star rating for everyone regardless of performance, out of convenience or habit
Ideation
Idea #1 : Progressive Feedback
This idea proposes the client leaving feedback throughout the process as opposed to just at the end. There are 3 key points that are a part of every freelancing project regardless of the type or size of project:
Onboarding: After the contract has been signed and first meeting done
Process: At a milestone or time mark around 75% completion
Delivery/Closing: At the end after the client receives the deliverables
Clients give feedback in these 3 stages and it is also presented on the freelancer’s profile in these 3 stages.
More Specific Feedback for Freelancers to Improve
Contributes in Finding the Right Freelancer
Accurate Journey & Reviews Depicted to Clients
Idea #2 : Word Tags and Reviews
This part of our solution proposes getting rid of the numerical star based system and replacing it with word tags; one to three words that describe different aspects of a freelancer.
These word tags carry over the 3 stages of a project listed above and are divided into them as well as into two separate sections:
What the freelancer is good at or known for?
What the freelancer needs to work on or improve in?
Opportunity #1 : How can freelancers tell if they’re improving?
With the same tags being on positive and negative ends, freelancers have opportunities to track their progress with how those tags shift from negative to positive or vice versa.
We started with ideas on splitting text into color, to represent how their improvement has changed over time. This however presented accessibility issues with color blind people.
Opportunity #2 : How can freelancers and clients match?
While posting a job, a client is given the opportunity to add tags that are important to them and prioritize them.
The freelancers they are matched with are based on the other information they provide like type of work, budget, timeline, etc. as well as these tags which function as another layer of filtering.
Limitations
We encountered limitations in our design that impacted our research and testing of our designs:
1
During our interview process, we had to be extremely lenient on our inclusion and exclusion criteria considering we had a difficult time finding freelancers and clients. One of our users did not even have experience using freelancing platforms.
2
Numeric based ratings are valuable to users and our solution takes that away. Completely getting rid of numeric systems can reduce trust in the rating system at least with users’ mental models about rating systems today.
3
Our usability testing was scenario based and while we got insights into how we could improve our designs, we could have had deeper assessments from our intended user groups, i.e. clients and freelancers
4
Our rating system requires more effort from the client which makes them less likely to leave reviews.
Limitations
We encountered limitations in our design that impacted our research and testing of our designs:
1
During our interview process, we had to be extremely lenient on our inclusion and exclusion criteria considering we had a difficult time finding freelancers and clients. One of our users did not even have experience using freelancing platforms.
2
Numeric based ratings are valuable to users and our solution takes that away. Completely getting rid of numeric systems can reduce trust in the rating system at least with users’ mental models about rating systems today.
3
Our usability testing was scenario based and while we got insights into how we could improve our designs, we could have had deeper assessments from our intended user groups, i.e. clients and freelancers
4
Our rating system requires more effort from the client which makes them less likely to leave reviews.
Want to check out my other work?
Visit the Archives section!
Let's go!
UX Design
|
UX Research
Rethinking the Review System
@ Fiverr
Implementing Word Tags and Multi-Stage Feedback for Improved Accuracy and Better Client-Freelancer Matching
THE TEAM
Chloe Jun Chen
Harsheil Arora
Rohan Pant
Me!
MY ROLE
Primary and Secondary Research
User Interviews
Ideating and Sketching
Wireframing and Prototyping
Usability Testing
TIMELINE
September - October 2022
(4 weeks approximately)
DESIGN OUTCOME
Designing and iterating platform features
The Brief
Our everyday work environments have, over the past few years, seemed to become increasingly more accepting of “gig” work. Platforms like 99designs, Fiverr, and Upwork are facilitating and enabling this change in how we understand and approach freelance gig work. One of the most important mechanisms that make this type of gig economy work possible is how the individual freelancer is evaluated and, subsequently, how those collective evaluations are presented to a potential customer.
How might we alter the way freelancers are evaluated, highlight the user journey of working with a freelancer and encourage clients to provide a fair and balanced assessment of their work?
The Brief
Our everyday work environments have, over the past few years, seemed to become increasingly more accepting of “gig” work. Platforms like 99designs, Fiverr, and Upwork are facilitating and enabling this change in how we understand and approach freelance gig work. One of the most important mechanisms that make this type of gig economy work possible is how the individual freelancer is evaluated and, subsequently, how those collective evaluations are presented to a potential customer.
How might we alter the way freelancers are evaluated, highlight the user journey of working with a freelancer and encourage clients to provide a fair and balanced assessment of their work?
The Problem with Current Rating Systems
Freelancers get either quantitative feedback which isn’t actionable or qualitative feedback which is hard to analyze. Either ways freelancers cannot improve.
Clients struggle finding the right freelancer for the job and comparing different freelancers
Clients don’t know what working with a freelancer may be like
5 star reviews are unfair and usually biased and skewed
The Solution
Before I begin - in case you find yourself to be a person who doesn’t fancy reading, feel free to check out this simple presentation video for this project OR just keep scrolling!
Our solution proposes the following qualities to mitigate the problems we came across:
Word tags instead of explicitly qualitative or numerical quantitative data can be used to make reviews constructive for the freelancers and to make ratings more clear to clients scouting for their next big project.
Feedback divided in 3 stages of onboarding, process, and delivery can be implemented to make reviews more accurate and fair as well as to give a bigger picture into the process of working with a freelancer clear.
Matching the clients to the freelancers using the word tags to enable clients to find the freelancers that will be right for them and their project
QUESTIONS! QUESTIONS! so many questions!
We conducted secondary research on perspectives to describe a rating system grabbed our attention.
The first perspective focuses on who should give ratings. Based on this dimension, we researched on topics, including Quantitative Rating Systems, Qualitative Rating Systems and Unilateral and Bilateral Rating Systems.
The second perspective refers to what scale should be used, which discusses qualitative and quantitative feedback. The quantitative feedback system includes the 5 star rating system and the binary rating scale (like-dislike). The qualitative systems include open reviews and word tags.
Competitor Platforms and Identifying Patterns
1
All platforms highlight and prioritize the 5 star rating system
2
Clients are less likely to read through paragraphs when looking for freelancers
3
All platforms showcase the total number of reviews left
4
Most of the Platforms showcase number of projects done by the freelancer
Finding Out What Clients and Freelancers Think
We conducted user interviews with 5 freelancers and 1 ‘client’ who all had experience with one or more of the freelancing platforms and had worked on at least 2 projects. Our questions revolved around how ratings / reviews affect freelancers and client hiring as well as clients’ process for leaving reviews.
Narrowing down the Problem Frame
1
Freelancers don’t get actionable feedback or good qualitative reviews they can use to improve
2
It is hard to differentiate between freelancers with similar ratings.
3
Clients cannot find the right freelancer sometimes because of the amount of emphasis on numerical ratings which steers them away from what it’s actually like working with a freelancer and their portfolios
4
Ratings tend to be skewed because some clients may leave the same star rating for everyone regardless of performance, out of convenience or habit
Narrowing down the Problem Frame
1
Freelancers don’t get actionable feedback or good qualitative reviews they can use to improve
2
It is hard to differentiate between freelancers with similar ratings.
3
Clients cannot find the right freelancer sometimes because of the amount of emphasis on numerical ratings which steers them away from what it’s actually like working with a freelancer and their portfolios
4
Ratings tend to be skewed because some clients may leave the same star rating for everyone regardless of performance, out of convenience or habit
Ideation
Idea #1 : Progressive Feedback
This idea proposes the client leaving feedback throughout the process as opposed to just at the end. There are 3 key points that are a part of every freelancing project regardless of the type or size of project:
Onboarding: After the contract has been signed and first meeting done
Process: At a milestone or time mark around 75% completion
Delivery/Closing: At the end after the client receives the deliverables
Clients give feedback in these 3 stages and it is also presented on the freelancer’s profile in these 3 stages.
More Specific Feedback for Freelancers to Improve
Contributes in Finding the Right Freelancer
Accurate Journey & Reviews Depicted to Clients
Idea #2 : Word Tags and Reviews
This part of our solution proposes getting rid of the numerical star based system and replacing it with word tags; one to three words that describe different aspects of a freelancer.
These word tags carry over the 3 stages of a project listed above and are divided into them as well as into two separate sections:
What the freelancer is good at or known for?
What the freelancer needs to work on or improve in?
Opportunity #1 : How can freelancers tell if they’re improving?
With the same tags being on positive and negative ends, freelancers have opportunities to track their progress with how those tags shift from negative to positive or vice versa.
We started with ideas on splitting text into color, to represent how their improvement has changed over time. This however presented accessibility issues with color blind people.
Opportunity #2 : How can freelancers and clients match?
While posting a job, a client is given the opportunity to add tags that are important to them and prioritize them.
The freelancers they are matched with are based on the other information they provide like type of work, budget, timeline, etc. as well as these tags which function as another layer of filtering.
Limitations
We encountered limitations in our design that impacted our research and testing of our designs:
1
During our interview process, we had to be extremely lenient on our inclusion and exclusion criteria considering we had a difficult time finding freelancers and clients. One of our users did not even have experience using freelancing platforms.
2
Numeric based ratings are valuable to users and our solution takes that away. Completely getting rid of numeric systems can reduce trust in the rating system at least with users’ mental models about rating systems today.
3
Our usability testing was scenario based and while we got insights into how we could improve our designs, we could have had deeper assessments from our intended user groups, i.e. clients and freelancers
4
Our rating system requires more effort from the client which makes them less likely to leave reviews.
Limitations
We encountered limitations in our design that impacted our research and testing of our designs:
1
During our interview process, we had to be extremely lenient on our inclusion and exclusion criteria considering we had a difficult time finding freelancers and clients. One of our users did not even have experience using freelancing platforms.
2
Numeric based ratings are valuable to users and our solution takes that away. Completely getting rid of numeric systems can reduce trust in the rating system at least with users’ mental models about rating systems today.
3
Our usability testing was scenario based and while we got insights into how we could improve our designs, we could have had deeper assessments from our intended user groups, i.e. clients and freelancers
4
Our rating system requires more effort from the client which makes them less likely to leave reviews.
Want to check out my other work?
Visit the Archives section!
Let's go!
🍭 By Anam Nasim @ 2024
Made with love and pixels ✨
UX Design
|
UX Research
|
Usability Testing
Rethinking the Review System
@ Fiverr
Implementing Word Tags and Multi-Stage Feedback for Improved Accuracy and Better Client-Freelancer Matching
THE TEAM
Chloe Jun Chen
Harsheil Arora
Rohan Pant
Me!
MY ROLE
Primary and Secondary Research
User Interviews
Ideating and Sketching
Wireframing and Prototyping
Usability Testing
TIMELINE
September - October 2022
(4 weeks approximately)
DESIGN OUTCOME
Designing and iterating platform features
The Brief
Our everyday work environments have, over the past few years, seemed to become increasingly more accepting of “gig” work. Platforms like 99designs, Fiverr, and Upwork are facilitating and enabling this change in how we understand and approach freelance gig work. One of the most important mechanisms that make this type of gig economy work possible is how the individual freelancer is evaluated and, subsequently, how those collective evaluations are presented to a potential customer.
How might we alter the way freelancers are evaluated, highlight the user journey of working with a freelancer and encourage clients to provide a fair and balanced assessment of their work?
The Problem with Current Rating Systems
Freelancers get either quantitative feedback which isn’t actionable or qualitative feedback which is hard to analyze. Either ways freelancers cannot improve.
Clients struggle finding the right freelancer for the job and comparing different freelancers
Clients don’t know what working with a freelancer may be like
5 star reviews are unfair and usually biased and skewed
The Solution
Before I begin - in case you find yourself to be a person who doesn’t fancy reading, feel free to check out this simple presentation video for this project OR just keep scrolling!
Our solution proposes the following qualities to mitigate the problems we came across:
Word tags instead of explicitly qualitative or numerical quantitative data can be used to make reviews constructive for the freelancers and to make ratings more clear to clients scouting for their next big project.
Feedback divided in 3 stages of onboarding, process, and delivery can be implemented to make reviews more accurate and fair as well as to give a bigger picture into the process of working with a freelancer clear.
Matching the clients to the freelancers using the word tags to enable clients to find the freelancers that will be right for them and their project
QUESTIONS! QUESTIONS! so many questions!
We conducted secondary research on perspectives to describe a rating system grabbed our attention.
The first perspective focuses on who should give ratings. Based on this dimension, we researched on topics, including Quantitative Rating Systems, Qualitative Rating Systems and Unilateral and Bilateral Rating Systems.
The second perspective refers to what scale should be used, which discusses qualitative and quantitative feedback. The quantitative feedback system includes the 5 star rating system and the binary rating scale (like-dislike). The qualitative systems include open reviews and word tags.
Competitor Platforms and Identifying Patterns
1
2
3
4
All platforms highlight and prioritize the 5 star rating system
Clients are less likely to read through paragraphs when looking for freelancers
All platforms showcase the total number of reviews left
Most of the Platforms showcase number of projects done by the freelancer
Finding Out What Clients and Freelancers Think
We conducted user interviews with 5 freelancers and 1 ‘client’ who all had experience with one or more of the freelancing platforms and had worked on at least 2 projects. Our questions revolved around how ratings / reviews affect freelancers and client hiring as well as clients’ process for leaving reviews.
Narrowing down the Problem Frame
Freelancers don’t get actionable feedback or good qualitative reviews they can use to improve
Clients cannot find the right freelancer sometimes because of the amount of emphasis on numerical ratings which steers them away from what it’s actually like working with a freelancer and their portfolios
1
3
It is hard to differentiate between freelancers with similar ratings.
Ratings tend to be skewed because some clients may leave the same star rating for everyone regardless of performance, out of convenience or habit
2
4
Ideation
Idea #1 : Progressive Feedback
This idea proposes the client leaving feedback throughout the process as opposed to just at the end. There are 3 key points that are a part of every freelancing project regardless of the type or size of project:
Onboarding: After the contract has been signed and first meeting done
Process: At a milestone or time mark around 75% completion
Delivery/Closing: At the end after the client receives the deliverables
Clients give feedback in these 3 stages and it is also presented on the freelancer’s profile in these 3 stages.
More Specific Feedback for Freelancers to Improve
Contributes in Finding the Right Freelancer
Accurate Journey & Reviews Depicted to Clients
Idea #2 : Word Tags and Reviews
This part of our solution proposes getting rid of the numerical star based system and replacing it with word tags; one to three words that describe different aspects of a freelancer.
These word tags carry over the 3 stages of a project listed above and are divided into them as well as into two separate sections:
What the freelancer is good at or known for?
What the freelancer needs to work on or improve in?
Opportunity #1 : How can freelancers tell if they’re improving?
With the same tags being on positive and negative ends, freelancers have opportunities to track their progress with how those tags shift from negative to positive or vice versa.
We started with ideas on splitting text into color, to represent how their improvement has changed over time. This however presented accessibility issues with color blind people.
Opportunity #2 : How can freelancers and clients match?
While posting a job, a client is given the opportunity to add tags that are important to them and prioritize them.
The freelancers they are matched with are based on the other information they provide like type of work, budget, timeline, etc. as well as these tags which function as another layer of filtering.
Limitations
1
During our interview process, we had to be extremely lenient on our inclusion and exclusion criteria considering we had a difficult time finding freelancers and clients. One of our users did not even have experience using freelancing platforms.
2
Numeric based ratings are valuable to users and our solution takes that away. Completely getting rid of numeric systems can reduce trust in the rating system at least with users’ mental models about rating systems today.
3
Our usability testing was scenario based and while we got insights into how we could improve our designs, we could have had deeper assessments from our intended user groups, i.e. clients and freelancers
4
Our rating system requires more effort from the client which makes them less likely to leave reviews.
We encountered limitations in our design that impacted our research and testing of our designs:
Want to check out my other work?
Visit the Archives section!
Let's go!
🍭 By Anam Nasim @ 2024
Made with love and pixels ✨