Visual Data Analysis under
Differential Privacy
Differential Privacy (DP) has recently emerged as a new privacy-providing mechanism that guarantees the desired level of an individual's privacy. However, providing privacy always comes with a cost that is usually a loss of utility. Despite the significant growth of DP mechanisms and their applications in academia, there are plenty of unanswered questions about the growth and adoption of DP in the real world and for different groups of people who are going to use the privatized data.
Problem Definition & Target Users
Consult Experts • Related Research Papers
Consult Experts
To narrow down our problem space, we decided to consult some of the experts specializing in Differential Privacy and identify the most common issues that currently exist in this field. After a few conversations with UMass Amherst professors and reading related research papers, we decided to focus on the intersection of DP and Information Visualization.
Differential Privacy operates by injecting different amounts of noise into the data which might change original data patterns and characteristics and lead to incorrect decision-making.
Related Research Papers
We still needed to narrow down the problem space to a more clear and solvable problem. The next step toward this goal was reading the related research papers. We found that there are only a few papers investigating the challenges at the intersection of DP and Information Visualization. For example:
-
Investigating Visual Analysis of Differentially Private Data, in IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 2, pp. 1786-1796, Feb. 2021, DOI: 10.1109/TVCG.2020.3030369
-
Challenges of Visualizing Differentially Private Data, PDF Link
However, these research papers are not informative enough to let us know the most common issues that currently exist for users of differentially private data. In the rest of this project, we will follow a user-centered design process to first, identify the main user issues and try to propose solutions.
General Information
School
Course Name
Project Duration
Team
Tools
Skillset
University of Massachusetts Amherst
COMPSCI690A: Advanced Methods in Human-Computer Interaction
January 2021 to May 2021
Collaborated with four advisors across fields of Education, HCI, and Psychology
Figma, PowerPoint, Zoom, Google Forms, PyCharm
Interview, Prototyping, Usability Testing, HTML, CSS, JavaScript
Project Process
The focus of this project is on solving a problem that exists for actual users of differentially private data. Therefore, to make sure that we are solving the right problem for the right users, we are taking a user-centered design approach.
Empathize
Competitive Analysis
Interviewing
Define
Problem Statement
Personas
System-Independent Task Model
Ideate
Solution
System-Dependent Task Model
Paper Prototyping
Wireframing
Hi-Fi Prototyping
Release & Evaluation
Evaluation Design
Heuristic Evaluation
Phase 1: Empathize
Competitive Analysis • Interviewing
Competitive Analysis
We searched different combinations of keywords to find applications that targeted the same issue. However, due to the novelty of the topic, so far, there is no similar applications working at the intersection of Differential Privacy and Information Visualization.
Interviews
Currently, Differential Privacy (DP) is mostly developed in research projects and there aren't many people working with the privatized data especially when we are narrowing the topic down to visualization of Differentially Private Data. Therefore, to build empathy for our users and identify their actual concerns and frustrations, we decided to conduct interviews with 4 experts in information visualization who are familiar with DP.
Why interview?
Learning from participants through unstructured conversations
We have limited # of available users
Creating a relaxed atmosphere for participants
Limited time & budget
In the following slideshow, you can find more about our interview.
Phase 2: Define
Problem Statement • Personas • System-Independent Task Model
Problem Statement
Visual Data Analysts (VDAs) need to increase their understanding of Differential Privacy and its main parameters. This is crucial because Differential Privacy can change the characteristics and patterns of the real data during the process of privatization. Our findings from interviews demonstrated that visual data analysts are still unaware of what differential privacy is and how it operates, therefore, they cannot talk confidently about their possible challenges. The lack of knowledge about effects of DP may lead to wrong or inaccurate decisions about the data being analyzed. Thus, at this step, it is crucial for VDAs to learn more about DP and its main characteristics.
Personas
Using the results that we achieved in the Empathy phase, we are creating a primary personas for our project.
System-Independent Task Model
In order to show how individual users are currently solving the problem without our system, we are creating a task model as follows.
Phase 3: Ideate & Design
Solutions • System-Dependent Task Model • Paper Prototyping • Wireframing • Hi-Fi Prototyping
Ideating Solutions
After carefully reviewing the results from the Define phase and holding several meetings with my advisors, we decided to create a web-based visualization tool that includes the following main features. To design our solutions, we emphasized on users goals, motivations, and frustrations from our user persona to assure that their experience will be enjoyable and helpful.
Feature #1
Creating A Visualization (E.g., Histogram) from a Given Input Data
Allows our persona to visualize privatized or actual data
We chose Histogram, because it is the most common one being used by our persona
Will be used to satisfy Steps 2.1.1., 2.2.1., 2.1.3., 2.1.4., 2.2.3., and 2.2.4. in our user task model
Feature #2
Privatizing Data Using the User's Input for DP Algrithm and Privacy Budget
Allows our persona to learn by exploration of different parameters
Includes two of the most important parameters in Differential Privacy that our persona needs to learn
Will be used to satisfy Steps 1., 2.1.2., and 2.2.2. in our user task model
Feature #3
Comparing Multiple Visualizations Effectively
Allows our persona to compare different visualizations
Facilitates the comparison of different visualizations that is one of the frustrations of our persona
Will be used to satisfy Steps 2.2.4., and 2.1.4. in our user task model
System-Dependent Task Model
Now that we decided on our solution, in order to visualize and make more tangible the individual steps that a user makes during the use of our system, we have created a hierarchical task-analysis diagram (HTA).
Paper Prototyping
The first step in our Design phase is low-fidelity paper prototyping that allows the team to get early feedback about the usability of the proposed solution. We were then able to apply the required changes with less effort and cost before moving to the high-fidelity prototyping.
Wireframing
Paper prototyping gave us more confident that our design idea could satisfy users' needs. Now, we are moving to the next step that is interactive wireframing. In our wireframe, users are able to click on some elements and move between pages.
We used Figma to create the wireframe of our system. You can explore the wireframe using the following embedded element from Figma.
Findings from Wireframe
By showing our wireframe to a group of students and professors in our HCI course, we received the following feedback and will try to satisfy them in Hi-Fi prototype.
-
Users are satisfied with the general functionalities and our design layout
-
There are concerns about locations of a few elements that doesn't match other similar applications.
-
Users preferred larger visualizations in the screen to make it clearer.
High Fidelity Prototyping
Visual Design
In the next few parts, we present our decision for elements of the high fidelity prototyping.
Color Palette
We are following the 60-30-10 rule to decide about our color palette. We chose colors that could evoke our needed emotions, vibes, and aesthetics.
Background
Primary Color
#1E555C
Histogram Bars #1
#F15152
Button #1_inactive
Secondary Color
#F4D8CD
Histogram Bars #2
#F2D349
Button #1_active
Secondary Color
#F4D8CD
Button #2_default
Accent Color
#3A2E39
Selected Item's Stroke
#88D886
Fonts
Same as colors, we chose typefaces that match our application and its users.
Aa Bb Cc ... Zz
Abhaya Libre Medium-Regular
Aa Bb Cc ... Zz
AppleMyungjo
Aa Bb Cc ... Zz
Gentium Book Basic-Regular
Text Styles
Description
43 px / #FFFFFF /Abhaya Libre Medium-Regular
List Items
40 px / #F4D8CD /Gentium Book Basic
List Titles
17 px / #FFFFFF /Gentium Book Basic-Regular
Navigation Bar
27 px / #F4D8CD /AppleMyungjo
Buttons
23 px / #000000 /AppleMyungjo
Buttons
Buttons are one of the important parts of designing. We chose our secondary and accent colors to design buttons. Our buttons have two statuses of Default and Hovered as it is shown below.
Lists
Same as buttons, we chose our secondary color for designing list items. Our design here keeps the simplicity and aesthetic that we had in mind.
Glyphs
Designing High-Fidelity Prototype
We used Figma to design the High Fidelity prototype of our application. You can explore it more in the following embedded element from Figma.
Phase 4: Release & Evaluation
Evaluation Design
Our goal in this step is to minimize our application's design deficiencies to optimize its usability. To this end, we decided to conduct a Heuristic Evaluation.
Why Heuristic Evaluation?
To minimize design deficiencies and optimize UI usability
It is doable at every stage of the design
It is cheap and fast to conduct
It provides early feedback from experts before moving to implementation
In the following slideshow, you can find more about our heuristic evaluation.
Constraints
#1 Time and Cost limitation
I completed this project as a student in the HCI course and in a timeframe of 4-5 months without being financially supported. These limitations applied constraints on recruiting participants and collecting large data sets. Time limitation also prevented us from haveing multiple iteration of the design process.
If there was more time available, we could do more research on other parameters of Differential Privacy or how these parameters affect these and other types of data visualization.
#2 Novelty of Differential Privacy outside academia
The other important constraint was that Differential Privacy is still developing in academia and has not been adopted widely in industry and by large group of workers. Therefore, exploring the effects of DP on visual data analysis has a long way ahead.
#3 Lack of related works
Currently, there aren't any applications following the same goal we had to educate visual data analysts about the effects of DP on data visualizations. Even though it allows various work in this area, it also brings lots of vagueness and uncertainties.
Reflection
#1 Teaching applications require extra attention to user experience design
Through this project, I learned that proper user experience design plays a significant role in teaching an emerging concept to users. In learning situations, users need to focus on learning the concept and any distraction like user interface problems hinder their learning process.
#2 Design of interview questions plays an important role in identifying real needs and problems
Throughout the interviews, we understood that unbiased, clear, short, and organized questions lead users to explain their actual needs and problems which then leads researchers to identify the focal points.
#3 Iterating the user-centered designing is essential
Due to our time limitation, we were not able to iterate the design process multiple times. However, we felt the lack of this essence throughout our project. For example, if time allowed, we were able to examine more design ideas and select the best approach.
#4 Putting yourself in users' shoes
It is a very difficult and essential skill for UX designers to put themselves in users' shoes instead of applying their own believes and interests. It is especially crucial when the designer is expert in a field and user is starting to learn the topic.