Fall 2022
Research Project
In this course, you will complete a mini-research project of a self-contained idea in visualization.
This project will be completed individually, or in groups of two. The milestones are as follows (all deadlines are 11:59pm EDT):
-
DUE 14 OCT.
Project abstract (5 pts) -
DUE 4 NOV.
Mid-term project video (15 pts) -
DUE 14 NOV.
Peer feedback on projects (5 pts) -
DUE 7 DEC.
Project presentation (15 pts) -
DUE 14 DEC.
Research report (20 pts)
At a high level, successful projects raise a question about visualization, and plan and execute a project for answering that question.
Grading Scheme
| Project Abstract - 5 pts |
| The strength of the research problem (i.e., do pick something that's research, not just development) and appropriateness of the scope (i.e., don't pick something too large — it's only a 12-week term.) Is it executed in ways necessary to understand your research question? (It doesn't need to be functional in ways orthogonal to your research question.) |
| Mid-term Video Presentation - 15 pts |
| A short video explaining your problem motivation, research question and research methodology |
| Peer Feedback on Video Presentation - 5 pts |
| Comments, suggestions and questions to other students on their project |
| Final Presentation - 15 pts |
| A presentation of your research results to the class (and possibly external guests). |
| Final Report - 20 pts |
| Explanation of your ideas, motivation, and related work. How well did you figure out what you set out to learn? |
In general, for teams of two students, both members of the team will receive the same grade. However, if there is a significant contribution disparity, that will be reflected in the grade.
What do you mean by "research project"?
Your project must be about research, and not engineering!
While most projects build and evaluate a prototype system, programming is not strictly necessary. Some perceptual or cognitive psychology studies, e.g. (Xiong, van Weelden, and Franconeri, 2019), (Batman et al., 2010), taxonomies, e.g. Dimara et al., 2018), in-depth qualitative user research, e.g. (Walny et al., 2018), research practices and methods (Bressa et al., 2019), and beyond-the-desktop exploratory works, e.g. (Wun et al., 2019), are all examples of research contributions that do not involve programming. A research contribution is not equal to engineering: the methodology employed, the originality of the research and technical soundness of the work are most important . A well-crafted application will not be a subsitute for research quality and contribution.
Determining your main research contribution.
You should carefully think about what your project's main contribution is. Tamara Munzner's guide to writing information visualiztion research papers will help you define where your contribution lies, through the description of paper types (Munzner, 2007): technique, design study (which may or may not involve the implementation of a functional prototype), systems, evaluation, and model. Being able to clearly articulate your main research contribution (and secondary contributions) is essential. You should engage in this exercise early in the project development, and keep iterating on the formulation of your contribitions. It will help you choose the appropriate validation method(s), will help you guide your work, will help you better frame your paper, and ulitmately, it will help the reader understand what your work has to offer.
Every research project requires validation.
Note that validation doesn't necessarily mean a lab experiment. While thorough user studies will be expected of projects that do not involve system building (e.g. design studies, perceptual studies), validation of a prototype can be in the form of use case scenarios for instance. There are many validation methods to choose from. An appropriate validation method depends on the research question and the main contribution. Tamara Munzner's Process and Pitfalls in Writing Information Visualization Research Papers (Munzner, 2007) write-up discusses validation methods for different paper types. A more complete, excellent resource is her "Nested Model for Visualization Design and Validation" (Munzner, 2009). Discuss with your instructor what validation steps are crtical to integrate in your case, if you are unsure. Similarly to a project involving developement, the methodology, originality and technical soundness of the work will be evaluated.
Work hard. Aim for the best.
We will encourage author(s) of excellent projects to further extend their work and submit to the IEEE VIS Short Papers, or Posters track (deadline in June). You must have IRB clearance prior to conducting your study if you plan a later submission of your work that includes user data.
Project Abstract
An important part of your course work is to articulate a research question and make a plan to address it. To do this, you'll need to: 1) read a lot of papers in an area of interest, and 2) generate a lot of ideas. Leveraging an existing project on campus will get you to a novel idea quicker, increase your impact, and be more rewarding. The goal of the project abstract is to help you scope your work appropriately and devise a solid plan to conduct your research.
Description
Submit a meaningful title and an abstract describing your project.
The abstract (max 1 page, excluding images and references) can be seen as a preliminary draft of the introduction of the research paper (i.e. project report) that you will submit at the end of the term. The abstract should describe the topic of your project: motivation, e.g., why it is important (why care? who cares?), why it is difficult) and the research question that you are trying to address, a brief positioning (what has been done in the area, and why these prior works are limited; what are prior works you bear inspiration from and why), a description of your envisioned research methodology and expected contributions as well as the associated limitations and risks.
How will it be graded?
The following components will be evaluated in this assignment.| Motivation (1 pt) |
| Research Question (1 pt) |
| Positioning (1 pt) |
| Envisioned Research Methodology and Expected Contributions (1 pt) |
| Limitations and Risks (1 pt) |
Submission instructions
Submit your abstract (max 1 page; images and references, do not count toward the page limit) in the PDF format on Quercus, by the submission deadline.Make sure your submission includes a project title!
Mid-term Project Video
Another important part of your course work is to effectively communicate your research using different communication mediums. In human-computer interaction and visualization research in particular, video presentations are commonplace to explain and demonstrate research contributions involving (oftentimes highly interactive) visual content. The goal of this assignment is for you to practice with the design and making of a research video. To do this, you'll need to: 1) watch a lot of research videos for inspiration, 2) plan your video carefully using storyboarding and scripting, and 3) make a professional-quality, concise video explaining and demonstrating your research. This video will then be shared to the rest of the class, who will provide feedback, comments and questions to each group.
Description
Submit a short video (3-5min) describing your project.
The content of the video is entirely up to you. The focus is on effective communication of what your research project is about, and what is your approach to address the research question(s). Typically, the video presentation would cover the following aspects: motivation, e.g., why it is important (why care? who cares? why is it difficult?) and the research question(s) that you are trying to address, a description of your envisioned research methodology and associated limitations and risks. Usually, including prior works in research videos is not necessary, however, it is up to you to include, if you feel it best serves your message. You may also include progress to date and/or mock sketches of envisioned solution or study design. Be creative, be compelling, engage your viewers! Tell your audience an intersting story through your video.
Useful guides on how to prepare a good video include the UIST video guide, UIST video previews guide, the CHI Guide to a Successful Video Submission.
How will it be graded?
The following components will be evaluated in this assignment.| Organization and Clarity (5 pts) |
| The video presents information in a clear, logical and interesting sequence that the viewer can follow. |
| Research Objectives and Methodology (5 pts) |
| Research objectives and methodology are clearly stated, appropriately detailed and strongly supported throughout the video. |
| Video Production (5 pts) |
| Video is of professional quality. Video runs smoothly from shot to shot. Appropriate transitions are used to assist in communicating the main idea. Shots and scenes work well together. Visual content and audio are of high quality, and explain and reinforce key points in the video. |
Submission instructions
Submit your video on Quercus by the deadline.Resolution must be at least 720p.
Video presentations will be shared to the class, for peer feedback.
Peer Feedback on Projects
Good research doesn't happen in a vaccum. Rather, it is often the result of multiple iterations, building on constructive critique and questions from peers, domain experts, and beyond. This is why communicating about your work at different stages of its development is essential for obtaining valuable feedback to improve. The goal of this assignment is to generate such feedback to help every group improve on their work.
Description
Provide written feedback to other groups assigned to you on their project, based on the video presentation (these will be communicated via Quercus mail).
Your feedback should include positive comments on what you found to be good; constructive and actionable comments, such as what you had difficulties understanding and could be clarified, what the researchers may want to consider but didn't talk about; and questions to the group, if any. While comments on the video format are welcome, they are secondary: your feedback should focus primarily on the research itself.
The book Discussing Design : Improving Communication and Collaboration Through Critique, by Adam Connor and Aaron Irizarry, O'Reilly Media, 2015. ISBN: 9781491902400. is a useful guide on how to seek, and provide feedback. In particular, the Chapter 1: Understanding Critique and Chapter 2: What Critique Looks Like are important resources that you are strongly encouraged to read.
How will it be graded?
The following components will be evaluated in this assignment.| Feedback is complete, pointed and detailed (2 pts) |
| Feedback is clearly articulated (1.5 pts) |
| Feedback is actionable (1.5 pts) |
Submission instructions
Write feedback for each project and submit as a separate document. Include the title of the project you provide feedback for (where available), but do not include your name on these documents.Submit your feedback documents (pdf) on Quercus before the deadline.
Project presentation
The most important thing about doing projects is sharing the final result with other people. The goal of this assignment is to explain your entire project and process to the audience (i.e. the class and possibly external guests), in an engaging way, to convince them that your project idea and research contribution are solid, important and intersting.
Description
The final presentation should mostly focus on the work you have realized, your research contribution(s) (i.e., field study, design rationales, demo/description of the solution, usability study, evaluation, ...). Intro / motivation / related work, while worth including, should remain concise. Each group will have 7 minutes to present. This time limit will be strictly enforced, and your time begins as soon as the previous group ends. There will be 3 minutes of Q&A after each presentation.
The final presentation should be seen as a simulation of conference presentation. Given that both live remote presentations and pre-recorded presentations are now standard modes of delivery in conference, groups can choose between practicing their live presentation skills, or record their presentation in advance. It's up to your team which mode of delivery you prefer.
How will it be graded?
The following components will be evaluated in this assignment.
| Organization and Clarity (5 pts) |
| The presentation follows a clear, logical and interesting sequence of ideas that the audience can follow easily. |
| Research Objectives and Methodology (5 pts) |
| Research objectives, methodology and results are clearly stated, appropriately detailed and strongly supported throughout the presentation. |
| Presentation Quality (5 pts) |
| The presentation slides are well crafted and used appropriately. (i.e. excellent visual aids, seamless use of visual aids). In other words, visual content is of high quality, and helps reinforce key points in the presentation without competing with the speaker. The presentation delivery is professional and of high quality: smooth effective delivery, good voice control, appropriate eye contact, appears relaxed, speaks without notes. |
Submission instructions
If you prefer to do a live presentation: submit your slides on Quercus by the deadline.If you prefer us to play a pre-recorded video of your final presentation: submit your pre-recorded presentation or link to your pre-recording on Quercus by the deadline.
Research report
Description
Final papers should be 3-5 pages long in the two-column format as follows: Word | LaTeX | Overleaf. While this may sound short, it is much harder to write an effective, complete short paper than it is to ramble. A good approach to writing a great short paper is to write a long one first, and then trim it down to the most vital parts. Appendices are acceptable and optional (they don't count towards the page limit), but won't be graded. Add one for materials you want an interested reader to see (for example, when we post your project on the website for next year), but don't need to be graded. Page limit excludes references. The final report should be seen as a research paper submission, short format (3-5 pages, references do not count toward page limit). As such, it should include all of the components of a reasearch paper (strong motivation, positioning, clear statement of the reseach question and contribution, ...). Most of the papers presented in class are good example of good quality research papers (you should read all papers presented in this class!). You are also strongly encouraged to carefully read Muzner's advice on writing visualization research papers, as well as these simple rules to structuring research papers.
Submission instructions
Submit a 3-5 pages paper PDF using the provided template by the deadline, on Quercus.Make sure to include the full name of every group member.
Notes on IRB
▪ NO data collected as part of a class project may EVER be published unless you receive IRB approval BEFORE beginning to collect data. This is not something that can change later. We encourage PhD students taking the class to apply for IRB, especially given that the IRB process for studying interfaces is streamlined, i.e., usually receives Exempt status. (This is why we offer extra credit for completing the CITI training course, so that you're set up to apply for IRB.) With IRB approval, you can publish your work if something great emerges.
▪ Even for class projects that are not subject to IRB jurisdiction, you still have a responsibility to protect the welfare of those participating in your class project.
▪ If you would like to record sensitive personal information (like health information) you MUST show me written IRB approval/exemption prior to doing so, even if you don't intend to publish your work.