Recent Changes

Monday, November 30

  1. page IA04 Observation Study edited Due: Friday, December 11 Points: 66 points (about 7% of your overall grade) Assignment Overview …
    Due: Friday, December 11
    Points: 66 points (about 7% of your overall grade)
    Assignment Overview
    One of the key responsibilities of the HCI professional is to ensure that applications and websites work for all users (Universal Usability). In some cases, making the system work for blind users is a federal requirement; in other cases it just makes good business sense. But it is hard to truly understand how a blind person interacts with a computer without witnessing the interaction.
    One thing that you find is that many people who are blind or sight-impaired are experts at using screen readers. You will not be able to come anywhere near their level of expertise. But even with rudimentary use of a screen reader, you can discover how frustrating it is when websites ignore basic usability principles.
    In this assignment, you and your partner will experience a website using a screen reader and the keyboard, and identify issues with websites that you can fix by paying enough attention.
    What to Do
    Pick one partner to work with. This does not need to be someone from your project team; it can be anyone in the class. You can use the discussion board in Canvas to help you find a partner. Please do this right away, and let me know if you are having a hard time finding someone to work with.
    Pick a website that (at least one of) you are familiar with that involves a transaction. It can be a shopping site, or some other type of transaction, but it must require that at least one form field be filled out. Don't choose a major site like amazon.com or ebay.com. If you choose a shopping site, you can stop at the point where you need to put in payment information, as long as you have already filled out at least one form field. Decide on a task using the website that involves submitting at least one page and getting some sort of confirmation.
    Make an appointment at the Adaptive Technology Lab to use a computer with JAWS installed, or set up your own computer with an assistive screen reading program (see the list below). Use the system to browse some pages on the Internet, to make sure the installation works correctly and to gain some familiarity with the software. You will need to use the keyboard exclusively to get around. I suggest you make a cheat sheet of the most common/useful keyboard shortcuts for your system. If you are using JAWS, for example, here are some tips for web surfing with JAWS.
    One partner should close your eyes or put on a blindfold, and go to the website you selected in step 1. Perform your selected task using the site. The other partner will record your progress and where you have problems. Use a talk-aloud protocol as you navigate the site, so your partner can record not just your actions but your intentions and expectations.
    For three of the spots where you had problems, identify the cause and come up with a way to resolve the problem. These should be problems principally due to the way the website was built, rather than due to your inexperience using the screen reading tool. Refer to the tutorial from reading [[/courses/1156593/assignments/3975829|R09]], or do your own research to find these resolutions. You will probably need to look at the HTML source of the Web page to precisely determine the problem. For most browsers, you can get to the source by right-clicking on the page. If possible, you can use a debugger to help you find the problematic HTML. If you are using the AT Lab computer, you may need to revisit the site on your own computer to use a debugger. For a quick lesson on browser debugging, see <website or document address here>
    Partners should switch roles, and the one who did not use the screen reader for the first task should use the reader now. Go to the Chicago Lighthouse website (an exemplary website, one that makes screen reading very easy) and, using the screen reader, complete the following task: find a braille watch, and order two. The task is over once you confirm that the watch has been placed in the shopping cart, you have changed the quantity to 2, and you have activated the button to check out. You don't need to fill in anything on the checkout screen. Make a note of the price of the watch (while still blindfolded) for your report.
    Screen Reader Tools
    There are many screen readers available, but the most popular one is JAWS (Job Access With Sound), which works only on Windows PCs. You can download a free trial, or you can make arrangements to use a JAWS-equipped computer in the Adaptive Technology Lab at McKeldin Library. Alternatively, you can use a browser add-on or the screen reader available from the technology office.
    JAWS - McKeldin Library Adaptive Technology Lab. Contact Dan Newsome (wdn@umd.edu) or call the lab (301-314-7958) to make an appointment.
    JAWS - download free trial from Freedom Scientific
    Window-Eyes - available from UMd's Terpware free of charge (with some requirements). If you choose to use this, Dan at the AT Lab would be particularly interested to know about how the download and install process works for you (he has heard that some people have had problems with that).
    Non-Visual Desktop Access (NVDA) - a free open-source screen reader for Windows from WebAIM
    VoiceOver - a screen reader that comes installed on new Macs. See the article from WebAIM for help getting started.
    ChromeVox - screen reader for Chrome browsers and OS.
    Evaluation Report (66 points)
    Title and Background (1 point)
    Title (IA05: Accessibility Observation)
    Names of both partners
    Name of screen reader
    Computer it ran on (AT Lab or, for personal computers, the OS)
    Browser used
    Overall Experience (5 points)
    Briefly recount your experience during this project, without going into any detail about the two tasks. In particular, I am interested in your experience installing and activating the software (if you used your computer and not the one in the AT Lab), what you think about the utility provided by screen reading software, and/or something that you will do differently in the future when you build websites.
    Task Completion Experience (20 points)
    Describe your experience, answering the following questions:
    URL of the website you visited
    describe the task you completed on the site
    how much time did you spend on the task, from initial blindfolded interaction to completion?
    how accessible do you think this website is?
    principle source(s) of difficulty due to website design and construction
    Problems and Remediation(20 points)
    Specify three problems that you found where the website is not as accessible as it should be. Describe each problem, the cause of the problem from a web design or implementation perspective, and a solution that would increase the site's compatibility with screen readers. Include a screen shot with annotations to point to the areas where the problems occurred.
    Exemplary Site Experience (20 points)
    Describe your experience, answering the following questions:
    dis you have any difficulties completing the task (other than lack of experience with the screen reader)?
    how much time did you spend on the task, from initial blindfolded interaction to completion?
    how accessible do you think this website is?
    how does this site compare to the other one you looked at, from an accessibility perspective?
    Submitting
    Upload your report to Canvas as a PDF by the due date. BOTH partners should upload the SAME REPORT (to keep Canvas happy and make grading easier).

    (view changes)
    6:22 pm

Monday, November 23

  1. page space.menu edited ... IA03 Project Pitch - 9/6 IA04 Project Pitch Vote - 9/8 ... Study - tbd 12/10 IA06 Prot…
    ...
    IA03 Project Pitch - 9/6
    IA04 Project Pitch Vote - 9/8
    ...
    Study - tbd12/10
    IA06 Prototype Tool Review - 11/1
    IA07 Project Critiques - 11/12
    (view changes)
    5:07 pm

Sunday, November 15

  1. page TA08 Usability Testing & Report edited ... The Report-Section 1: Abstract1. Abstract The abstract should provide a 5-7 sentence overview…
    ...
    The Report-Section 1: Abstract1. Abstract
    The abstract should provide a 5-7 sentence overview of your report including: (i) a description of your application, (ii) a description of your three primary tasks, (iii) a description of your evaluation method, (iv) a summary of your primary findings, and (v) design implications for future revisions.
    ...
    Prototype Implementation and Video
    Describe your prototype implementation:implementation (TA07): what language
    ...
    readings, discussions).
    Briefly discuss your video-making process (TA09). I realize that your video may not be completed when you turn in this report, but you can tell me what equipment and software you used or plan to use, how you settled on the idea, who worked on what, and so forth.

    The Report-Section 3. Task Descriptions3. Task Descriptions
    Describe your three primary tasks (iterate from previous incarnations) This should be a four paragraph section. The intro paragraph summarizes the three tasks. Each subsequent paragraph should start with a style Heading 2 with the name of the task and a more lengthy description. The descriptions should include screenshots of your actual interactive prototype in the style of either: a sequential storyboard, a state transition diagram, or a branching storyboard (see this Greenberg reading).
    (view changes)
    5:53 pm
  2. page TA08 Usability Testing & Report edited Deadline: Deadline: Thursday, December Points: This assignment is worth 70 points (about 16%…

    Deadline:
    Deadline: Thursday, December
    Points: This assignment is worth 70 points (about 16% of your team project, 7% overall)
    Assignment Overview
    (view changes)
    5:50 pm
  3. page TA09 Final Video edited Deadline: Deadline: Tuesday, December Points: This assignment is worth 40 points (about 9% o…

    Deadline:
    Deadline: Tuesday, December
    Points: This assignment is worth 40 points (about 9% of your team project, 4% overall)
    Assignment OverviewAssignment Overview
    (view changes)
    5:49 pm
  4. page TA09 Final Video edited Deadline: Tuesday, December 15, 7:00AM (Final Exam). Points: This assignment is worth 40 poin…

    Deadline: Tuesday, December 15, 7:00AM (Final Exam).
    Points: This assignment is worth 40 points (about 9% of your team project, 4% overall)
    (view changes)
    5:48 pm
  5. page TA09 Final Video edited ... December 15, 8:00AM 7:00AM (Final Exam). Points: This assignment is worth 40 points (abou…
    ...
    December 15, 8:00AM7:00AM (Final Exam).
    Points: This assignment is worth 40 points (about 9% of your team project, 4% overall)
    Assignment OverviewAssignment Overview
    (view changes)
    5:48 pm
  6. page TA09 Final Video edited ... Points: This assignment is worth 40 points (about 9% of your team project, 4% overall) Assign…
    ...
    Points: This assignment is worth 40 points (about 9% of your team project, 4% overall)
    Assignment OverviewAssignment Overview
    ...
    slot (8:00AM-10:00AM May 16). TheDecember 15 in CISC 2117). You supply the video, I will supply the food.
    The
    videos should
    ...
    of Q/A. You supply the video, I will supplyuse a time and be very strict about staying on track, so that every team has the food.same amount of time. AND we will start PROMPTLY at 8 am -- though I advise you to get there a little early to get some coffee and breakfast, and a good seat.
    Have fun with this assignment. For many of you, it will be your first video assignment at UMD. Pacing is important. Like everything in this class, brainstorm, sketch, and prototype ideas and get feedback before committing effort to one idea in particular. Use music. Use humor (but be professional).
    The video should be a brief but comprehensive look at your application and all the work that you've done this semester.
    Required PiecesRequired Pieces
    Each video must include:
    A title screen with tagline
    ...
    Bunch style). Please be sensitive to your teammates feelings about disclosing personal information in a video (even if the YouTube settings are set to unlisted).
    A narrative
    ...
    your application (skits seem to work well here).
    An exposition of your three primary tasks, which can either be done via a skit or as simple walkthroughs of your prototype. This is up to you.
    A brief comparison to the most relevant competing applications
    A historical look at the evolution of your designs from initial ideas during the "pitch" period to the sketches and storyboards to the final interactive prototype (and everywhere in between) along with critical learnings from your user testing and design critiques. Try to be creative with this retrospective and focus on key points! Think about your audience, what are the key take aways from your design evolution? How did things change and why?
    ...
    prototypes) though I realize some ofideally you maywill have recorded video of this as well.along the way.
    You should
    ...
    TA05 and TA07TA08 user testing
    What to DoWhat to Do
    Begin by brainstorming with your group on how you want to structure the video. Do you want a skit? How do you want to show the three tasks? Look at the example videos below for inspiration.
    ...
    For recording your final interactive prototype, you can use an external video camera (e.g., to capture both the user, the interactions, and the screen at the same time), screen recording software (e.g., Quicktime or TechSmith's Camtasia Studio), or some combination of the two.
    Edit the video (and shoot more video as necessary, the process will likely be iterative). You can use whatever tool you want to make the video include Windows Live Movie Maker, Adobe Premiere, Adobe After Effects, Final Cut Pro, iMovie. Recall that UMD students get free access to Adobe tools (link).
    ...
    video by 10AM7AM on May 13th (e.g., via Dropbox, YouTube, Vimeo).December 15th on Canvas. If you
    ...
    publicly viewable.
    Discuss

    Briefly discuss
    your video
    ...
    process in TA09.TA08 - Usability Testing and Report.
    Some Example VideosSome Example Videos
    These example videos are meant to give you ideas and to inspire your creativity. As you watch them, remember that they are from different intro HCI courses, and do not fulfill all of this assignment's criteria.
    (view changes)
    5:48 pm
  7. page TA08 Usability Testing & Report edited Deadline: Thursday, December 10 (last class) Points: This assignment is worth 70 points (about…

    Deadline: Thursday, December 10 (last class)
    Points: This assignment is worth 70 points (about 16% of your team project, 7% overall)
    ...
    Begin by reading over TA07 Final Interactive Prototype and TA09 Final Video. Operationally, I suggest you assign "leaders" within your team to manage different components of TA07, TA08, and TA09 so that nothing falls through the cracks and to divide labor more effectively. While I expect that each individual member should contribute to all three assignments, I think having a specific leader will help organize the remaining tasks.
    Once you implement the three primary tasks for TA07, you must perform usability tests with three users. Ordinarily, this would be done with "target" users pulled from your expected user population; however, in this case, you can use members of this class who are not on your team (this is a rule change compared to the paper prototype test). It is an interesting experience being on the subject side of usability testing, and I'd like for class members to have a chance to have the experience. That said, I strongly encourage you to test with at least one non-classmate (and it's perfectly fine if none of your test subjects are classmates). If you would like to recruit from the other CMSC434 section, please contact Dr. Vibha Sazawal (vibha@umiacs.umd.edu). Each user testing session must be done with at least two experimenters present (one to conduct the experiment and one to record observations/take notes). Before you conduct any test sessions, you must...
    ...
    refer to the Usability Testing reading.Lazar, Feng and Hochheiser, Research Methods in Human-Computer Interaction. The user
    After you write your study plan, test it out on a member of your team. This pilot test should follow the same protocol that you use for your three external users (i.e., it is a dress rehearsal for the real thing--try to make it as real as possible). You will include a section on how you conducted this pilot test, the results (if any), and how you changed your study protocol as a consequence (if you did). See "The Report" below.
    Now you're ready for the real usability tests. At a high level, each team should roughly follow this protocol:
    ...
    After the informed consent process, you can begin testing your final interactive prototype. You might want to start with a brief "pre-study survey" asking basic demographic question (e.g., age, profession, computer proficiency). You will test each task in order. You can either read the task description aloud or have the participant read it him/herself. While executing the task, ask the participant to "think-aloud" (recall the think-aloud strategy we talked about earlier in the semester). One experimenter should record notes about how the participant is using the prototype, the problems/successes encountered, and any comments made during the testing session, as well as observations about non-verbal cues such as body language, facial expressions, and utterances. The experimenters should also record quantitative data such as how long each task took and the number of "problems" the user experienced. Include the qualitative and quantitative data notes for each participant in the appendix.
    After the three primary tasks have been tested, provide your participant with a short paper survey (called a "post-study survey") with a few questions inquiring about their overall impressions of the application and any specific comments for improvement. It's good to include at least one open-ended question where they can make any comment they want. Again, scan in and include these responses in the report appendix.
    ...
    survey data). Again, see the RR9 reading (e.g., Section 10.4.8). Your report
    ...
    design revisions. See Section 10.4.8 in Lazar, Feng and Hochheiser, Research Methods in Human-Computer Interaction for more information.
    The ReportThe Report
    Submit a report of no more than 5 pages of text (approximately 2,500-3,500 words in a font similar to 11 pt. Times). Images, tables, figures, etc. are strongly encouraged, do not count against the page limit, and are thus effectively free.
    Please construct your report with the following full sections and start each section with the header below (style Headings 1); it makes it easier for us to grade. So, for example, your first section should have the heading 1. Abstract and the next 2. Prototype Implementation.
    The Report-Title PageTitle Page
    ...
    each of TA06, TA07, TA08, and TA08 (and TA09, the optional website, if you are doing that).TA09. If there's
    The Report-Section 1: Abstract1. Abstract
    ...
    5-7 sentence (1/2 page) overview of
    The Report-Section 2. Prototype Implementation2. Prototype Implementation
    Describe your prototype implementation: what language did you use? Why? How did you build it? What libraries, if any, did you use? How were your design decisions informed? Did you rapidly iterate and test as you built the system? What interactions/features did you leave out but hoped to build? Justify your design decisions based on course content (e.g., lectures, readings, discussions).
    The Report-Section 3. Task Descriptions3. Task Descriptions
    Describe your three primary tasks (iterate from previous incarnations) This should be a four paragraph section. The intro paragraph summarizes the three tasks. Each subsequent paragraph should start with a style Heading 2 with the name of the task and a more lengthy description. The descriptions should include screenshots of your actual interactive prototype in the style of either: a sequential storyboard, a state transition diagram, or a branching storyboard (see this Greenberg reading).
    ...
    experiences of TA05,TA06, etc.). You
    ...
    reference your TA08TA09 Final Video
    The Report-Section 4: Usability Tests4. Usability Tests
    ...
    Heading 2. See the RR9 "Usability Testing" reading.
    Describe how
    ...
    the group (or friends/family) and the
    Describe how you recruited participants and their demographics (relevant to the project).
    Describe your study method, which includes how you performed your study including the study protocol, the location, the length of the study, and a description of the data collection instruments (e.g., the video recording setup, the post-study survey).
    ...
    The Report-Section 5: Results from User Testing5. Results from User Testing
    This section should detail the primary results from user testing and implications for design.
    Provide a table of the quantitative results of your testing.
    Summarize your qualitative observations, with a high-medium-low severity level for each one (for positive results, just note "good" or "positive" as the severity level).
    Discuss the quality of your overall interface design. Identify limitations in the design (some of these may be due to the initial scope of the project). Identify design elements that work particularly well.
    Provide recommendations for the next logical steps. Do you feel your design has been validated? Are minor adjustments needed, but the overall approach is validated? Is the concept still worth investigating but serious problems have been identified? Based on this conclusion, what are the logical next steps that you would recommend be done on this project?

    The Report-Section 6: If We Had To Do It All Over Again...6. If We Had To Do It All Over Again...
    I wantReflect upon the design process you to reflect on whether you thinkfollowed in this project. What aspects of the human-centered, iterative design process we usedworked well? What aspects did not work so well? It may be effective to do this semester actually improvedas a brainstorm session with your final interactive prototype. Why or why not?group. For example, you could start with questions like:
    What were the most significant ways in which the design concept and the actual interface design changed under the influence of user involvement?
    What wouldwere the biggest surprises for you—the things you do differently if you could? Dolearned from or about users that you think it would be better to trynot have predicted based on your own experience and code up simple designs early (say, starting from week 1)intuition?
    Did the methods you chose for your evaluation
    and focus on improving these?prototyping get at what you were looking for? In hindsight, would a different approach (process, not specifics of your interface) have been better?
    What were the most and least valuable among the methods you used, either generally or specifically for your project?

    Appendix
    The appendix should include the following. Each of the following bullets should start on their own page and should be titled Appendix A: <title>, Appendix B: <title>, etc. in Heading 1 style.
    (view changes)
    5:37 pm
  8. page TA08 Usability Testing & Report edited ... Points: This assignment is worth 70 points (about 16% of your team project, 7% overall) Assig…
    ...
    Points: This assignment is worth 70 points (about 16% of your team project, 7% overall)
    Assignment Overview
    ...
    conjunction with TA06TA07 Final Interactive Prototype and TA08TA09 Final Video. Together, TA06,TA07, this assignment (TA07)(TA08) , and TA08TA09 represent the
    ...
    viewing your TA08TA09 final videos).
    While TA06TA07 and TA08TA09 focus on
    What To DoWhat To Do
    ...
    components of TA06, TA07, TA08, and TA08TA09 so that
    ...
    through the cracks.cracks and to divide labor more effectively. While I
    ...
    must perform a usability testtests with three
    ...
    population; however, given the constraints of this class, I am looseningin this restriction. Thus,case, you can
    ...
    your team for usability tests (this is
    ...
    least one non-CMSC434 studentnon-classmate (and it's
    ...
    are classmates). If you would like to recruit from the other CMSC434 section, please contact Dr. Vibha Sazawal (vibha@umiacs.umd.edu). Each user
    ...
    to collect, and the interview/survey
    ...
    plan to use.use, and the informed consent form you will have participants sign. For example,
    ...
    this, refer back to the Usability Testing reading assignment (RR9) and to the sample protocol I posted from my work at NCI. In addition, thereading. The user testing
    After you write your study plan, test it out on a member of your team. This pilot test should follow the same protocol that you use for your three external users (i.e., it is a dress rehearsal for the real thing--try to make it as real as possible). You will include a section on how you conducted this pilot test, the results (if any), and how you changed your study protocol as a consequence (if you did). See "The Report" below.
    Now you're ready for the real usability tests. At a high level, each team should roughly follow this protocol:
    Recruit participants. You will have to describe your recruitment methodology in your report.
    ...
    with TA05, download and modify this IRB "informed consent" template to fit your project [link].have participants sign an informed consent. Make sure
    ...
    video recording of the session,
    After the informed consent process, you can begin testing your final interactive prototype. You might want to start with a brief "pre-study survey" asking basic demographic question (e.g., age, profession, computer proficiency). You will test each task in order. You can either read the task description aloud or have the participant read it him/herself. While executing the task, ask the participant to "think-aloud" (recall the think-aloud strategy we talked about earlier in the semester). One experimenter should record notes about how the participant is using the prototype, the problems/successes encountered, and any comments made during the testing session, as well as observations about non-verbal cues such as body language, facial expressions, and utterances. The experimenters should also record quantitative data such as how long each task took and the number of "problems" the user experienced. Include the qualitative and quantitative data notes for each participant in the appendix.
    After the three primary tasks have been tested, provide your participant with a short paper survey (called a "post-study survey") with a few questions inquiring about their overall impressions of the application and any specific comments for improvement. It's good to include at least one open-ended question where they can make any comment they want. Again, scan in and include these responses in the report appendix.
    ...
    The Report-Section 6: If We Had To Do It All Over Again...6. If We Had To Do It All Over Again...
    I want you to reflect on whether you think the iterative design process we used this semester actually improved your final interactive prototype. Why or why not? What would you do differently if you could? Do you think it would be better to try and code up simple designs early (say, starting from week 1) and focus on improving these?
    The Report-Section 7: Video Making Process7. Video Making Process
    Include approximately 2-3 paragraphs on your video making process for TA08. How did you divide up the work? What editing software did you use? How did you decide on your particular approach? How did you record the actual screen interactions? Note: Because this text is part of TA08, it does not count toward the 5-page limit.

    Appendix
    The appendix should include the following. Each of the following bullets should start on their own page and should be titled Appendix A: <title>, Appendix B: <title>, etc. in Heading 1 style.
    ...
    Raw notes from the three user testing sessions (notes should be clearly marked with a timestamp and session number)
    A scan of the post-study paper survey responses (these should also have session numbers)
    Scanned, signed informed consent forms
    (view changes)
    5:15 pm

More