Test for Accessibility Challenge

 

The WCAG (Web Content Accessibility Guidelines) are a collection of guidelines aimed at improving the web experience for people with disabilities or limitations.

In this challenge, we will start to look at areas of the Topcoder platform to discover accessibility shortfalls that prevent those with disabilities from being able to compete in our challenges.  This is the first in a series of challenges focusing on making the site fully accessible.  Persons with disabilities are encouraged and very welcomed to compete.

There is a 48hr study period for this challenge.  You should become familiar with the accessibility guideline manual (WCAG), creating issues in JIRA and download the necessary tools for this challenge.  

Issue reporting will open 48 hrs after the challenge starts.  Please do not report any issues in JIRA until then.  They will not be accepted.  The announcement will be made in the forum when the time comes.  So, please watch the forum.

 There is no coding or designing needed in this challenge.

 

Challenge Objectives

• Exploratory testing

• Desktop site focusing on Competitor Flow (Edge, Chrome, Safari, Firefox)

• Welcome those with disabilities to compete in this challenge

 

Assets

• You will use the WCAG as a reference when looking for issues to report:

               https://www.w3.org/TR/WCAG21/
   
           These guidelines will hold detailed knowledge of what to look for while testing for accessibility

 

• Also reference this google sheet to see the potential method you can use to test each portion of the WCAG and also to get a shorthand version of the guidelines:
  
  https://docs.google.com/spreadsheets/d/1rlzDAdRWSJzYcTyyf7RQIZLSEc5-Y35mFjNhmGMAmsM/edit?usp=sharing

 

What to Test (Scope)

The following areas are in scope for testing:
 

Log in & registration pages

https://accounts.topcoder.com/member

https://accounts.topcoder.com/member/registration

 

Challenge feed

https://www.topcoder.com/challenges
 

Challenge specification page/Registration/Submission workflow

 A sample challenge has been set up for testing of the registration and submission workflow.  Please refer to the forum posting 'Sample Challenge Link'.          

 Please only test at the sample challenge. Forums are currently out of scope for accessibility testing.
 

• Accepted types of bugs: WCAG Categories: Perceivable, Operable, Understandable,Robust.  Accepted subcategories are all listed in the JIRA field drop down

• Primary target device: Desktop

• Browser requirements: Latest Google Chrome, Firefox, Safari and Edge on Windows/macOS. IE11 and below versions are out of scope. Safari on Windows is out of scope.

• Screen Reader Requirements:  

              There are many free screen readers available to choose from.  We will mainly focus on compatibility with the following:
 

               JAWS- If you previously already have it

NonVisual Desktop Access (NVDA) for Windows

VoiceOver for Mac OS

Any screen reader included with OS is acceptable to use.

 

• Accessibility Audit Tool Requirements:

               This tool is used to check web pages for general accessibility compliance.  We are only focused on
             following WCAG guidelines.   

             These free tools are acceptable to use for testing for this challenge:
 

                508Checker - http://www.508checker.com

               Accessibility Checklist - http://romeo.elsevier.com/accessibility_checklist/ 

               CynthiaSays - http://www.cynthiasays.com/
 
   If there are tools other than those listed that you have found and would like to use, please request it first in
   the challenge forum.  
 

• Any issues other than those that don’t follow the WCAG are OUT OF SCOPE. 

• Also other minor UI, Usability, Content issues, Tooltip, Broken Link issues that are not affecting the accessibility of the platform are likely to be REJECTED.

• Important: Because you are testing a live website DO NOT Test/send data via contact forms. They are out of scope.

• An edge case would be anything that does not reflect typical user behavior. They are accepted in accordance to the impact to the end-user and based on the workarounds available.

 

How to Participate

    Jira: Please request access to Jira in the forums if you have not received an email invite.  
            A help link with a tutorial video is posted in the forum.

    One Drive:  Please submit all screenshots and video to the OneDrive link posted in the forum. 

 

Issue Weights and Scoring

• Scoring will be based on the number of points per bug.

• Only verified issues will be counted.  Tickets created for enhancements or that are not bugs will not be counted. Duplicate issues will be closed and not counted. Log issues according to the guidelines above. Issues that do not follow these guidelines may be rejected due to lack of information.

• For challenge scoring, the user with the most verified points will be selected as the winner. If two users submit the same issue, the user that submitted the issue first will receive credit.

• Please focus on accessibility testing based on the requirements of the WCAG, all bug reports based on your own assumptions will be rejected.

• Bug found with use of screen reader                          - 2 Points

• Manual bug found or use of an audit tool                   - 1 Point

Submitters that do not take 1st, 2nd or 3rd place will be paid $5 for each non-duplicate and verified issue up to a maximum of the 3rd place prize. 

 

Important Notice

• Follow the standard topcoder Bug Hunt Rules.

• If you do not properly document your bug reports, they will likely be rejected due to lack of information or documentation. If you submit the same bug in multiple areas/pages, (for instance, Same validation issue of a form can be found in different pages/sections) you will likely get credit for the original bug report only. The others will all be closed as duplicates.

• Please don’t make adjustments or change labels of existing issues logged by other competitors.   

• DON'T RE-OPEN the issues in the review phase and anyone who RE-OPENS a ticket will be disqualified from the challenge.

• DON'T create the same issue on different platforms; instead, merge them into one; All the others will be marked as Duplicate.

• If you see multiple broken links on the same page, combine them into one ticket. Others will be marked as DUPLICATE.

• You must not edit the bug report once created, so make sure you enter all the details at the time you create the issue, otherwise, your issue will be moved to the end of the queue. If you really need to edit an issue you must use the comments section for this (i.e. add a comment to describe any changes you want to make to the issue), and we'll decide whether the changes are major enough to move the issue to the end of the queue. You are allowed to add screen shots in the comments section though, assuming your issue report contains all the details when created.

• You must specify the test data you have used in the 'Reproduction Steps', All the issues will be marked as 'Incomplete', if the correct test data is not provided.

• Keep an eye on the issues being submitted by other participants to minimize the time you may be spending on duplicate efforts. Knowing what has already been reported will allow you to better focus your time on finding yet undiscovered issues.

• There will be no appeals phase. The decision of PM/Copilot for validity and severity of each filled issue will be final.

 

 

Final Submission Guidelines

 

For each report of a limitation or bug, we need the following information:

• Device- This type of device used.  In this case, only laptop/desktop is acceptable

• Operating System- OS you are using.  In this case, either Windows or macOS

• Browser- Latest version of Chrome, Firefox, IE, Edge, Safari is acceptable

• Be sure to put the onedrive link in the appropriate field in JIRA.

• Screen Reader:  Screen Reader you used

• Accessibility Audit Tool Used: Accessibility Audit Tool you used

• WCAG Category: 4 categories- choose from the dropdown

• WCAG Success Criterion: Multiple categories- Choose only 1

• Steps to reproduce, including any additional information (Must list all the steps that need to reproduce the bug, DON'T list only the URL without test data)

• Actual result: What actually happens

• Expected results: What should happen

• Testing collateral: In One Drive,  upload the

  1. Video*, and/or

  2. Screenshot(s)

  3. Console log

  4. Screenshot of OS version

  5. Screenshot of browser version

  6. Any other supporting documents to the OneDrive link listed in the forum
     Be sure to put the onedrive link in the appropriate field in JIRA.
     
     
     ***A duplicate report for the same bug on a different browser is not allowed.*** 
     
     * Sometimes, including a video is necessary...such as when you're esing screen reader compatibility.
      

When you are done with your reporting, please submit a .txt file in Direct using the "Submit" button before the submission phase ends. In this file please include the following details:

• TopCoder Handle: <>

• Jira Username: <>

Important Notes:

• DON'T use any other link to create new issues or submit as document, they won't count and won't be paid.

• DON'T use youtube or public shared location for uploading your videos and screenshots.

 

- ALL THE SUBMISSIONS WITHOUT ABOVE INFORMATION WILL BE REJECTED AND WON’T BE PAID.

- IMPORTANT: Submit the above details before the Submission Phase ends. If you can't submit due to technical difficulties within the Submission Phase please email your submission with above details to support@topcoder.com.

- Participants who haven't submitted will not be paid.

- DON'T use any other link to create new issues OR submit as document, they won't count and won't be paid.