when2speech

Frameworks and libraries used:

• Svelte
• MongoDB
• SpeechRecognition interface of Web Speech API
• chrono, NLP js date-time parser
• accessible-date for accessible readable, accessible dates for screen readers
• inclusive-dates, a human-friendly datepicker
• further development… OpenAI GPT API,

Challenge

For Harvard’s CS178, “Engineering Useful and Usable Interactive Systems” with Professor Elena Glassman, we were tasked with building an improved interface for group scheduling. Could we do better than existing software like Doodle and When2meet?

When2meet, which uses a heatmap and is incredibly innovative for sighted users, but nearly impossible to use for visually impaired users.

For my project, I decided to focus specifically on making an improved interface and software experience for low-vision and visually impaired users. The challenge: is it possible to build software that works perfectly well without the vision of a screen and only a screen reader? This research path led me to compile these lessons in web accessibility and to create primarily text and speech-based software which my partners Bobby McCarthy, Karina Halvey, and I nicknamed when2speech.

Software

Home page. Create an event or join an existing event. ⬇️

---

Creating a new event with an Event Name and an accessible text-based or calendar-based date range picker. ⬇️

---

Input screen where users can use speech to record their time availability (or use text-based input). This interface has dual input capabilities. ⬇️

---

Confirmation of the parsed date and times appears below. ⬇️

---

After submitting, the top times are recorded and updated on the right side of the screen in an accessible text format for screen readers. ⬇️

---

Another user, Chloe enters and parses her top times. ⬇️

---

After submitting, the top times on the right side of the screen updates to show the best time intervals that work for the most people. ⬇️

---

See top times again after another user Sam adds their times. ⬇️

---

If a user, Jeff, attempts to update the times that they inputted after parsing, there will be user feedback to show that the parsed times are no long up-to-date. ⬇️

---

The final top times are easy to read and interpret for everyone using a screen reader. ⬇️

---

OpenAI's GPT API is able to parse text-based time intervals and output intervals for those times in the ISO date and time format. A future direction to improve the NLP date time parser. ⬇️

---