MightyFields Voice

Voice-based Mobile Application Interaction

Entering extensive statements on touch–screen devices can be tedious and time-consuming. There are various cases in which field workers have difficulties fulfilling the forms and would benefit from using hands-free data entering. In some cases, fulfilling the forms with a voice interface can even be safer than typing out all the information on a mobile device. In addition, the voice interface can support the inclusivity of (often older) workers with low vision.

MightyFields Voice (MFVoice) simplifies fieldwork and makes it as efficient as possible. MF Voice serves as an extension of the MightyFields app, by supporting in–app voice interactions. Users can issue voice commands for transitioning between application screens and filling out forms.

Who can MFVoice assist?

Work in hazardous conditions – It is not always easy to use a mobile device when working with high voltage machinery, on heights or with dangerous materials. MFVoice helps employees collect data more efficiently while also enhancing employee safety.

Individuals with low vision and/or disabilities now have the option to collect data with ease. For example, it can be difficult for someone with poor vision to enter data on a small mobile device, and it would be much easier for them to use voice commands.

Technical overview:

MFVoice is composed of three components which can also be seen in the figure on the right. The first component is an extension to the MigthyFields mobile app. This extension waits for the users’ voice commands, and when they are issued, the context of the application screen is collected. The context is the collection of buttons, links, input fields and selection elements. Afterwards, the collected data is sent to Google’s speech–to–text service, which exclusively converts captured voice commands to text.

In the second step, text and context are sent to our voice action recognition service, where, based on text and context, the algorithm predicts the action the user wishes to do.

In the last, third step, the predicted action is sent back to the extension, which executes the command.