## 2 A Brief History of Intelligent Personal Assistants

## 2.1 Timeline View of the Intelligent Personal Assistants History

Intelligent Personal Assistants (IPAs) have a long history of development. We depict the rough timeline of the IPA history in Figure 2. The development progress can be divided into four stages, each marked with a unique color in the figure.

The 1st stage spans from the 1950s to the late 1980s, which is mainly about the development of speech recognition techniques. The early stage of speech recognition started from basic digits and words. Bell Laboratories developed 'Audrey', which could recognize numbers 0-9 with about 90% accuracy. In 1962, the 'shoebox' [8] system came out from Advanced Systems Development Division Laboratory at IBM, which was capable to recognize for up to 16 words. From 1971 to 1976, the Speech Understanding Research (SUR) project, funded by the US Department of Defense, significantly advanced speech recognition technology. The Harpy system [9] was particularly representative, as it could understand sentences composed of 1011 words, equivalent to the proficiency of a three-year-old child. In 1986, IBM developed the Tangora speech recognition typing system [10], capable of recognizing 20,000 words and offering predictive and error-correction capabilities. The Tangora system utilized Hidden Markov Models [11], requiring individual speaker training for voice recognition, with pauses between each word.

The 2nd stage covers the period from the 1990s to the late 2000s, since speech recognition started to be integrated into software for certain advanced functions. In 1990, the 'Dragon Dictate' software [12] was released, which was the first speech recognition product for consumers. It was originally designed to work on Microsoft Windows, supporting discrete speech recognition. 'Speakable items' [13] was introduced by Apple in 1993, enabling users to control their

<!-- image -->

computer with natural speaking. In 1996, IBM launched 'MedSpeak' [14] for radiologists, which is also the first commercial product supporting continuous speech recognition. Microsoft integrated speech recognition into Office applications in 2002 [15], and Google added voice search to Google Mobile App on iPhone in 2008 [16].

The 3rd stage extends from the early 2010s. In this period, always-on virtual assistant services began to appear on mobile devices such as smartphones and personal computers. Siri [1], widely considered as the first intelligent personal assistant installed on modern smartphones, was integrated into Apple's iPhone 4S in 2011. Since its launch, Siri has remained a key built-in software for Apple devices, including iPhones, iPad, Apple Watch, HomePod and Mac, continuously undergoing updates and iterations to incorporate new features. Similar to Siri, many other virtual intelligent assistant started to appear in the period. In 2014, Microsoft released Cortana [17], and gradually integrated it into desktop computers and other platforms. Amazon released Alexa [3] in the same year, which could complete tasks such as voice interaction, music playing, setting alarms, etc. Beyond voice search, Google Assistant [2] was unveiled in 2016, supporting users to interact with both speaking and keyboard input.

The 4th stage started recently when LLMs start to draw attention from all over the world. Based on LLMs, there emerged many intelligent chatbots ( e.g., ChatGPT [18]), as well as some LLM-powered IPA software installed on personal devices ( e.g., Copilot [19]). The details of this stage will be covered in Section 2.2.4.