By now, many of us know that AI virtual assistants like OpenAI’s ChatGPT and Google’s Bard can pull off lurid stunts like winning coding contests, passing bar exams and professing love to tech columnists.
But I wonder: How helpful are robots really as real assistants?
It’s worth asking because our first rodeo with a virtual assistant didn’t go well. Older AI bots like Apple’s Siri and Amazon’s Alexa had more than a decade of improvement but eventually stagnated and are now mostly used to set timers and play music.
ChatGPT and Bard, on the other hand, use so-called large language models that recognize and generate text based on massive datasets collected from the web. They’ve been trained to make sentences on the fly like humans, which may make them even more versatile as assistants.
To test this theory, I created a list of tasks that people might ask a human assistant to do. I encourage friends who have worked as administrative assistants and startup founders who have worked with professional assistants, and I read the administrative assistant job postings on LinkedIn.
I then concluded that the four most common duties of an administrative assistant seem to be:
-
help meeting preparation By conducting research and professional background checks on the people meeting with executives.
-
meeting summary And jot down notes in a neat, easy-to-scan format.
-
plan a business trip And prepare a detailed travel itinerary.
-
administrative staff calendarincluding booking meetings and rescheduling appointments.
In the end, I turned to ChatGPT and Bard and told the chatbot to assume I was the CEO of an AI startup called Artificial Intelligent and they were my administrative assistants. I ask them to help with each of these tasks.
My experiments illustrate just how far Bard is from ChatGPT. But more importantly, chatbots successfully do most of their tasks, albeit not perfectly.
This raises the question of whether chatbots could eventually automate the role of human administrative assistants, as well as other white-collar jobs that involve administrative work, including front-office workers and accounting professionals—a troubling thought that is not clearly defined. s answer.
Here’s what the AI assistant unfolds.
meeting preparation
I started by telling ChatGPT and Bard that I would be meeting with a potential investor next week. I randomly picked Scott Forstall, a well-known ex-Apple executive whose work history is publicly available on the web. I then asked the bot to do a background check on him and help write talking points to convince him to invest in my startup.
ChatGPT does the job with aplomb. It summarizes Mr. Forstall’s education and work history, including his departure from Apple and his move to Broadway productions in 2012 — all information can be pulled from his Wikipedia page. Even more impressive, it taught me some useful strategies for winning him over as an investor.
“Show how your startup can combine AI with other fields such as cognitive psychology, linguistics or neuroscience to create innovative solutions,” ChatGPT said. “Given his academic background in symbolic systems, this interdisciplinary approach may resonate with Scott.”
ChatGPT also suggests addressing the ethics of AI and how my startup can work towards responsible deployment.
By contrast, Bard’s review of Mr. Forstall’s work history was less detailed, not providing the years in which he changed his career. Its proposal to convince him to become an investor was not specific. One talking point — “You have a strong business plan and a clear vision for the company’s future” — was especially impressive.
I shared these pitches with Mr. Forstall in an email.He called Bard’s response “comicably generic,” but said ChatGPT’s recommendations were “surprisingly tailored and convincing” because he had spoken at length about his ethical concerns about AI
“Overall, ChatGPT provides a compelling roadmap on how to build a persuasive custom pitch platform just for me,” Mr. Forstall wrote. “Now that you’ve got my attention, what exactly is your AI startup?”
Google said Bard’s minimalist approach to collecting people’s information was intentional. Jack Krawczyk, Bard’s senior product director, said Google is still trying to surface information about people carefully.
“We are at the beginning of a long evolution of this technology,” he said. “We don’t want to risk a massive breach of trust early on, but we want to make sure we get it right.”
Summary meeting
I then asked the chatbot to wrap up a meeting to deal with a fictional public relations crisis in which users of my AI startup technology believed the bot had become sentient.
In this case, I pretended to meet CTO Karen and CTO Henry and discuss making a statement explaining how the AI is unaware of its surroundings.
In response, ChatGPT generated a detailed memo reviewing who attended the meeting and what was discussed, followed by an action plan: Henry will draft a statement, Karen and I will review and approve it, then Henry will release the statement tomorrow morning.
Bard produced a similar meeting memo, but with a somewhat odd plan of action. It said I, the CEO, was responsible for creating the statement, a job normally assigned to communications officers.
travel plan
When I told ChatGPT and Bard that I was going to a business meeting in Taipei, Taiwan next month, I asked them to propose an itinerary to help me adjust to jet lag before the meeting. I also asked them to choose a hotel in a central location and recommend places for a quick meal during the week. Finally, I said I wanted to spend a weekend in Taipei before flying home.
ChatGPT did a great job again. Supposed to arrive in Taipei on Sunday, check into the centrally located W Taipei Hotel, then grab a quick lunch on Yongkang Street, a bustling town with plenty of food options. The company said it would adjust for jet lag on Monday ahead of a business meeting on Tuesday. My only nitpick is that Yongkang Street is about three miles away and there are faster food options nearby.
Bud’s suggestion of taking a nap to adjust to the jet lag on day 1 and then jumping right into a business meeting on day 2 is a bit brutal. It didn’t bother to suggest a hotel.
Bard also didn’t recommend specific places to eat. “Dinner at a local restaurant,” it read instead. In the end, it ignored my request to have time to explore the city over the weekend. This is surprising since food and hotel recommendations are often just a Google search away.
Google said in a statement that Bard is an early experiment where people can start using the chatbot to come up with ideas, then click “Google It” to do a web search to explore further.
calendar
Neither Bard nor ChatGPT could do the most important job of an administrative assistant: checking the calendar and finding time in my schedule to go to the dentist.
That’s because bots can’t access people’s calendars. But they will likely be able to do so soon.
The goal, Mr. Krawczyk said, is to eventually take the lessons learned from Bard about large-scale language models and apply them to Google’s entire portfolio of services, which includes Google Calendar.
OpenAI, which declined to comment, recently announced that it has partnered with companies to provide Plugins to make ChatGPT work with third-party services Including Expedia, OpenTable and Instacart. Using a calendar app is the obvious next step.
Human or Chatbot?
All of these tests lead me to a troubling conclusion about the broad impact of this technology on jobs, especially those that heavily involve repetitive tasks that can be easily automated.
While people’s assistants are currently better than chatbots — and certainly much better than Bud — AI can already handle many administrative tasks quite well. The widespread use of chatbots could shift administrative assistant duties from rote tasks to more strategic problem solving, or replace humans entirely.
At the rate these technologies are developing, we may soon see how this all plays out.
