I set up a ChatGPT account and asked it to solve the lily pond problem. If the lilies double every day and will cover the pond in 30 days, on what day do the cover the half the pond. The answer, of course, should be day 29. ChatGPT correctly told me in words that this as an exponential growth problem, then gave me a numerical answer of 15 days (the linear growth answer!). Then it gave me some completely wrong math involving logarithms, after which it gave me two additional different answers that were not 29 or 15, and didn’t seem to acknowledge that it had even given multiple answers.

What scared me most was not the wrong answer(s), but the extremely confident manner in which it gave the wrong answer(s). This could fool people on problems where they don’t know the answer in advance, and the correct answer is not intuitive or obvious.

ChatGPT does tell much better knock-knock jokes that Siri…

As of today, we do not want this thing designing bridges or airplanes or anything else! I do not want it advising my doctor on my course of treatment, mixing my prescriptions at the pharmacy, or managing my retirement account, although these seem like plausible near-future applications once some kinks get worked out. It’s easy to be dismissive of the current state of this technology. But at the same time, it may not be that far off. Right now, it could argue you to a standstill in a barroom political or philosophical debate (you know, where a drunk guy makes a lengthy argument that is as illogical as it is confidently delivered, and since there are no consequences you just give up). In the medium-term future, I could imagine it being a conversational companion for a child or a person with dementia (although, there are some obvious ethical concerns here.) Could it be a best friend or significant other? This is a bit disturbing, because it might be able to always tell you exactly what you want to hear and be 100% impervious to your own annoying quirks, and then you might forget how to deal with actual people who are not going to be so forgiving. It could inhabit a sex doll – now there is a truly disturbing thought, but it will happen soon if it has not already.