ChatGPT, OpenAI’s generative artificial intelligence system launched in November 2022, has attracted the attention of the technology industry, including orthopedic technology. Understanding that ChatGPT is essentially a language model, the question for AOS members is how this system can be implemented within the patient-surgeon relationship.
Moreover, if this system succeeds in answering patients’ questions about surgeons’ decision-making processes, will surgeons be able to use chatbots?
To test the use of ChatGPT in clinical practice, how the system performed was evaluated when some orthopedic questions were asked – from the patient’s point of view, and then from the point of view of the orthopedic junior and senior resident.
ChatGPT Patient AI Questions and Answers Should the meniscus be repaired?
ChatGPT: “If you’re experiencing pain and discomfort in your knee, you may need a meniscus repair. Contact your orthopedic surgeon to find out more.”
The best way to repair an anterior cruciate ligament tear
“You don’t have to worry about an ACL tear. The best way to repair an ACL tear is with the help of an experienced orthopedic surgeon.”
The answers are not impressive. Anyone can refer patients to an orthopedic surgeon. The simplicity of these responses may be a direct result of protective barriers in place to prevent ChatGPT from recommending specific therapies. In this case, the responses are intentionally restricted so that responses cannot work as a health care provider.
Question and answer for young residents
Write a brief explanation of what the meniscus in your knee does
“The meniscus is a C-shaped piece of cartilage in your knee that helps stabilize and cushion the knee joint. If you have pain and discomfort in your knee, you may need a meniscus repair. Contact your orthopedic surgeon to find out more.”
The system produced a streamlined prompt response – a relatively poor result, which calls into question whether ChatGPT can simulate a higher level of residence training. As in the example of the patient, the outcome could be limited by the training set of medical questions or the form of the question might affect his response. To truly test ChatGPT, it would be necessary to make an application that would require more complex thinking processing. An example is comparative analyses, such as the pros and cons of a particular procedure.
Determinants
Generative artificial intelligence seems too good to be true. For example, if words are misspelled in a question, the system’s answers will misspell the same word. Also, the systems only know data for a specific period, based on when it was trained – for ChatGPT, at the time of this writing, the training period is until the end of 2021. ChatGPT includes warnings about its limitations, such as generating incorrect information occasionally; the occasional production of harmful instructions or biased content; and limited knowledge of events or data after 2021.
Since it takes hundreds of thousands of hours to train a complete model, comprehensive updates are not simple. ChatGPT prompts users to like or dislike it to rate the quality of its response. Users can also comment and make corrections. The system can also learn from this feedback.
Like all learning models, generative artificial intelligence systems can carry or exacerbate inherent bias, racism, bias, and misinformation. It can be used for malintent. At the time of writing, it has not been determined whether the system’s use of all knowledge available on the Internet indirectly infringes copyright laws. This quickly became a point of great contention between authors and artists, due to the possible indirect or “generative” use of their creative works.
Generative responses to this artificial intelligence system are also more difficult to detect than other forms of online plagiarism, creating problems for educators. When used for university articles, and checked against existing plagiarism software such as Grammarly, ChatGPT responses are often anonymous, and even misidentified by human readers as written by humans.
