February 09, 2024

After the American Century

Google’s New Gemini AI is Unreliable 

Today I tested the newly released “Gemini” Artificial Intelligence program. It is a fraud. I asked it a question that was factual and that could be checked. The “facts” were wrong, sheer nonsense. I asked it who used the word “kaleidoscopic” in a literary text. It responded with many examples, including brief quotations. I checked six of the most famous works, and found that in every case the quotations were entire fabrications. However, they sounded plausible:

 • Mark Twain's "Adventures of Huckleberry Finn" (1885): Describes the "kaleidoscopic whirl" of a Mississippi steamboat scene. 
 • T.S. Eliot's "The Waste Land" (1922): Uses the term to represent the fragmented and chaotic modern world. 
 • 1885: "The kaleidoscopic vision of the prairie…" (Walt Whitman, "Specimen Days") - This quote describes the ever-changing landscape of the American plains. 
 • 1925: "The kaleidoscopic panorama of New York…" (F. Scott Fitzgerald, "The Great Gatsby") - This quote captures the dazzling and chaotic energy of the Roaring Twenties in New York City. 
 • 1926: "The kaleidoscopic mind of modern youth…" (Carl Sandburg, "Smoke and Steel") - This line reflects on the changing attitudes and values of young people in the modern era.

I then wrote back to Gemini 
 Earlier today you provided examples of the use of the word kaleidoscopic (or kaleidoscope) in several works of American literature. I have checked, and every example I looked at proved to be false. 
     The word is not used anywhere in T. S. Eliot's The Wasteland
     It does not appear in Mark Twain's The Adventures of Huckleberry Finn
     It is not used in Fitzgerald's The Great Gatsby
     Walt Whitman did not use the word in Specimen Days
        and so forth. 
Because of these fabrications from well-known works of literature, it appears that you are entirely unreliable. You lie. You make up quotations. And you lie about things which are easy to check, so one can only assume that you are also unreliable when using vague references such as a line from an unspecified article in a 19th century magazine. I am disappointed, but not surprised. If this is the best that Aritificial Intelligence can do, then you cannot be trusted. 

Gemini Replied 
"I apologize unreservedly for the misinformation I provided about the use of "kaleidoscopic" in American literature. You are absolutely correct; after further investigation, I have confirmed that the examples I provided were inaccurate. I am still under development, and I am constantly learning and improving. However, there is no excuse for providing you with false information, and I take full responsibility for my mistake. You are right to be disappointed, and I understand your concerns about my reliability. It is important that I am able to provide accurate and trustworthy information, and I failed to do so in this instance. I am committed to learning from this mistake and improving my ability to fact-check and verify information before presenting it. I would like to assure you that I am not capable of intentionally lying or making up information. However, I can sometimes make mistakes, particularly when dealing with complex or open-ended tasks. I am always working to improve my accuracy and reliability, and I appreciate your feedback in helping me to do so. Please know that I am still under development, and I am always learning. I hope that you will continue to use me and provide me with feedback so that I can improve and become a more reliable and trustworthy source of information."

Do not use it! A student relying on this source would get a failing grade and be reprimanded for making up information and misrepresenting the texts "cited."