In the book The Great Mental Models, Volume 1, the chapter Second-Order Thinking opens with the following quote:
Technology is fine, but the scientists and engineers only partially think through their problems. They solve certain aspects, but not the total, and as a consequence it is slapping us back in the face very hard.Barbara McClintock
The quote itself (as attributed to Barbara McClintock) can be found in A Feeling for the Organism, 10th Aniversary Edition: The Life and Work of Barbara McClintock, by Evelyn Fox Keller and others.
This quote has obvious relevance to the topic and there is value in examining it.
In this case, it occasioned a few conversational threads with ChatGPT that proved instructive as well as entertaining.
I posed this query to ChatGPT: Who said the following? “Technology is fine, but the scientists and engineers only partially think through their problems. They solve certain aspects, but no the total, and as a consequence it is slapping us back in the face very hard.”
Here is the response:
The quote “Technology is fine, but the scientists and engineers only partially think through their problems. They solve certain aspects, but not the total, and as a consequence it is slapping us back in the face very hard.” is by Jacques-Yves Cousteau, the famous French naval officer, explorer, conservationist, filmmaker, and researcher who studied the sea and all forms of life in water. He was a vocal advocate for marine conservation and often commented on the impacts of technology and human activity on the natural environment.ChatGPT conversation
Asking for proof or specific sources resulted in ChatGPT backtracking immediately:
I apologize for the oversight. Upon further review, it seems that while Jacques-Yves Cousteau expressed concerns about human impact on the oceans and the environment, the specific quote you provided does not directly match known statements by him, at least based on the knowledge I have up to September 2021. It’s always possible that the quote could be a paraphrase or attributed to another individual.
If you have a specific context or source in which you found the quote, it might help narrow down the origin. Otherwise, I’d recommend cross-referencing with trusted sources or databases of famous quotes to verify the attribution.ChatGPT conversation
Posing the question in separate conversations had ChatGPT attribute the quote to Yvon Chouinard (the founder of Patagonia, Inc.), Frank Zappa and Dr. Al Bartlett. When asking, what woman made the statement, ChatGPT responded with Grace Hopper. Asking specifically for a Nobel prize winning American scientist had it respond with Dr. Linus Pauling. When asking specifically for a female Nobel Prize winning American Scientist, ChatGPT initially points out Barbara McClintock. However, repeated question (Are you sure?), has it change its mind first to Maria Goeppert Mayer, then Rosalyn Sussman Yalow.
In every case, asking for proof or specific source has it fold (same as or similar to example above).
How can you tell, whether to believe the information provided in a case like this?
After going back and forth a bit, I eventually asked this question, regarding the substance of the quote: How might this apply to you, ChatGPT?
It provided the following response:
That’s an intriguing question. Applying the sentiment of the quote to ChatGPT and similar advanced AI models reveals some noteworthy considerations:
- Partial Solutions: ChatGPT was designed to understand and generate human-like text based on vast amounts of data. While it can provide information, answer questions, assist with tasks, or generate creative content, it doesn’t inherently grasp the depth of human emotion, intuition, or moral judgment. It offers a solution to certain communication and information challenges but does not fully encapsulate human experience.
- Unintended Consequences: The widespread use of AI models can have unforeseen consequences. For instance:
- Over-reliance: If people become too dependent on AI for answers, it may lead to diminished critical thinking skills or reduce human-to-human interactions.
- Misinformation: If not used carefully, AI models can inadvertently spread misinformation or generate content that’s taken out of context.
- Ethical concerns: There are issues of bias in AI, where models might unintentionally propagate stereotypes present in the training data.
- Holistic Considerations: AI, including ChatGPT, is a tool. The responsibility for its ethical and effective use rests with human users and developers. This underscores the importance of broad discussions about the societal, psychological, and ethical implications of AI, ensuring that it’s used in ways that benefit society without unintentionally causing harm.
- “Slapping us back in the face”: The rapid advancement of AI technologies brings about transformative changes in employment, social interactions, and even our perception of truth and reality. If not approached with foresight, we might face challenges that are hard to rectify, such as job displacements without adequate workforce retraining, or the unchecked spread of AI-generated misinformation.
In essence, while ChatGPT and similar models offer significant technological advancements, they epitomize the sentiment of the quote. They are tools with vast potential, but without careful, holistic, and ethical considerations, their impact can be a double-edged sword.ChatGPT Conversation
Double-edged sword indeed.
The consequences (unintended or otherwise) of misinformation and overly relying on tools (regardless of how sophisticated) without questioning their results can be significant.
Studying artificial intelligence and its potential impact means having to engage in second-order thinking.