Thanks for joining me for another edition of the SerenityThroughSweat blog.
AI has been in the news quite a bit recently with the continuing advancement of ChatGPT and the drama surrounding its upper management.
I came across some of the grassroots origin of AI, in the form of computational linguistics, while continuing research on my communications project.
I am far from a subject matter expert on AI, language, or communication, but here is my two cents nonetheless. And, you should take it, we are due for a recession anyway.
Computational linguistics really began as a field before it ever had a chance. By that I mean the right tools for the job hadn’t even been invented yet.
ChatGPT and other Large Language Models, LLM’s, require enormous datasets and computing power. Before the internet, and the personal computer, this meant manual entry and analysis of all those words.
The LLMs function less by looking at the “rules of language”, and more by analyzing the likelihood of what the answer should be based on existing information.
From the analysis on computational linguistics, “Members of the IBM research team flaunted their ignorance of linguistics as if to taunt the other researchers. Fred Jelinek is famously quoted as saying, ‘Every time I fire a linguist from our project, the performance of our system gets better’
I think the easiest way to think about these LLM’s is as probability engines. This work was pioneered by Claude Shannon (whose work I have covered in quite a few other posts)
The LLM absorbs and analyzes a huge amount of data. An unimaginable amount of data. Think about reading the entire contents of the internet. Every tweet, every news article, every blog. Then statistically analyzing all those words to look for patterns.
From a previous post covering the work of Shannon, “As Shannon showed, this model also describes the behavior of messages and languages. Whenever we communicate, rules everywhere restrict our freedom to choose the next letter and the next pineapple*” “Because you’re completely aware of those rules, you’ve already recognized that ‘pineapple’ is a transmission error. Given the way the paragraph and the sentence were developing, practically the only word possible in that location was ‘word’ “
When Shannon completed his mathematical theory of communication, the internet wasn’t even a pipe dream, and he did a tremendous amount of work developing the earliest computers.
His theories and ideas, though, would pave the way for how these LLMs operate. They look for patterns by searching and analyzing all of the current written work on a topic. They then recombine words in a statistically viable way to answer questions
You can debate whether or not this constitutes, learning, or understanding, or consciousness, but that’s not really the point. It is here now, in this current form, and it can be an extremely useful tool. It can also spit out unintelligible garbage. So how do you engage with LLMs in a way that is useful and productive?
I think the answer has already been covered in the AI action warning movie Irobot. “My responses are limited, you must ask the right questions”
In this light, the rise of ChatGPT and other LLMs has led to the creation of a new host of jobs, one of which is the prompt engineer.
I first heard about the prompt engineer from episode 556 of the freakonomics podcast.
Prompt engineers discern what it is that their customer wants, and then find a way to effectively communicate that to the LLM.
Asking the right questions, adding the right context and constraints, make all the difference. If you think about it, the same concept applies to communicating with our kids. Or with other adults who may be operating outside their area of expertise.
If you want your five year old to do something, you need to set up some guideraills, and provide clear expectations. If you want a coworker to complete a new task, you need to provide the context and desired outcome, in order to get the finished product you want.
LLMs function much like the very intelligent five year old. You can be amazed what they are able to produce if given the right prompt.
Sometimes, it is hard to know what exactly we want. It is even harder to find the right combination of words to effectively transmit that want to someone else. Asking the right questions, setting the right context and guardrails, can help us in the endeavor. Finding the right prompt, might just lead to some serenity.
Thanks for joining me, stay safe and stay sweaty my friends.