In recent years, Artificial Intelligence (AI) has made a quick rise and is the central component of many processes today. Language processes are no exception: concepts such as Prompt Engineering, RAG, and Fine-tuning appear everywhere. It is easy to get lost in this AI term jungle.
But no worries! We have compiled the most important AI terms for you so that you can follow the progress of AI with the same enthusiasm that we have. And don’t hesitate to ask questions – get in touch any time. We are happy to advise you on AI and language topics!
AI - Basic terminology
Artificial intelligence (AI)
The first challenging concept is Artificial Intelligence itself. Even though AI has been on everyone’s mind lately, it is not trivial to explain what artificial intelligence means: AI describes a subfield of computer science that deals with how computers or computer programs imitate human cognitive abilities.
These include computer programs that draw conclusions or solve problems without human intervention. AI abilities can be achieved through programming processes or Machine Learning. AI is the foundation of many applications that support people in everyday tasks.
Machine Learning (ML)
Machine Learning is a subfield of Artificial Intelligence in which computers learn from data. In the field of Machine Learning, certain tasks are learned via training with sample data that resemble the patterns to be imitated very well. The result of this operation is a so-called model.
There is a distinction between supervised and unsupervised Machine Learning. The difference lies, among other things, in the preparation of training data: in supervised learning the patterns that the model needs to learn are explicitly labeled in the training data, unsupervised learning uses unlabeled data, and the patterns to be imitated are determined in the learning process without intervention.
Deep Learning
Deep Learning is a form of unsupervised Machine Learning. A multi-layered Neural Network imitates the learning process of the human brain. This enables a model, that has been trained with Deep Learning, to “understand” and reproduce complex relations and patterns.
Today, Deep Learning forms the base of almost all AI systems, including Large Language Models (LLMs) and GPTs (Generative Pretrained Transformer). By the way, Neural Machine Translation is also based on Deep Learning.
Large Language Model (LLM)
Another term that often appears in the context of AI is “Large Language Model”. This term explains the concept through its components: It is a Language Model because it is the product of a Machine Learning process trained to perform a task in a language context. It is Large because the underlying training data is huge.
LLMs can recognize language content, generate new ones, summarize those, and more. The best-known LLMs are GPT from OpenAI (Attention: Don’t confuse it with ChatGPT!), Llama from Meta, and Mistral.
Natural Language Processing (NLP)
Natural Language Processing is a discipline between computer science and linguistics. It deals with understanding, editing, and creating natural language using machines.
The development of language-based AI systems, such as LLMs, became possible through NLP methods and the use of Deep Learning in the first place.
Generative Pretrained Transformer (GPT)
A Generative Pretrained Transformer is a Deep Learning language model that uses the Transformer Architecture, a special Deep Learning algorithm. The Transformer Architecture allows the model to understand semantic connections between all words in a text in one go, and to reproduce what has been learned. It constantly calculates on-the-fly which word is most likely to follow the previous one.
The focus of GPTs lies in generating new language content. That’s why they are the perfect language model for chatbots.
Chatbots
A chatbot is an application that allows a natural language question-answer communication between user and machine via a user interface, e.g. a chat window. The core of a chatbot is currently almost always an LLM, usually a GPT.
Probably the best-known example of a chatbot based on a GPT is ChatGPT.
"Why does ChatGPT often create nonsense answers?"
The processes behind answering a user request in ChatGPT are not always optimal. One reason for this is that the abilities of a GPT are limited to generating language by calculating a probable response based on training data. The correctness of the answer does not matter – the focus is on how natural it sounds. The topics, that are underrepresented in the training corpus, are considered a lot less than topics with a lot of data from which the model can learn. The training data for an LLM mostly originate from the Internet. This explains why ChatGPT works very well for general language tasks, but is likely to reach its limits quickly for specific tasks.
Fortunately, some methods make AI a very powerful tool. In the following sections, we will explain some of these methods in more detail.
Prompt, Prompting, Prompt Engineering
The most common concept associated with chatbots is the Prompt. The prompt describes an input interface for communication between humans and the chatbot. Over time, people started calling the actual input, meaning the request to the model, also Prompt.
Prompt Engineering is the creation of a prompt that is as efficient as possible, i.e. optimizing input creation to achieve the best possible results. Prompt Engineering is a time-consuming process that requires practice and expertise. It should be used specifically where the output quality needs to be high and results reliable.
Retrieval-Augmented Generation (RAG)
Retrieval-Augmented Generation is a method that optimizes output results in context by integrating external data, for example, company data. By doing so, a subject area that is underrepresented in the model’s training data can be prioritized to increase the correctness of the answers. One major advantage of RAG is, that the model is implicitly specialized without performing real, specific training. Another advantage: the data pool can flexibly be modified and expanded at any time without having to adapt the model.
Fine-tuning
Fine-tuning is a complex method for enabling specific queries of a language model. In fine-tuning, a further training phase is carried out for a generic pre-trained language model with a specified data set. Despite significantly higher effort, fine-tuning is indispensable for complex technical tasks. A disadvantage of fine-tuning is the limitation of the language model to a specific field, which allows little flexibility in the answers. A further disadvantage is that the model has to be retrained with each change in the application area, which increases effort.
Would you like to use AI in your language processes?
When AI is to be integrated into a company’s language processes, quality and accuracy of results are essential. Our experts examine each use case individually, without losing sight of the overall context.
We will be happy to advise you on which AI method is best suited to your case. Get in touch now to get more out of your language processes with the help of AI!
