Rolf Dyrnes Svendsen, NorwAI’s communication manager and former editor at Adresseavisen, digs into the impact of AI on text generation and how Trondheim and the Norwegian language can be part of this new future.
Trondheim’s role in the GPT revolution
Right now, Norway’s most powerful supercomputers are running around the clock to create an essential building block for the country’s technological future — and it’s all happening right here in Trondheim.
NTNU’s supercomputers are putting together a Norwegian language GPT model with 20 billion parameters. To give you an idea of just how big that is, it’ll require the supercomputers’ entire processing power for whole two months to create it. But what is the purpose of this model? And why is it so important for Norway?
AI is here
Silicon Valley's ChatGPT is taking the world by storm. Created by OpenAI — which is backed by Microsoft and Elon Musk among others — the chatbot has incredible conversational skills. The possibilities shown by this technology have made it clear to anyone how artificial intelligence can contribute to their work and life. But while ChatGPT is great in English, it’s not so impressive in Norwegian. The supply of text material to train AIs is much greater in English than in smaller languages, which means language models gradually become worse for less-used languages.
But while ChatGPT is great in English, it’s not so impressive in Norwegian.
The race to develop the biggest and best language models is currently going on between the great powers in San Francisco and Beijing — but Norway has its own small horse here in Trondheim. Large language models can potentially transform industries like the internet and the smartphone did in the past. The models provide new content, knowledge and creativity with AI-based applications. Several challenges remain to be solved, but the language models will accelerate productivity and growth.
If other language communities are not to become completely dependent on English or Chinese-based technology, they must develop their own language models to safeguard their digital independence, competence, and competitiveness. The EU has 24 official languages. Norwegian is not one of them.
Norwegian alternative
The research centre NorwAI at NTNU has already developed two smaller GPT test models since the centre opened two years ago. Now, at the start of 2023, the centre is preparing for a big leap. NorwAI will have its model ready later this spring, based on Norwegian input, among it data from the media group Schibsted. The partners in the NorwAI consortium will be able to play-test the model for their purposes. The next step will be to build a twice as large version of 40 billion parameters before expanding further.
The next step will be to build a twice as large version of 40 billion parameters before expanding further.
GPT stands for ‘Generative Pretrained Transformers’ and builds on artificial intelligence to learn and understand the input. Billions of words and information from online and lexical sources are processed. In short, the large language models' basic functionality is to predict the next word in a context via probability calculations. This enables impressive language solutions that generate texts for self-selected topics, summarize complex matters, answer questions in ongoing dialogues, etc. Language models are also capable of producing speech and illustrations.
Will affect us all
Throughout history, only humans have been able to express complex prose creatively. Now the large language models (LLM) show astonishing results. They will reduce the price of knowledge and creativity. They will impact all creative and knowledge-based professions, and they work as text-to-text, speech-to-text, and text/speech-to-illustrations.
They will affect our relationship with machines. Professions will change, some will be replaced, and new ones created. The new profession of AI-operator will be born. AI is entering education, sales and marketing, and professions such as lawyers, journalists, programmers, designers, etc. will be affected. The reason is simple. In creative subjects, mistakes are cheap and acceptable because people are involved in the process anyway.
Professions will change, some will be replaced, and new ones created.
There is still a lot of development work to be done for large language models. They do not always distinguish between fact and fiction, they hallucinate. They are language models, not sources of knowledge. Verifying language models will become a major area of research. But improvements will come — and Trondheim will be a part of them.
Interested in AI and media technology?
This will be the topic in our Innovation Breakfast 15th of September. Read more and sign up here.
Note: The breakfast will be in Norwegian.
No items found.
Trondheim Tech Port + The List = Sant
Visste du at Trondheim har et gratis, engelsk kulturmagasin som deles ut i hele byen? Tidligere har enkeltvise utgaver hatt teknologifokus, men i år vil hver eneste utgave ha minst 7 sider teknologirelatert innhold - som seg hør og bør i en teknologihovedstad! Dette er gjort mulig gjennom et samarbeid mellom Trondheim Tech Port og The List. Denne saken finner du i Vårutgaven av magasinet, utgitt i mars 2023.
Visste du at Trondheim har et gratis, engelsk kulturmagasin som deles ut i hele byen? Tidligere har enkeltvise utgaver hatt teknologifokus, men i år vil hver eneste utgave ha minst 7 sider teknologirelatert innhold - som seg hør og bør i en teknologihovedstad! Dette er gjort mulig gjennom et samarbeid mellom Trondheim Tech Port og The List.
Les mer
