AI needs no introduction. Its implications for the broadcast industry have been known and seen for a long time, but perhaps never so prominently as in 2023, with the Writers Guild of America on strike.
On the flip side, there was huge excitement about the potential of AI and ML in enabling churn prediction, metadata enrichment, personalisation strategies and more. In this episode, our hosts Simon McGrath and Alex Borland are joined by Dr Paul Stallard, Merapar Principal Architect, and Klaus Seiler, co-founder and CTO of Merapar. They discuss the potential that AI has for broadcasters worldwide.
Inside the episode
The first of two episodes on AI and machine learning, recorded about nine months after ChatGPT reached the public. Paul Stallard explains what makes a large language model large, and the conversation turns to what such models cost to run, and why a company might run a smaller one itself.
The second half is about practice. For structured data, traditional machine learning is often faster, cheaper and good enough. The guests cover metadata enrichment, automatic curation of archives, chaining models together, hallucinations, and explainability, including what one churn model revealed. Simon McGrath closes on the cost and sustainability of large models.
Highlights
- 07:37 “The key characteristic of a large language model is it really is large.” Paul Stallard
- 13:20 “These generative models are designed for generative tasks.”
- 17:24 Automatic curation: ask an archive for all the Formula 1 crashes of the last year, and get a virtual episode guide.
- 21:59 Models with a finite context drift into hallucination in long conversations: “there probably is always a place for having some sort of validation.”
- 24:06 “The number of different shows that you’re watching in a month is really important.” Paul Stallard, on churn prediction.
- 26:34 The economics and sustainability of large language models, flagged as a subject for a future episode.
Key questions answered
Why did AI suddenly become the topic of 2023?
Not because AI was new to media. Alex Borland notes that operators had used it for years, for recommendations, operations and predicting network usage; what changed was ChatGPT reaching the open market. Paul Stallard traces the technology to the transformer, from a 2017 Google Research paper: it allows a much larger context, uses attention to learn which parts of it matter, and can be trained in parallel, so with enough GPUs these models could learn from enormous amounts of text.
Why would a company run its own model?
Size, cost and data. The largest models are expensive to run, Klaus Seiler points out. Data is another concern: nobody knows exactly what personal data or bias the big models were trained on, and prompts may be used to train them further, so companies risk leaking their own IP. Running a slimmed-down model on premises, such as the Llama 2 models Facebook had just released, gives more control, and with transfer learning the last layer of a network is retrained on your own data, giving a smaller model specialised to one application.
Does every AI task need a large language model?
No. Generative models suit generative tasks, such as writing a synopsis. For structured data, anything that fits in a table, traditional models such as logistic regression, random forest and XGBoost produce very good results, and where a model sits in the real-time path of a user experience, a small one that makes quick inferences matters.
What can AI do for metadata and archives?
Enrich it: from automating subtitles, and using them to generate a synopsis, to processing video directly for keywords, moods and scenes. That builds indexes and chapters, opens archives up to search and helps monetise a back catalogue, down to curated niche or personalised channels. Paul Stallard says customers were already deploying several models at different stages: extracting data from content, personalisation and recommendations, marketing, advertising and churn prediction.
Why does explainability matter?
Because people want to know why a model decides what it does. Large deep learning models are essentially opaque; simpler ones show what led to a decision. Paul Stallard describes a customer whose churn model weighed subscription length, content watched, hours watched and the number of different shows watched each month. The number of different shows mattered much more than hours. Many hours in one month can mean someone binging the series they joined for, then leaving; watching many different shows, or creating several profiles, predicts that they stay.
Speakers
- Dr Paul Stallard, Principal Architect, Merapar
- Klaus Seiler, co-founder and CTO, Merapar
- Simon McGrath, Merapar (host)
- Alex Borland, VP Strategy & Business Development, Merapar (host)
