Understanding LLMs ONLINE

Aims

This one-day intensive course aims to demystify the transformer architecture that powers modern large language models (LLMs) such as GPT, Claude, and LLaMA. By the end of the course, participants will be able to: (1) explain the core components of transformer-based LLMs, including token embeddings, and the attention mechanism; (2) distinguish between model architectures and understand when each is appropriate; (3) describe how LLMs are trained and apply effective prompting strategies; (4) access and apply pre-trained models for practical tasks using the Huggingface ecosystem; and (5) critically evaluate LLM applications in their own research and begin experimenting with these tools.

Content

Large language models (LLMs) have rapidly transformed research and practice across disciplines, yet their inner workings often remain opaque to users. This course provides a conceptual foundation for understanding how modern LLMs work and how to use them effectively in research. Topics covered include: (1) an introduction to LLMs and the transformer architecture; (2) token embeddings and the attention mechanism; (3) model heads and the distinction between feature extraction and text generation; (4) the differences model architectures; (5) model training procedures and effective prompting strategies; (6) a hands-on introduction to the Huggingface ecosystem and accessing pre-trained models; and (7) an overview of practical applications across research domains. The day combines conceptual sessions with an applied demonstration, structured as follows: Introduction to LLMs; A peek into Huggingface; Embeddings and attention; Text generation; and An overview of applications.

Methods

The course combines interactive lectures with a hands-on, code-based demonstration. Conceptual sessions introduce the transformer architecture and its components, with worked examples and visualisations. The applied session walks participants through accessing and using pre-trained models from the Huggingface ecosystem, so that they can replicate and adapt the workflow afterwards. Discussion and Q&A are integrated throughout to connect the material to participants’ own research questions.

Target Group

All Doctoral Candidates & Postdocs

Requirements

No prior programming experience is required, though familiarity with a programming language (e.g., Python or R) will be helpful for following along with the applied demonstrations.

About the Trainer

Dirk Wulff is a Senior Research Scientist at the Center for Adaptive Rationality at the Max Planck Institute for Human Development in Berlin and a Senior Adjunct Researcher at the University of Basel. His research sits at the intersection of computational cognitive science, large language models, behavioral science, and psycholinguistics, with a particular focus on using LLMs as cognitive models and as tools for behavioral research. He has over 15 years of experience in conducting postgraduate and professional trainings on various topics in data science and artificial intelligence.

Zak Hussain is a researcher at the Center for Cognitive and Decision Sciences at the University of Basel. After earning an MSc in Cognitive and Decision Sciences from University College London, he pursued a PhD focused on applying modern language modeling techniques to behavioral science research. He is an active proponent and educator on the use of language models in social science research more broadly. He has acted in this capacity internationally, having run introductory workshops on open-source tools for language modeling.

Workload

Preparation: 2h
Course attendance: 8h
Follow-up (optional): 0-20h

Feature

Once registration is open, applications will be collected for 24 hours and course places allocated by lot. All registrations received after the initial 24h period will be put on a waiting list and assigned on a first come, first served basis.

Course places/places on the waiting list will be confirmed by e-mail. Course registrations can only be canceled before the registration period ends (send an e-mail to grace@unibas.ch). Full course attendance is mandatory. Participants who fail to attend a course without prior notification or withdraw after the registration deadline are subject to a fee of CHF 30. In addition, participants who cancel their course registration at a later point in time, are absent without an excuse or do not attend the entire course will, for reasons of fairness, not be considered for course registration in the following semester and will be removed from other courses offered in the same semester. Please find the detailed regulations on the Transferable Skills Homepage.