What Is a Large Language Model?

A large language model (LLM) is a neural network trained on a very large collection of text to predict which word, or part of a word, comes next. Repeating that prediction one token at a time lets it write answers, summaries, translations and code. ChatGPT, Claude and Gemini are chat products built on LLMs.

Each part of the name means something. Large refers to the number of parameters, the learned weights inside the network, which run from a few billion in small models to hundreds of billions in the largest, and to the training data, often trillions of tokens. Language is what it is trained on and produces. Model is the trained network itself: a file of weights plus the code that runs them.

LLMs are a type of deep learning model and the best-known form of generative AI. Many newer models also accept images or audio as input, but the core idea is the same.

Almost every current LLM is based on the transformer, the architecture described by Google researchers in the 2017 paper “Attention Is All You Need”. A prompt passes through four stages.

Training happens in stages. Pretraining runs the model over a huge text corpus, adjusting its parameters each time it predicts the next token wrongly. No labeling is needed, because the text itself supplies the answers. This stage takes weeks on clusters of data centre GPUs.

Fine-tuning then teaches the model to follow instructions and hold a conversation, using smaller sets of example prompts and answers. Many developers add reinforcement learning from human feedback (RLHF), where people rank answers and the model is tuned towards the preferred ones. After training, using the model to answer prompts is called inference, and it needs far less computing power than training.

A handful of terms come up whenever LLMs are compared or sized for hardware.

Precision is where most hardware decisions start. A model stored at 16-bit precision needs about 2 bytes per parameter, so a 7-billion-parameter model needs roughly 14 GB for its weights alone. INT8 quantization halves that, and 4-bit formats roughly quarter it.

LLMs split into closed models, used through the developer’s own apps or API, and open-weight models that anyone can download and run.

An LLM predicts plausible text; it does not look facts up. It can state wrong information confidently, which is known as hallucination. Its knowledge stops at its training cut-off date, and the largest models need costly, power-hungry hardware to run.

The usual remedy is retrieval-augmented generation (RAG): the system first searches your own documents, then passes the relevant passages to the model with the question, so answers draw on current, checkable sources. Sensitive data is another concern, because prompts sent to a cloud service leave the site. Running an open-weight model on your own hardware keeps them in the building.

Smaller open-weight models, quantized to 8-bit or 4-bit, now run on embedded hardware. That lets a machine answer questions without a network connection and keeps operational data on site. Our Jetson Thor computers carry up to 128 GB of memory on the Jetson T5000 module, enough for models of tens of billions of parameters once quantized. Larger models, or many users at once, suit edge AI servers with data centre GPUs.

Typical industrial uses include an HMI that answers maintenance questions from the machine’s manuals, a robot that takes spoken instructions and a system that summarizes shift logs or alarms. Our glossary of AI and GPU computing terms explains the related vocabulary.

We build edge AI computers and GPU servers for on-site LLM inference with the operating system, drivers and AI runtime installed as standard, and validate each system before it ships. Tell us the model size, number of users and site conditions, and we will specify the memory, GPU and cooling to match.