Large Language Models (LLMs) & Generative AI (GenAI)¶
Info
Please refer to the University's AI Governance Policy and related AI Tools Guidance document for general advice on responsible use of AI across the University.
This is a fast-moving field; We are currently trialling a number of different tools and deployment models for research-related LLM/genAI use.
Access¶
Email rtis.support@otago.ac.nz to discuss how we can help you support your particular usecase. When you are onboarded, you will receive a personal API access key. This should give you access to a relevant set of LLM models.
The eResearch Solutions LLM gateway is exposed as an OpenAI-compatible endpoint.
- API base URL:
https://llm.uod.otago.ac.nz/v1
Most OpenAI-compatible SDKs or tools should be able to interface with this. This is accessible on-campus/via VPN only.
Models¶
Cloud models¶
A selection of cloud LLM models on MS Foundry can be made available via our LLM gateway for research usecases. Please contact us to discuss your requirements.
The current set includes gpt-oss-120b, gpt-5.4, claude-opus-4-6, Kimi-K2.5, DeepSeek-V3.2, text-embedding-3-small.
Cloud-hosted models should not be used for processing of any input/data that is considered sensitive, subject to data sovereignty, etc. Please refer to the AI Tools Guidance document.
Local on-campus models¶
We have a few small models hosted entirely on-campus without dependencies on external cloud services, available for select research trial usecases.
Running these models locally within the campus environment requires significant resources that are currently limited due to our present hardware capabilities. Therefore usage should focus on research scenarios where on-campus processing is absolutely necessary (e.g. sensitive data).
Local Model list¶
Local models are prefixed with ONCAMPUS/.
The selection of available models is subject to change, depending on specific demand, usecases and on available hardware resources.
Currently, eResearch Solutions makes the following models available for inference on-campus:
| name | model | parameters | quantisation | input | license |
|---|---|---|---|---|---|
| ONCAMPUS/Qwen3.6-27B-FP8 | Qwen 3.6 27B FP8 | 27B | FP8 | text-only | Apache-2.0 |
| ONCAMPUS/Qwen3.5-9B | Qwen 3.5 9B BF16 | 9B | BF16 | text-only | Apache-2.0 |
| ONCAMPUS/gemma-4-12B-it | Google Gemma 4 12B IT BF16 | 12B | BF16 | multimodal | Apache-2.0 |
| ONCAMPUS/BAAI/bge-m3 | BAAI/bge-m3 | 0.568B | FP16 | multilingual text embedding | MIT |
Model features to consider¶
Modality¶
LLM modality refers to the different types of data that LLMs can process, such as text, images, and audio. Multimodal LLMs integrate these modalities to enhance understanding and interaction, mimicking human-like comprehension of diverse information sources.
Parameters¶
LLM parameters -typically expressed in billion (B)- are the internal weights learned during training that capture patterns in language such as grammar, context and relationships between words.
More parameters allow the model to capture more complex patterns in the data at training, potentially improving accuracy, but at the cost of increased resource use and speed; It's a trade-off between complexity and performance.
Quantisation¶
Quantisation is a model compression technique that reduces the precision of weights and activations, making LLMs more efficient.
A full-size model might use FP32 -i.e. 32 bits to represent each number inside the model-, which is great for accuracy but terrible for size. Quantization shrinks that down: 16 bits (FP16 and BF16), 8 bits (FP8), or even smaller in some cases (GGUF). The smaller the bit size, the lighter the model.
While quantised models can run more efficiently on less powerful hardware with reduced cost and environmental footprint, they sacrifice some accuracy compared to their full-precision counterparts. Advancements in quantisation techniques are helping to minimise this accuracy loss, making quantised models increasingly viable for various applications.
License¶
Truly Open, Open-Weights, or Restricted-Weights Open Access? All models labeled "open" are not equal. Some follow open-source principles by offering the full stack — code, weights, training methodology, documentation under an open-source license. Others only publish the trained model weights, often under restrictive conditions that limit their usage or redistribution; These are commonly referred to as "open-weight" models.
Distillation¶
Distilled LLM models are smaller, more efficient versions of large language models that retain much of the original model's performance while requiring less computational power and resources. This process, known as model distillation, involves transferring knowledge from a larger model (the "teacher") to a smaller model (the "student") to optimize it for specific tasks. Distilled models are optimized for single-GPU setups and can deliver decent performance compared to the full model with much lower resource requirements.
Chain-of-Thought¶
Chain-of-thought (CoT) is a technique that enhances a model's "reasoning" abilities by guiding them to articulate a series of intermediate steps before arriving at a final answer (i.e. seemingly "thinking" about a problem before providing a final answer). This method helps break down complex problems into manageable parts, often improving accuracy and clarity in responses, and also allows the user the opportunity to study the "reasoning" behaviour of the model.
Most models can be prompted to employ CoT reasoning (e.g. by adding "Let's think step by step." to your prompt), but some 'reasoning'/'thinking' models are particularly well-trained for this and will apply Auto-CoT automatically.
Note that this is a simulated reasoning-like process that quickly breaks down once confronted with out-of-domain logical problems that don't match the specific logical patterns found in the model's training data. (See 'Limitations' below)
LLM limitations and considerations¶
Large language models (LLMs) have gained significant attention due to their impressive capabilities. However there is also a lot of confusion and marketing hype surrounding LLMs and GenAI.
These models excel in generating text that is contextually relevant by estimating the probability of different words or phrases occurring in a given context, allowing it to create coherent and contextually relevant responses.
However, it is crucial not to anthropomorphise LLMs' abilities. LLMs do not possess human-like reasoning or thinking capabilities. The transformer architecture underlying these models allows them to generate plausible text based on probabilistic pattern matching but does not enable genuine reasoning, introspection, intention or causal understanding.
Prompting an LLM model is essentially guiding a statistical text generator to produce outputs based on your prompts.
The effectiveness of LLMs lies in their ability to generate the most plausible response to an input, showcasing strong performance on various tasks and emergent abilities. Nonetheless, it is essential to recognise and set realistic expectations for what LLMs can and cannot do, understanding their limitations.
Other considerations
- LLMs may confidently produce plausible-sounding but factually incorrect or nonsensical responses
- LLMs are costly, resource-intensive, ethically complex, and carry a heavy environmental burden
We trust our researchers to thoughtfully balance the use of our LLMs (and their associated cost) against the value they deliver to their work, the University, and the broader community.