With the “Efficient Language Model for On-Device Deployment” (ELMOD-2.7B), Fraunhofer IIS is releasing a compact large language model (LLM) optimized for German, specifically designed for use in resource-constrained environments. Now available on open-source platform Hugging Face, it addresses key challenges such as infrastructure cost, cloud dependency, and data privacy.
Scaling LLMs: Why Bigger Isn’t Always Better
These design decisions are motivated by broader developments in the field of large language models. The continuous increase in model parameters, training data, and computational resources has significantly advanced the capabilities of LLMs in understanding and generating natural language.
However, these gains have come with structural trade-offs: increasing dependence on centralized cloud infrastructure, persistent network connectivity, and energy-intensive compute environments. For many users, these constraints raise questions around reliability, operational cost, and, critically, data sovereignty.
From Scale to Efficiency: Shifting Toward Local LLMs
Fraunhofer IIS’ model ELMOD-2.7B originated from the DSgenAI research project (Digital Signal Processing using Generative Artificial Intelligence). It is a compact language model with 2.7 billion parameters, trained on a multilingual dataset including German, English, and code, with a particular focus on German-language performance.
The model’s compact size enables on-premises and on-device deployment scenarios, including execution directly on a smartphone chip. Deploying a LLM locally has several benefits for users:
- Reduced latency through local inference
- Operation without persistent connectivity
- Improved control over sensitive data, as processing can remain entirely on-device
- Lower infrastructure requirements, compared to large cloud-hosted models
These properties are particularly relevant in regulated industries and public sector environments, where data protection and operational independence are often primary concerns.
End-to-End Development and Compute Efficiency
ELMOD-2.7B was developed independently by Fraunhofer IIS, including the full model training pipeline, from data processing to training and evaluation. The model was trained on HPC resources at NHR@FAU with a limited compute budget of approximately 55,000 H100 GPU hours and is based exclusively on publicly available data.
This constraint required a focus on training efficiency and data quality. In addition, we introduced a multi-stage quality filtering and rephrasing process aimed at increasing the educational quality of the training data. While this step increased overall compute requirements, it allowed us to prioritize data quality over volume, a trade-off that proved relevant for model performance at smaller scales.
A First-of-Its-Kind German Data Pipeline
A central aspect of ELMOD-2.7B is its first-of-its-kind German dataset pipeline, designed to address linguistic characteristics that are often underrepresented in predominantly English-centric models.
Fraunhofer IIS developed a suite of German-specific preprocessing techniques, including:
- Handling of morphological variation (e.g. word inflections)
- Processing of compound words
- Consideration of orthographic conventions (such as umlauts or capitalization)
Further details on model development, architecture, and training data can be found in the accompanying paper.
ELMOD-2.7B in Comparison: Strong Performance at Small Scale
ELMOD-2.7B was evaluated using the EleutherAI Language Model Evaluation Harness across a range of reasoning and knowledge benchmarks, with a focus on German-language tasks.
Thanks to the architectural model choices as well as data choices, including prefiltering, the educational-quality filtering and rephrasal to raise the educational-quality, ELMOD-2.7B is the strongest performer in its size class (<3B), matching the performance of 7B parameter models in German.
Publication of the research results on Hugging Face
To support transparency and further development, Fraunhofer IIS is releasing the base model, and an instruction-tuned variant on the open-source platform Hugging Face. The base model provides the underlying language capabilities, while the instruction-tuned variant has been further optimized for following user prompts and completing task-oriented instructions, making it more suitable for interactive and application-facing use cases.
“ELMOD-2.7B does not aim to replace larger models. Instead, it shows that targeted design choices, especially in data processing, training efficiency, and a focused development approach, can significantly influence model performance within a limited parameter budget,” says Joel Schlotthauer, group lead for Natural Language Processing in the Generative AI department. “With the open release of the model weights and the accompanying paper, we want to show that strong German-language capabilities are possible even with a compact model. We also share practical insights into the development process behind ELMOD-2.7B, from data preparation to training choices.”
See a short demonstration of ELMOD-2.7B running on-device to illustrate its deployment characteristics and performance in a real-world setting. More information on the activities and other research projects of Fraunhofer IIS in the field of LLMs can be found under Allinga LLM.