- Developers building on-device or edge AI applications
- Teams evaluating local LLM deployment for privacy-sensitive agents
- Researchers comparing mobile and embedded inference frameworks
Models
LiteRT-LM
Google's open-source inference framework for deploying large language models on edge devices.
5.5K Stars
Apache-2.0 License
0.6K Forks
Open sourceLocal first
About
LiteRT-LM overview
LiteRT-LM is Google's open-source, production-oriented inference framework for running LLMs on edge devices. It is relevant for teams evaluating local, mobile, and on-device agent stacks where latency, privacy, and hardware constraints matter.
Edge-first inference
LiteRT-LM focuses on deploying LLMs on edge and on-device environments.
Local inference can reduce latency, preserve privacy, and keep agents useful when cloud access is constrained.Google AI Edge ecosystem
The project sits under Google's AI Edge GitHub organization.
Teams already watching Google's mobile and edge AI stack get a relevant open-source inference option to evaluate.Production-oriented model serving
The repository describes LiteRT-LM as a production-ready inference framework.
Agent builders need model layers that can move beyond notebooks and into real devices. Use cases
When to use LiteRT-LM
On-device assistants
Evaluate LiteRT-LM when an assistant needs local responses on mobile, desktop, or embedded hardware.
Private local inference
Use edge deployment to reduce dependence on cloud APIs for sensitive workflows.
Model runtime comparison
Compare LiteRT-LM with other local inference projects before choosing an agent model layer.
Compare
How it compares
When to choose LiteRT-LM
Compare it with nearby models by looking at hosting model, integration surface, license, and whether the official docs show the workflow you need.
FAQ
Questions
Is LiteRT-LM open source?
Yes. The GitHub repository is listed under the Apache-2.0 license.
Who should evaluate LiteRT-LM?
Teams building edge, mobile, desktop, or privacy-sensitive AI applications should evaluate it.
Tags
Capabilities
Decision brief
JSON
Should you use LiteRT-LM?
- Teams that only need hosted API access to frontier models
- Users looking for a complete agent framework rather than an inference layer
- Verified 2026-06-10
- License: Apache-2.0
- Repo: google-ai-edge/LiteRT-LM
- Open-source signal
Deployment
local, cloud
Permission surface
external services
Decision signals
Local first
Agent packet
Structured decision data for LiteRT-LM
This packet is the compact machine-readable view agents should use before following source links or taking action.
Capabilities
local inference, inference
Constraints
open source, local first
Deployment
local, cloud
Permission surface
external services
Recommended workflows
Local or private AI stack
Overview
What LiteRT-LM does
What it is
It is a model runtime layer for local and on-device LLM applications rather than a complete agent framework.
Why it matters
As agents move closer to devices, local inference becomes a core deployment choice.
How to evaluate it
Start from the official repository and documentation, then evaluate latency, supported models, device targets, and integration fit.
Facts
Known metadata and operating surface
These fields are separated from editorial interpretation so agents can reason over facts and missing checks.
Resource type model
Category Models
Maturity active
Difficulty Unknown
License Apache-2.0
Pricing open source
Verified 2026-06-10
Source confidence high
Risk level low
Fit matrix
Where LiteRT-LM fits in an agent stack
strong
Local or private AI stack
LiteRT-LM has multiple signals for local or private ai stack, including matching tags, capabilities, category, or positioning.
- Verify hardware requirements, data path, storage, and whether all calls stay in your environment.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Coding agent workflow
LiteRT-LM has at least one signal for coding agent workflow, but should be checked against a real task before adoption.
- Run a small repository change and inspect the diff, tests, and rollback path.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Connector or protocol layer
LiteRT-LM has at least one signal for connector or protocol layer, but should be checked against a real task before adoption.
- Connect one low-risk service, then inspect schemas, auth scope, errors, and logs.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
partial
Evaluation and observability
LiteRT-LM has at least one signal for evaluation and observability, but should be checked against a real task before adoption.
- Add one repeatable test case and confirm results can run again in review or CI.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
weak
Browser automation
LiteRT-LM is not primarily positioned for browser automation in the current metadata.
- Run one non-sensitive website task and inspect clicks, waits, retries, and changed URLs.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
weak
Memory or RAG workflow
LiteRT-LM is not primarily positioned for memory or rag workflow in the current metadata.
- Create, update, retrieve, correct, and delete memory or retrieval objects with real data.
- Confirm official docs, current maintenance, license, and runtime constraints before production use.
Inputs and outputs
What an agent should inspect
Likely inputs
- Repositories, files, issues, terminal output, and test results
- Tool schemas, API requests, service resources, and auth scopes
- Prompts, messages, documents, images, or model inputs
- Official setup instructions and a small real workflow
Likely outputs
- Diffs, commits, explanations, test results, or review notes
- Scores, traces, regression results, dashboards, or failure cases
- A decision on whether this resource fits the target workflow
Evidence
Sources, claims, and missing checks
Claims are marked separately from source links so future crawlers and reviewers can update them without rewriting the page.
GitHub github
Repository source for code, license, issues, releases, and implementation details.
Homepage homepageOfficial or project-controlled source for this resource profile.
LiteRT-LM is listed as open source.
License metadata: Apache-2.0LiteRT-LM has a recorded GitHub repository: google-ai-edge/LiteRT-LM.
Resource facts and GitHub source link.LiteRT-LM supports these recorded deployment modes: local, cloud.
OpenAgent decision signal metadata.LiteRT-LM is tagged with local inference, inference capabilities.
OpenAgent capability taxonomy.- Dedicated docs link is missing.
- Repository freshness has not been recorded.
Next action
How to start evaluating LiteRT-LM
Inspect repository
Check license, recent activity, issues, examples, and security-sensitive code paths.
Open sourceOpen Homepage
Start from the official source before adopting third-party instructions.
Open source Compare
Alternatives and nearby resources
Use related resources to compare category fit, license, deployment model, and first-workflow behavior.
FAQ
Common questions about LiteRT-LM
Is LiteRT-LM open source?
Yes. The GitHub repository is listed under the Apache-2.0 license.
Who should evaluate LiteRT-LM?
Teams building edge, mobile, desktop, or privacy-sensitive AI applications should evaluate it.