Skip to content

feat(litert): add a LiteRT-LM provider built on react-native-litert-lm - #376

Draft
john-rocky wants to merge 1 commit into
callstackincubator:mainfrom
john-rocky:feat/litert-provider
Draft

john-rocky wants to merge 1 commit into
callstackincubator:mainfrom
john-rocky:feat/litert-provider

Conversation

@john-rocky

Copy link
Copy Markdown

Summary

Adds @react-native-ai/litert, an AI SDK provider for LiteRT-LM, built on react-native-litert-lm. It runs .litertlm models on iOS and Android, with generation, streaming, tool calling and structured output.

import { litert } from '@react-native-ai/litert'
import { generateText } from 'ai'

const model = litert('hf://litert-community/gemma-4-E2B-it-litert-lm')
await model.prepare()

const { text } = await generateText({ model, prompt: 'Write a haiku about coding.' })

Sampling and tools apply when the model loads, so changing them between calls reloads the model and returns a warning. Options the runtime doesn't support return an unsupported warning.

Given a Hugging Face repo, the provider picks the file: the GPU-only file on iOS and the full bundle on Android, because the GPU-only Gemma 4 E2B file gave wrong text, with no error, on a Pixel 11 Pro.

Docs are in website/src/content/docs/docs/litert/, and the root README gets a row and a section.

Test plan

  • Unit tests, typecheck and lint pass.
  • In an Expo app on the packed package, from a clean install with litert-community/gemma-4-E2B-it-litert-lm, 30 checks pass on a Pixel 11 Pro (Android 17) and on an iPhone 18 Pro (iOS 27.2). They cover generation, streaming, tools, structured output, conversations, and these failures: no such file or repo, a broken or truncated file, an abort during a load and during a call, a tool call that doesn't parse (injected), tool arguments that fail the schema, and a prompt that's too long.
  • Not verified on a device: backend: 'npu', speculativeDecoding, a repo with a litertlm_manifest.json, a gated model's accessToken, and Android asking for memory back.
  • The provider isn't in apps/example or harness yet.

@vercel

vercel Bot commented Nov 6, 2026

Copy link
Copy Markdown

@john-rocky is attempting to deploy a commit to the Callstack Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant