runanywhere-sdks

RunAnywhere AI - Flutter Example

RunAnywhere Logo

iOS 13.0+ | Android 7.0+ Flutter 3.24+ Dart 3.5+ License

A production-ready reference app demonstrating the RunAnywhere Flutter SDK capabilities for on-device AI. This app showcases how to build privacy-first, offline-capable AI features with LLM chat, speech-to-text, text-to-speech, and a complete voice assistant pipeline—all running locally on your device.

Running This App (Local Development)

Important: This sample app consumes the RunAnywhere Flutter SDK through local path dependencies. A clean clone needs Flutter packages plus the Android JNI libraries and iOS XCFrameworks staged into the Flutter plugin packages.

Clean-Clone Bring-Up

Prerequisites:

From a fresh checkout:

cd examples/flutter/RunAnywhereAI
flutter pub get

# Build or refresh local native artifacts when the checkout has no staged binaries.
cd ../../..
./scripts/build/build-core-android.sh arm64-v8a
./sdk/runanywhere-swift/scripts/build-core-xcframework.sh
cd examples/flutter/RunAnywhereAI

flutter analyze
flutter build apk --debug
flutter build ios --simulator --debug

Notes:

How It Works

This sample app’s pubspec.yaml uses path dependencies to reference the local Flutter SDK packages:

This Sample App → Local Flutter SDK packages (sdk/runanywhere-flutter/packages/)
                          ↓
              Local XCFrameworks/JNI libs (in each package's ios/Frameworks/ and android/src/main/jniLibs/)
                          ↑
           Built by: ./sdk/runanywhere-swift/scripts/build-core-xcframework.sh + ./scripts/build/build-core-android.sh

Repo-root native build scripts (called from project root):

  1. ./sdk/runanywhere-swift/scripts/build-core-xcframework.sh — builds iOS XCFrameworks and stages them into sdk/runanywhere-flutter/packages/*/ios/Frameworks/.
  2. ./scripts/build/build-core-android.sh <ABI> — builds Android .so libraries and stages them into sdk/runanywhere-flutter/packages/*/android/src/main/jniLibs/<ABI>/.

Local consumption is enabled by the runanywhere.useLocalNatives=true Gradle property (default for development checkouts).

After Modifying the SDK

See It In Action

Download on App Store Get it on Google Play

Try the native iOS and Android apps to experience on-device AI capabilities immediately. The Flutter sample app demonstrates the same features using the cross-platform Flutter SDK.

Screenshots

RunAnywhere AI Chat Interface

Features

This sample app demonstrates the full power of the RunAnywhere Flutter SDK:

Feature Description SDK Integration
AI Chat Interactive LLM conversations with streaming responses RunAnywhere.llm.generateStream()
Thinking Mode Support for models with <think>...</think> reasoning Thinking tag parsing
Real-time Analytics Token speed, generation time, inference metrics MessageAnalytics
Speech-to-Text Voice transcription with batch & live modes RunAnywhere.stt.transcribe()
Text-to-Speech Neural voice synthesis with Piper TTS RunAnywhere.tts.synthesize()
Voice Assistant Full STT to LLM to TTS pipeline with auto-detection RunAnywhere.voice
Model Management Download, load, and manage multiple AI models RunAnywhere.models / RunAnywhere.downloads
Storage Management View storage usage and delete models RunAnywhere.downloads.getStorageInfo()
Offline Support All features work without internet On-device inference

Architecture

The app follows Flutter best practices with a clean architecture pattern:

┌─────────────────────────────────────────────────────────────────────┐
│                        Flutter/Material UI                           │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐ │
│  │  Chat    │ │   STT    │ │   TTS    │ │  Voice   │ │  Settings  │ │
│  │Interface │ │  View    │ │  View    │ │Assistant │ │   View     │ │
│  └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └─────┬──────┘ │
├───────┼────────────┼────────────┼────────────┼─────────────┼────────┤
│       ▼            ▼            ▼            ▼             ▼        │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │                 Feature ViewModels + UI State                 │   │
│  │           (SDK facades, Services, ListenableBuilder)          │   │
│  └──────────────────────────────────────────────────────────────┘   │
├─────────────────────────────────────────────────────────────────────┤
│                                                                     │
│                    RunAnywhere Flutter SDK                          │
│  ┌──────────────────────────────────────────────────────────────┐   │
│  │  Core API (generate, transcribe, synthesize)                  │   │
│  │  Model Management (download, load, unload, delete)            │   │
│  │  Voice Session (STT → LLM → TTS pipeline)                     │   │
│  └──────────────────────────────────────────────────────────────┘   │
│                              │                                      │
│           ┌──────────────────┴──────────────────┐                  │
│           ▼                                      ▼                  │
│  ┌─────────────────┐                  ┌─────────────────┐          │
│  │   LlamaCpp      │                  │   ONNX Runtime  │          │
│  │   (LLM/GGUF)    │                  │   (STT/TTS)     │          │
│  └─────────────────┘                  └─────────────────┘          │
└─────────────────────────────────────────────────────────────────────┘

Key Architecture Decisions

Project Structure

RunAnywhereAI/
├── lib/
│   ├── main.dart                      # App entry point
│   │
│   ├── app/
│   │   ├── runanywhere_ai_app.dart    # SDK initialization, model registration
│   │   └── content_view.dart          # Main tab navigation (5 tabs)
│   │
│   ├── core/
│   │   ├── design_system/
│   │   │   ├── app_colors.dart        # Color palette with dark mode support
│   │   │   ├── app_spacing.dart       # Spacing constants
│   │   │   └── typography.dart        # Text styles
│   │   │
│   │   ├── models/
│   │   │   └── app_types.dart         # Shared type definitions
│   │   │
│   │   ├── services/
│   │   │   ├── audio_recording_service.dart  # Microphone capture
│   │   │   ├── audio_player_service.dart     # TTS playback
│   │   │   ├── permission_service.dart       # Permission handling
│   │   │   ├── conversation_store.dart       # Chat history persistence
│   │   │   └── device_info_service.dart      # Device capabilities
│   │   │
│   │   └── utilities/
│   │       ├── constants.dart         # Preference keys, defaults
│   │       └── keychain_helper.dart   # Secure storage wrapper
│   │
│   ├── features/
│   │   ├── chat/
│   │   │   └── chat_interface_view.dart   # LLM chat with streaming
│   │   │
│   │   ├── voice/
│   │   │   ├── speech_to_text_view.dart   # Batch & live STT
│   │   │   ├── text_to_speech_view.dart   # TTS synthesis & playback
│   │   │   └── voice_assistant_view.dart  # Full STT→LLM→TTS pipeline
│   │   │
│   │   ├── models/
│   │   │   ├── models_view.dart           # Model browser
│   │   │   ├── model_selection_sheet.dart # Model picker bottom sheet
│   │   │   ├── model_list_view_model.dart # Model list logic
│   │   │   ├── model_components.dart      # Reusable model UI widgets
│   │   │   ├── model_status_components.dart # Status badges, indicators
│   │   │   ├── model_types.dart           # Framework enums, model info
│   │   │   └── add_model_from_url_view.dart # Import custom models
│   │   │
│   │   └── settings/
│   │       └── combined_settings_view.dart # Storage & logging config
│   │
│   └── helpers/
│       └── adaptive_layout.dart       # Responsive layout utilities
│
├── pubspec.yaml                       # Dependencies, SDK references
├── android/                           # Android platform config
├── ios/                               # iOS platform config
└── README.md                          # This file

Quick Start

Prerequisites

Clone & Build

# Clone the repository
git clone https://github.com/RunanywhereAI/runanywhere-sdks.git
cd runanywhere-sdks/examples/flutter/RunAnywhereAI

# Install dependencies
flutter pub get

# Run on connected device
flutter run

Run via IDE

  1. Open the project in VS Code or Android Studio
  2. Wait for Flutter dependencies to resolve
  3. Select a physical device (iOS or Android)
  4. Press F5 (VS Code) or Run (Android Studio)

Build Release APK/IPA

# Android APK
flutter build apk --release

# Android App Bundle
flutter build appbundle --release

# iOS (requires Xcode)
flutter build ios --release

SDK Integration Examples

Initialize the SDK

The SDK is initialized in runanywhere_ai_app.dart:

import 'package:runanywhere/runanywhere.dart';
import 'package:runanywhere_llamacpp/runanywhere_llamacpp.dart';
import 'package:runanywhere_onnx/runanywhere_onnx.dart';

// 1. Initialize SDK in development mode
await RunAnywhere.initialize();

// 2. Register LlamaCpp module for LLM models (GGUF)
await LlamaCpp.register();
RunAnywhere.models.register(
  id: 'smollm2-360m-q8_0',
  name: 'SmolLM2 360M Q8_0',
  url: Uri.parse('https://huggingface.co/prithivMLmods/SmolLM2-360M-GGUF/resolve/main/SmolLM2-360M.Q8_0.gguf'),
  framework: InferenceFramework.INFERENCE_FRAMEWORK_LLAMA_CPP,
  memoryRequirement: 500000000,
);

// 3. Register ONNX module for STT/TTS models
await Onnx.register();
RunAnywhere.models.register(
  id: 'sherpa-onnx-whisper-tiny.en',
  name: 'Sherpa Whisper Tiny (ONNX)',
  url: Uri.parse('https://github.com/RunanywhereAI/sherpa-onnx/releases/download/runanywhere-models-v1/sherpa-onnx-whisper-tiny.en.tar.gz'),
  framework: InferenceFramework.INFERENCE_FRAMEWORK_SHERPA,
  modality: ModelCategory.MODEL_CATEGORY_SPEECH_RECOGNITION,
  memoryRequirement: 75000000,
);

Download & Load a Model

// Download with progress tracking
final progressStream = RunAnywhere.downloads.start('smollm2-360m-q8_0');
await for (final p in progressStream) {
  if (p.stage == DownloadStage.DOWNLOAD_STAGE_COMPLETED) break;
}

// Load LLM model
await RunAnywhere.llm.load('smollm2-360m-q8_0');

// Check if model is loaded
final isLoaded = RunAnywhere.isLLMModelLoaded;

Stream Text Generation

// Generate with streaming (real-time tokens)
final stream = RunAnywhere.llm.generateStream(prompt, options);

await for (final event in stream) {
  if (event.isFinal) break;
  if (event.token.isNotEmpty) {
    setState(() {
      _responseText += event.token;
    });
  }
}

// Or non-streaming
final result = await RunAnywhere.llm.generate(prompt, options);
print('Response: ${result.text}');
print('Speed: ${result.tokensPerSecond} tok/s');

Speech-to-Text

// Load STT model
await RunAnywhere.stt.load('sherpa-onnx-whisper-tiny.en');

// Transcribe audio bytes
final result = await RunAnywhere.stt.transcribe(audioBytes);
print('Transcription: ${result.text}');

Text-to-Speech

// Load TTS voice
await RunAnywhere.tts.loadVoice('vits-piper-en_US-lessac-medium');

// Synthesize speech with options
final result = await RunAnywhere.tts.synthesize(
  text,
  TTSOptions(rate: 1.0, pitch: 1.0, volume: 1.0),
);

// Play audio (result.audio is Uint8List PCM16)
await audioPlayer.play(result.audio, result.sampleRate);

Voice Assistant Pipeline (STT to LLM to TTS)

// Subscribe to the voice agent event stream
final sub = RunAnywhere.voice.eventStream().listen((event) {
  if (event.hasUserSaid()) {
    print('User said: ${event.userSaid.text}');
  } else if (event.hasAssistantToken()) {
    print('Token: ${event.assistantToken.text}');
  }
});

// Initialize pipeline with loaded models
await RunAnywhere.voice.initializeWithLoadedModels();

// Cancel when done
await sub.cancel();

Key Screens Explained

1. Chat Screen (chat_interface_view.dart)

What it demonstrates:

Key SDK APIs:

2. Speech-to-Text Screen (speech_to_text_view.dart)

What it demonstrates:

Key SDK APIs:

3. Text-to-Speech Screen (text_to_speech_view.dart)

What it demonstrates:

Key SDK APIs:

4. Voice Assistant Screen (voice_assistant_view.dart)

What it demonstrates:

Key SDK APIs:

5. Settings Screen (combined_settings_view.dart)

What it demonstrates:

Key SDK APIs:

Supported Models

LLM Models (LlamaCpp/GGUF)

Model Size Memory Description
SmolLM2 360M Q8_0 ~400MB 500MB Fast, lightweight chat
Qwen 2.5 0.5B Q6_K ~500MB 600MB Multilingual, efficient
LFM2 350M Q4_K_M ~200MB 250MB LiquidAI, ultra-compact
LFM2 350M Q8_0 ~350MB 400MB Higher quality version
Llama 2 7B Chat Q4_K_M ~4GB 4GB Powerful, larger model
Mistral 7B Instruct Q4_K_M ~4GB 4GB High quality responses

STT Models (ONNX/Whisper)

Model Size Description
Sherpa Whisper Tiny (EN) ~75MB Fast English transcription
Sherpa Whisper Small (EN) ~250MB Higher accuracy

TTS Models (ONNX/Piper)

Model Size Description
Piper US English (Medium) ~65MB Natural American voice
Piper British English (Medium) ~65MB British accent

Testing

Run Tests

# Run all tests
flutter test

# Run with coverage
flutter test --coverage

# Run specific test file
flutter test test/widget_test.dart

Run Lint & Analysis

# Analyze code quality
flutter analyze

# Format code
dart format lib/ test/

# Fix issues automatically
dart fix --apply

Debugging

Enable Verbose Logging

The app uses debugPrint() extensively. Filter logs by:

# Flutter logs
flutter logs | grep -E "RunAnywhere|SDK"

Common Debug Messages

Log Prefix Description
SDK SDK initialization
SUCCESS Success operations
ERROR Error conditions
MODULE Module registration
LOADING Loading/processing
AUDIO Audio operations
RECORDING Recording operations

Memory Profiling

  1. Run app in profile mode: flutter run --profile
  2. Open DevTools: Press p in terminal
  3. Navigate to Memory tab
  4. Expected: ~300MB-2GB depending on model size

Configuration

Environment Setup

The SDK automatically detects the environment:

// Development mode (default)
if (kDebugMode) {
  await RunAnywhere.initialize();
}

// Production mode
else {
  await RunAnywhere.initialize(
    apiKey: 'your-api-key',
    baseURL: 'https://api.runanywhere.ai',
    environment: SDKEnvironment.SDK_ENVIRONMENT_PRODUCTION,
  );
}

Preference Keys

User preferences are stored via SharedPreferences:

Key Type Default Description
useStreaming bool true Enable streaming generation
defaultTemperature double 0.7 LLM temperature
defaultMaxTokens int 500 Max tokens per generation

Known Limitations

iOS Parity Notes

The iOS example app is the canonical reference. This app mirrors its tab structure, model catalog (lib/core/services/model_catalog_bootstrap.dart), model-picker filtering, generated solutions YAML, ViewModel layering, hybrid STT, and benchmarks. Intentionally unsupported iOS-only surfaces:

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

Development Setup

# Fork and clone
git clone https://github.com/YOUR_USERNAME/runanywhere-sdks.git
cd runanywhere-sdks/examples/flutter/RunAnywhereAI

# Create feature branch
git checkout -b feature/your-feature

# Make changes and test
flutter pub get
flutter analyze
flutter test

# Commit and push
git commit -m "feat: your feature description"
git push origin feature/your-feature

# Open Pull Request

License

This project is licensed under the Apache License 2.0 - see LICENSE for details.

Support

This site is open source. Improve this page.