llama_library_flutter 0.0.2

LLaMA Is Library for Inference of Meta's LLaMA model (and others)

Llama Library

Llama Library Is library for inference any model ai LLAMA / LLM On Edge without api or internet quota, but need resources depends model you want run

Copyright (c) 2024 GLOBAL CORPORATION - GENERAL DEVELOPER

📚️ Docs

1. Documentation
2. Youtube
3. Telegram Support Group
4. Contact Developer (check social media or readme profile github)

🔖️ Features

1. ✅ 📱️ Cross Platform support (Device, Edge Severless functions)
2. ✅ 📜️ Standarization Style Code
3. ✅ ⌨️ Cli (Terminal for help you use this library or create project)
4. ✅ 🔥️ Api (If you developer bot / userbot you can use this library without interact cli just add library and use 🚀️)
5. ✅ 🧩️ Customizable Extension (if you want add extension so you can more speed up on development)
6. ✅ ✨️ Pretty Information (user friendly for newbie)

❔️ Fun Fact

• This library 100% use on every my create project (App, Server, Bot, Userbot)

• This library 100% support all models from llama.cpp (depending on your device specs, if high then it can be up to turbo, but if low, just choose tiny/small)

📈️ Proggres

• 10-02-2025 Starting Release Stable With core Features

Resources

1. MODEL

📥️ Install Library

1. Dart

dart pub add llama_library_dart

2. Flutter

flutter pub add llama_library_flutter ggml_library_flutter

🚀️ Quick Start

Example Quickstart script minimal for insight you or make you use this library because very simple

import 'dart:convert';
import 'dart:io';
import 'package:llama_library/llama_library.dart';
import 'package:llama_library/scheme/scheme/api/send_llama_library_message.dart';
import 'package:llama_library/scheme/scheme/respond/update_llama_library_message.dart'; 

void main(List<String> args) async {
  print("start");
  File modelFile = File(
    "../../../../../big-data/deepseek-r1/deepseek-r1-distill-qwen-1.5b-q4_0.gguf",
  );
  final LlamaLibrary llamaLibrary = LlamaLibrary(
    sharedLibraryPath: "libllama.so",
    invokeParametersLlamaLibraryDataOptions: InvokeParametersLlamaLibraryDataOptions(
      invokeTimeOut: Duration(minutes: 10),
      isThrowOnError: false,
    ),
  );
  await llamaLibrary.ensureInitialized();
  llamaLibrary.loadModel(
    modelPath: modelFile.path,
  );
  llamaLibrary.on(
    eventType: llamaLibrary.eventUpdate,
    onUpdate: (data) {
      final update = data.update;
      if (update is UpdateLlamaLibraryMessage) {
        /// streaming update
        if (update.is_done == false) {
          stdout.write(update.text);
        } else if (update.is_done == true) {
          print("\n\n");
          print("-- done --");
        }
      }
    },
  );
  await llamaLibrary.initialized();

  stdin.listen((e) async {
    print("\n\n");
    final String text = utf8.decode(e).trim();
    if (text == "exit") {
      llamaLibrary.dispose();
      exit(0);
    } else {
      await llamaLibrary.invoke(
        invokeParametersLlamaLibraryData: InvokeParametersLlamaLibraryData(
          parameters: SendLlamaLibraryMessage.create(text: text),
          isVoid: true,
          extra: null,
          invokeParametersLlamaLibraryDataOptions: null,
        ),
      );
    }
  });
}

Reference

1. Ggerganov-llama.cpp ffi bridge main script so that this program can run

Copyright (c) 2024 GLOBAL CORPORATION - GENERAL DEVELOPER

Example Project Use This Library

1. AZKA GRAM / Global GRAM

Telegram Application with redesign with new some features userbot and other features which is not officially provided on Telegram First this project open source but we closed it to close source because our program is easy to read and allows other people to edit the source code and then use it for criminal acts

CHAT PAGE	SIGN UP PAGE	HOME PAGE	GUIDE PAGE