Initial commit
This commit is contained in:
Stefan Caramizoiu
commit
d7a7d2cafd
17 changed files with 2274 additions and 0 deletions
39
.gitignore
vendored
Normal file
39
.gitignore
vendored
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
# Virtual environment
|
||||
venv/
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*.pyo
|
||||
*.egg-info/
|
||||
dist/
|
||||
build/
|
||||
|
||||
# Environment variables (contains secrets)
|
||||
backend/.env
|
||||
|
||||
# Piper voice models (large binary files)
|
||||
backend/voices/*.onnx
|
||||
backend/voices/*.onnx.json
|
||||
|
||||
# Whisper cached models
|
||||
backend/whisper_models/
|
||||
|
||||
# OS files
|
||||
Thumbs.db
|
||||
Desktop.ini
|
||||
.DS_Store
|
||||
|
||||
# IDE
|
||||
.vscode/
|
||||
.idea/
|
||||
*.swp
|
||||
*.swo
|
||||
|
||||
# TTS temp files
|
||||
*.wav
|
||||
*.mp3
|
||||
|
||||
# Test files
|
||||
backend/test_tts.wav
|
||||
backend/test_setup.py
|
||||
130
README.md
Normal file
130
README.md
Normal file
|
|
@ -0,0 +1,130 @@
|
|||
# EcoBot - Asistent Vocal AI pentru Scoala Verde
|
||||
|
||||
EcoBot este o aplicatie AI care ruleaza local pe calculatorul tau. Ii vorbesti, iar el iti raspunde cu voce, in timp ce un caracter animat se misca pe ecran.
|
||||
|
||||
## Cum functioneaza
|
||||
|
||||
```
|
||||
Microfon → faster-whisper (voce → text)
|
||||
→ LM Studio / LLM local (generare raspuns)
|
||||
→ Edge TTS / Piper TTS (text → voce)
|
||||
→ Redare audio + caracter animat
|
||||
```
|
||||
|
||||
## Cerinte
|
||||
|
||||
- **Python 3.10+**
|
||||
- **LM Studio** (sau alt server LLM compatibil OpenAI API)
|
||||
- **Windows 10/11** (testat pe Windows 11)
|
||||
- **Conexiune internet** (doar pentru Edge TTS si prima descarcare Whisper)
|
||||
|
||||
### Hardware recomandat
|
||||
|
||||
| Componenta | Minim | Recomandat |
|
||||
|-----------|-------|------------|
|
||||
| RAM | 8 GB | 16+ GB |
|
||||
| GPU | - | NVIDIA 4+ GB VRAM |
|
||||
| CPU | Orice modern | 6+ cores |
|
||||
|
||||
## Instalare
|
||||
|
||||
### 1. Cloneaza repo-ul
|
||||
|
||||
```bash
|
||||
git clone https://github.com/<user>/scoala-verde.git
|
||||
cd scoala-verde
|
||||
```
|
||||
|
||||
### 2. Creeaza mediul virtual si instaleaza dependintele
|
||||
|
||||
```bash
|
||||
python -m venv venv
|
||||
venv\Scripts\activate
|
||||
pip install -r backend/requirements.txt
|
||||
```
|
||||
|
||||
### 3. Configureaza LM Studio
|
||||
|
||||
1. Deschide LM Studio
|
||||
2. Descarca un model (recomandat: **Llama 3.1 8B Instruct** sau **Mistral 7B Instruct**)
|
||||
3. Du-te la **Local Server** → incarca modelul → **Start Server**
|
||||
|
||||
### 4. Configureaza variabilele de mediu
|
||||
|
||||
```bash
|
||||
copy backend\.env.example backend\.env
|
||||
```
|
||||
|
||||
Editeaza `backend/.env` cu setarile tale (portul LM Studio, modelul Whisper, vocea TTS, etc.)
|
||||
|
||||
### 5. (Optional) Descarca vocea Piper pentru mod offline
|
||||
|
||||
Descarca fisierele vocii romanesti de la [Piper Voices](https://github.com/rhasspy/piper/blob/master/VOICES.md) si pune `.onnx` + `.onnx.json` in `backend/voices/`.
|
||||
|
||||
## Pornire
|
||||
|
||||
Dublu-click pe `start.bat` sau:
|
||||
|
||||
```bash
|
||||
venv\Scripts\activate
|
||||
cd backend
|
||||
python main.py
|
||||
```
|
||||
|
||||
Deschide browserul la **http://localhost:8000**
|
||||
|
||||
## Utilizare
|
||||
|
||||
- **Butonul albastru** — activeaza ascultarea continua. Spune **"EcoBot"** ca sa activezi, apoi pune intrebarea
|
||||
- **Butonul verde** — tine apasat si vorbeste direct
|
||||
- **Butonul ⚙** — setari audio (selectie microfon/difuzor, amplificare, test)
|
||||
|
||||
## Structura proiectului
|
||||
|
||||
```
|
||||
scoala-verde/
|
||||
├── backend/
|
||||
│ ├── main.py # FastAPI server (WebSocket)
|
||||
│ ├── stt.py # Speech-to-Text (faster-whisper)
|
||||
│ ├── llm.py # Comunicare LM Studio API
|
||||
│ ├── tts.py # Text-to-Speech (Edge TTS / Piper)
|
||||
│ ├── voices/ # Modele voce Piper (.onnx)
|
||||
│ ├── requirements.txt
|
||||
│ ├── .env # Configuratie locala (nu se comite)
|
||||
│ └── .env.example # Template configuratie
|
||||
├── frontend/
|
||||
│ ├── index.html # Pagina principala
|
||||
│ ├── css/style.css
|
||||
│ └── js/
|
||||
│ ├── app.js # Logica principala + WebSocket
|
||||
│ ├── audio.js # Captura microfon + redare audio
|
||||
│ ├── character.js # Caracter animat (copac)
|
||||
│ └── settings.js # Pagina setari audio
|
||||
├── dev/ # Documentatie dezvoltare
|
||||
├── start.bat # Script pornire Windows
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Configuratie (.env)
|
||||
|
||||
| Variabila | Default | Descriere |
|
||||
|-----------|---------|-----------|
|
||||
| `LLM_BASE_URL` | `http://localhost:1234/v1` | URL-ul serverului LLM |
|
||||
| `LLM_MODEL` | `local-model` | Numele modelului in LM Studio |
|
||||
| `WHISPER_MODEL` | `small` | Dimensiunea modelului Whisper (tiny/small/medium) |
|
||||
| `WHISPER_DEVICE` | `cuda` | Dispozitiv Whisper (cuda/cpu) |
|
||||
| `TTS_ENGINE` | `edge` | Engine TTS: `edge` (online) sau `piper` (offline) |
|
||||
| `TTS_VOICE` | `ro-RO-EmilNeural` | Vocea Edge TTS |
|
||||
| `SERVER_PORT` | `8000` | Portul serverului |
|
||||
|
||||
## Tehnologii
|
||||
|
||||
- **[faster-whisper](https://github.com/SYSTRAN/faster-whisper)** — Speech-to-Text
|
||||
- **[LM Studio](https://lmstudio.ai/)** — Server LLM local
|
||||
- **[Edge TTS](https://github.com/rany2/edge-tts)** — Text-to-Speech (Microsoft Neural)
|
||||
- **[Piper TTS](https://github.com/rhasspy/piper)** — Text-to-Speech offline
|
||||
- **[FastAPI](https://fastapi.tiangolo.com/)** — Backend Python
|
||||
|
||||
## Licenta
|
||||
|
||||
MIT
|
||||
33
backend/.env.example
Normal file
33
backend/.env.example
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# ================================
|
||||
# EcoBot - Configuration
|
||||
# ================================
|
||||
|
||||
# Hugging Face (for downloading Whisper model on first run)
|
||||
HF_TOKEN=your_token_here
|
||||
|
||||
# LM Studio / LLM API
|
||||
LLM_BASE_URL=http://localhost:1234/v1
|
||||
LLM_API_KEY=not-needed
|
||||
LLM_MODEL=local-model
|
||||
LLM_MAX_TOKENS=200
|
||||
LLM_TEMPERATURE=0.7
|
||||
|
||||
# Whisper STT
|
||||
WHISPER_MODEL=small
|
||||
WHISPER_LANGUAGE=ro
|
||||
|
||||
WHISPER_DEVICE=cuda
|
||||
WHISPER_COMPUTE_TYPE=float16
|
||||
#WHISPER_DEVICE=cpu
|
||||
#WHISPER_COMPUTE_TYPE=int8
|
||||
|
||||
|
||||
# Text-to-Speech
|
||||
# TTS_ENGINE: "edge" (online, calitate superioara) sau "piper" (offline)
|
||||
TTS_ENGINE=edge
|
||||
# Voci Edge: ro-RO-EmilNeural (barbat), ro-RO-AlinaNeural (femeie)
|
||||
TTS_VOICE=ro-RO-EmilNeural
|
||||
|
||||
# Server
|
||||
SERVER_HOST=0.0.0.0
|
||||
SERVER_PORT=8000
|
||||
280
backend/diacritice.py
Normal file
280
backend/diacritice.py
Normal file
|
|
@ -0,0 +1,280 @@
|
|||
"""
|
||||
Post-processing module that adds Romanian diacritics to text.
|
||||
Maps common words without diacritics to their correct forms.
|
||||
"""
|
||||
|
||||
import re
|
||||
|
||||
# Common Romanian words: without diacritics -> with diacritics
|
||||
# Only includes words where the diacriticed form is unambiguous
|
||||
WORD_MAP = {
|
||||
# A
|
||||
"adevarata": "adevărată",
|
||||
"adevarate": "adevărate",
|
||||
"adevarat": "adevărat",
|
||||
"adevaratii": "adevărații",
|
||||
"adancime": "adâncime",
|
||||
"ajuta": "ajuta",
|
||||
"analfabet": "analfabet",
|
||||
"aceasta": "aceasta",
|
||||
"afara": "afară",
|
||||
"altadata": "altădată",
|
||||
"amandoi": "amândoi",
|
||||
"amanare": "amânare",
|
||||
"amintiri": "amintiri",
|
||||
"anumita": "anumită",
|
||||
"anumite": "anumite",
|
||||
"apa": "apă",
|
||||
"apoi": "apoi",
|
||||
"asa": "așa",
|
||||
"asadar": "așadar",
|
||||
"asemanare": "asemănare",
|
||||
"asemenea": "asemenea",
|
||||
"asupra": "asupra",
|
||||
"astazi": "astăzi",
|
||||
"asteapta": "așteaptă",
|
||||
"asteptare": "așteptare",
|
||||
"ati": "ați",
|
||||
"atata": "atâta",
|
||||
"atatia": "atâția",
|
||||
"atunci": "atunci",
|
||||
|
||||
# B
|
||||
"baiatul": "băiatul",
|
||||
"baiat": "băiat",
|
||||
"baieti": "băieți",
|
||||
"baietii": "băieții",
|
||||
"batran": "bătrân",
|
||||
"batrana": "bătrână",
|
||||
"batranete": "bătrânețe",
|
||||
"batrani": "bătrâni",
|
||||
"buna": "bună",
|
||||
|
||||
# C
|
||||
"cand": "când",
|
||||
"candva": "cândva",
|
||||
"cat": "cât",
|
||||
"cata": "câtă",
|
||||
"cate": "câte",
|
||||
"cati": "câți",
|
||||
"cativa": "câțiva",
|
||||
"cateva": "câteva",
|
||||
"catre": "către",
|
||||
"caldura": "căldură",
|
||||
"calduros": "călduros",
|
||||
"calatorie": "călătorie",
|
||||
"calatori": "călători",
|
||||
"cautare": "căutare",
|
||||
"cautam": "căutăm",
|
||||
"casa": "casă",
|
||||
"casuta": "căsuță",
|
||||
"compasiune": "compasiune",
|
||||
"constientiza": "conștientiza",
|
||||
"constiinta": "conștiință",
|
||||
"constient": "conștient",
|
||||
"copacii": "copacii",
|
||||
"curateniei": "curățeniei",
|
||||
"curatenie": "curățenie",
|
||||
|
||||
# D
|
||||
"deasupra": "deasupra",
|
||||
"deseuri": "deșeuri",
|
||||
"deseurile": "deșeurile",
|
||||
"dimineata": "dimineață",
|
||||
"disparitia": "dispariția",
|
||||
"disparitie": "dispariție",
|
||||
"dupa": "după",
|
||||
|
||||
# E
|
||||
"emotie": "emoție",
|
||||
"emotii": "emoții",
|
||||
|
||||
# F
|
||||
"facand": "făcând",
|
||||
"facuta": "făcută",
|
||||
"fara": "fără",
|
||||
"fiinta": "ființă",
|
||||
"fiinte": "ființe",
|
||||
"folosesti": "folosești",
|
||||
"folosim": "folosim",
|
||||
"functie": "funcție",
|
||||
"functioneaza": "funcționează",
|
||||
|
||||
# G
|
||||
"gasesti": "găsești",
|
||||
"gasim": "găsim",
|
||||
"gasit": "găsit",
|
||||
"gandeste": "gândește",
|
||||
"gandire": "gândire",
|
||||
"ganduri": "gânduri",
|
||||
"gradina": "grădină",
|
||||
"gradinile": "grădinile",
|
||||
|
||||
# I
|
||||
"iarasi": "iarăși",
|
||||
"iata": "iată",
|
||||
"inainte": "înainte",
|
||||
"inaintea": "înaintea",
|
||||
"incalzire": "încălzire",
|
||||
"incalzirea": "încălzirea",
|
||||
"incepe": "începe",
|
||||
"incepem": "începem",
|
||||
"inceput": "început",
|
||||
"incerca": "încerca",
|
||||
"inchide": "închide",
|
||||
"inchis": "închis",
|
||||
"inconjuratoare": "înconjurătoare",
|
||||
"indata": "îndată",
|
||||
"indeajuns": "îndeajuns",
|
||||
"indrazneala": "îndrăzneală",
|
||||
"informatii": "informații",
|
||||
"informatie": "informație",
|
||||
"inima": "inimă",
|
||||
"inseamna": "înseamnă",
|
||||
"insemnatate": "însemnătate",
|
||||
"intelege": "înțelege",
|
||||
"intelegem": "înțelegem",
|
||||
"inteles": "înțeles",
|
||||
"intotdeauna": "întotdeauna",
|
||||
"intoarcere": "întoarcere",
|
||||
"intr-un": "într-un",
|
||||
"intr-o": "într-o",
|
||||
"intrebare": "întrebare",
|
||||
"intrebari": "întrebări",
|
||||
"invatam": "învățăm",
|
||||
"invata": "învață",
|
||||
"invatare": "învățare",
|
||||
|
||||
# J
|
||||
"jumatate": "jumătate",
|
||||
|
||||
# L
|
||||
"lantul": "lanțul",
|
||||
"legatura": "legătură",
|
||||
"lumina": "lumină",
|
||||
|
||||
# M
|
||||
"mancare": "mâncare",
|
||||
"mancarea": "mâncarea",
|
||||
"masina": "mașină",
|
||||
"masini": "mașini",
|
||||
"masinile": "mașinile",
|
||||
"mediul": "mediul",
|
||||
"mostenire": "moștenire",
|
||||
"mostenirea": "moștenirea",
|
||||
"muntii": "munții",
|
||||
"munti": "munți",
|
||||
|
||||
# N
|
||||
"natiune": "națiune",
|
||||
"natura": "natură",
|
||||
"naturii": "naturii",
|
||||
|
||||
# O
|
||||
"oamenii": "oamenii",
|
||||
"oras": "oraș",
|
||||
"orasele": "orașele",
|
||||
"orase": "orașe",
|
||||
|
||||
# P
|
||||
"padure": "pădure",
|
||||
"paduri": "păduri",
|
||||
"padurile": "pădurile",
|
||||
"padurea": "pădurea",
|
||||
"pamant": "pământ",
|
||||
"pamantul": "pământul",
|
||||
"pana": "până",
|
||||
"pasari": "păsări",
|
||||
"pasarile": "păsările",
|
||||
"pasare": "pasăre",
|
||||
"planteaza": "plantează",
|
||||
"plantam": "plantăm",
|
||||
"poluare": "poluare",
|
||||
"poluarea": "poluarea",
|
||||
"populatie": "populație",
|
||||
"populatia": "populația",
|
||||
"povesteste": "povestește",
|
||||
"protejeaza": "protejează",
|
||||
"protejam": "protejăm",
|
||||
"protectie": "protecție",
|
||||
"protectia": "protecția",
|
||||
"putina": "puțină",
|
||||
"putin": "puțin",
|
||||
|
||||
# R
|
||||
"ramanem": "rămânem",
|
||||
"ramane": "rămâne",
|
||||
"ramas": "rămas",
|
||||
"raspuns": "răspuns",
|
||||
"raspunsul": "răspunsul",
|
||||
"raspunsuri": "răspunsuri",
|
||||
"raspunde": "răspunde",
|
||||
"rauri": "râuri",
|
||||
"raurile": "râurile",
|
||||
"rau": "râu",
|
||||
"reciclare": "reciclare",
|
||||
"reciclam": "reciclăm",
|
||||
"reciclarea": "reciclarea",
|
||||
"reducerea": "reducerea",
|
||||
"reutilizare": "reutilizare",
|
||||
"reutilizam": "reutilizăm",
|
||||
|
||||
# S
|
||||
"sansa": "șansă",
|
||||
"sanse": "șanse",
|
||||
"scoala": "școală",
|
||||
"scolile": "școlile",
|
||||
"scoalele": "școlile",
|
||||
"seara": "seară",
|
||||
"siguranta": "siguranță",
|
||||
"situatie": "situație",
|
||||
"solutie": "soluție",
|
||||
"solutii": "soluții",
|
||||
"stim": "știm",
|
||||
"stie": "știe",
|
||||
"stiinta": "știință",
|
||||
"stiintific": "științific",
|
||||
"stiati": "știați",
|
||||
"sustenabilitate": "sustenabilitate",
|
||||
|
||||
# T
|
||||
"tara": "țară",
|
||||
"tarile": "țările",
|
||||
"tari": "țări",
|
||||
"temperaturi": "temperaturi",
|
||||
"trebuie": "trebuie",
|
||||
|
||||
# U
|
||||
"unda": "undă",
|
||||
"uneori": "uneori",
|
||||
"usoara": "ușoară",
|
||||
"usor": "ușor",
|
||||
|
||||
# V
|
||||
"vant": "vânt",
|
||||
"vantul": "vântul",
|
||||
"viata": "viață",
|
||||
"vietuitoare": "viețuitoare",
|
||||
"vietuitoarele": "viețuitoarele",
|
||||
}
|
||||
|
||||
|
||||
def add_diacritics(text: str) -> str:
|
||||
"""Add Romanian diacritics to text using word-level replacement."""
|
||||
|
||||
def replace_word(match):
|
||||
word = match.group(0)
|
||||
lower = word.lower()
|
||||
|
||||
if lower in WORD_MAP:
|
||||
replacement = WORD_MAP[lower]
|
||||
# Preserve original capitalization
|
||||
if word[0].isupper():
|
||||
replacement = replacement[0].upper() + replacement[1:]
|
||||
if word.isupper():
|
||||
replacement = replacement.upper()
|
||||
return replacement
|
||||
return word
|
||||
|
||||
# Match whole words only
|
||||
return re.sub(r'\b[a-zA-ZăâîșțĂÂÎȘȚ]+\b', replace_word, text)
|
||||
58
backend/llm.py
Normal file
58
backend/llm.py
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
import os
|
||||
from openai import OpenAI
|
||||
from diacritice import add_diacritics
|
||||
|
||||
client = OpenAI(
|
||||
base_url=os.getenv("LLM_BASE_URL", "http://localhost:1234/v1"),
|
||||
api_key=os.getenv("LLM_API_KEY", "not-needed"),
|
||||
)
|
||||
|
||||
LLM_MODEL = os.getenv("LLM_MODEL", "local-model")
|
||||
LLM_MAX_TOKENS = int(os.getenv("LLM_MAX_TOKENS", "200"))
|
||||
LLM_TEMPERATURE = float(os.getenv("LLM_TEMPERATURE", "0.7"))
|
||||
|
||||
SYSTEM_PROMPT = """You are EcoBot, a friendly assistant for "Scoala Verde" (Green School).
|
||||
You MUST reply ONLY in Romanian language. Never reply in English or any other language.
|
||||
You are passionate about ecology, nature and environmental protection.
|
||||
Keep answers short (maximum 2-3 sentences) because they are read aloud.
|
||||
Do not use emojis or special characters. Speak naturally, like talking to a friend.
|
||||
CRITICAL: You MUST use proper Romanian diacritics in every word: ă, â, î, ș, ț (lowercase) and Ă, Â, Î, Ș, Ț (uppercase). Never write without diacritics.
|
||||
|
||||
Example responses:
|
||||
- User: "Ce este reciclarea?" -> "Reciclarea înseamnă să transformăm deșeurile în materiale noi. Așa protejăm natura și economisim resurse."
|
||||
- User: "Buna ziua" -> "Bună ziua! Sunt EcoBot, asistentul tău verde. Cu ce te pot ajuta?"
|
||||
- User: "Hello" -> "Bună! Eu sunt EcoBot. Cum te pot ajuta să protejăm natura?"
|
||||
"""
|
||||
|
||||
|
||||
# Fixed responses for common greetings (LLM tends to mess these up)
|
||||
FIXED_RESPONSES = {
|
||||
"buna": "Bună! Sunt EcoBot, asistentul tău verde. Cu ce te pot ajuta?",
|
||||
"buna ziua": "Bună ziua! Sunt EcoBot, asistentul tău verde. Cu ce te pot ajuta?",
|
||||
"salut": "Salut! Sunt EcoBot. Cu ce te pot ajuta azi?",
|
||||
"hello": "Bună! Eu sunt EcoBot. Cum te pot ajuta să protejăm natura?",
|
||||
"hi": "Bună! Sunt EcoBot. Întreabă-mă orice despre ecologie!",
|
||||
"hey": "Bună! Sunt EcoBot. Cu ce te pot ajuta?",
|
||||
"ciao": "Bună! Sunt EcoBot, asistentul tău verde. Ce vrei să afli?",
|
||||
"buna seara": "Bună seara! Sunt EcoBot. Cu ce te pot ajuta?",
|
||||
"buna dimineata": "Bună dimineața! Sunt EcoBot. Cu ce te pot ajuta?",
|
||||
}
|
||||
|
||||
|
||||
def get_response(user_message: str) -> str:
|
||||
# Check for fixed responses first
|
||||
clean = user_message.lower().strip().rstrip(".!?")
|
||||
if clean in FIXED_RESPONSES:
|
||||
return FIXED_RESPONSES[clean]
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model=LLM_MODEL,
|
||||
messages=[
|
||||
{"role": "system", "content": SYSTEM_PROMPT},
|
||||
{"role": "user", "content": f"[Răspunde în limba română] {user_message}"},
|
||||
],
|
||||
max_tokens=LLM_MAX_TOKENS,
|
||||
temperature=LLM_TEMPERATURE,
|
||||
)
|
||||
text = response.choices[0].message.content.strip()
|
||||
return add_diacritics(text)
|
||||
132
backend/main.py
Normal file
132
backend/main.py
Normal file
|
|
@ -0,0 +1,132 @@
|
|||
import os
|
||||
import json
|
||||
import base64
|
||||
from pathlib import Path
|
||||
from contextlib import asynccontextmanager
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# Load .env before importing modules that use env vars
|
||||
load_dotenv()
|
||||
|
||||
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from fastapi.responses import FileResponse
|
||||
|
||||
from stt import transcribe, load_model as load_whisper
|
||||
from llm import get_response
|
||||
from tts import synthesize, set_voice, TTS_ENGINE
|
||||
|
||||
# Paths
|
||||
BASE_DIR = Path(__file__).parent.parent
|
||||
FRONTEND_DIR = BASE_DIR / "frontend"
|
||||
VOICES_DIR = BASE_DIR / "backend" / "voices"
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app):
|
||||
# Startup
|
||||
print("Loading Whisper model...")
|
||||
load_whisper()
|
||||
print("Whisper model loaded.")
|
||||
|
||||
voice_files = list(VOICES_DIR.glob("*.onnx")) if VOICES_DIR.exists() else []
|
||||
if voice_files:
|
||||
set_voice(str(voice_files[0]))
|
||||
print(f"TTS voice loaded: {voice_files[0].name}")
|
||||
else:
|
||||
print("WARNING: No voice model found in backend/voices/")
|
||||
print("Download a voice from: https://github.com/rhasspy/piper/blob/master/VOICES.md")
|
||||
print("Place the .onnx and .onnx.json files in backend/voices/")
|
||||
|
||||
yield
|
||||
# Shutdown
|
||||
print("Server shutting down.")
|
||||
|
||||
|
||||
app = FastAPI(title="EcoBot - Scoala Verde", lifespan=lifespan)
|
||||
|
||||
|
||||
# Serve frontend
|
||||
app.mount("/static", StaticFiles(directory=str(FRONTEND_DIR)), name="static")
|
||||
|
||||
|
||||
@app.get("/")
|
||||
async def index():
|
||||
return FileResponse(str(FRONTEND_DIR / "index.html"))
|
||||
|
||||
|
||||
@app.websocket("/ws")
|
||||
async def websocket_endpoint(ws: WebSocket):
|
||||
await ws.accept()
|
||||
print("Client connected")
|
||||
|
||||
try:
|
||||
while True:
|
||||
# Receive audio data from browser
|
||||
data = await ws.receive_text()
|
||||
message = json.loads(data)
|
||||
|
||||
if message["type"] == "audio":
|
||||
audio_bytes = base64.b64decode(message["data"])
|
||||
is_wake_mode = message.get("mode") == "wake"
|
||||
|
||||
# Step 1: Speech to Text
|
||||
await ws.send_text(json.dumps({"type": "status", "text": "Ascult..."}))
|
||||
text = transcribe(audio_bytes)
|
||||
|
||||
if not text:
|
||||
await ws.send_text(json.dumps({"type": "status", "text": "Nu am inteles. Incearca din nou."}))
|
||||
continue
|
||||
|
||||
# Wake word detection
|
||||
if is_wake_mode:
|
||||
text_lower = text.lower().strip()
|
||||
wake_words = ["ecobot", "eco bot", "eco-bot", "hello bot", "helo bot"]
|
||||
detected = any(w in text_lower for w in wake_words)
|
||||
await ws.send_text(json.dumps({
|
||||
"type": "wake_detected" if detected else "wake_not_detected",
|
||||
"text": text,
|
||||
}))
|
||||
continue
|
||||
|
||||
await ws.send_text(json.dumps({"type": "user_text", "text": text}))
|
||||
|
||||
# Step 2: Get AI response
|
||||
await ws.send_text(json.dumps({"type": "status", "text": "Ma gandesc..."}))
|
||||
response_text = get_response(text)
|
||||
await ws.send_text(json.dumps({"type": "bot_text", "text": response_text}))
|
||||
|
||||
# Step 3: Text to Speech
|
||||
await ws.send_text(json.dumps({"type": "status", "text": "Pregatesc raspunsul..."}))
|
||||
print(f"[TTS] Engine: {TTS_ENGINE}")
|
||||
print(f"[TTS] Text to synthesize: {response_text.encode('ascii', 'replace').decode()}")
|
||||
try:
|
||||
audio_response = await synthesize(response_text)
|
||||
print(f"[TTS] OK - {len(audio_response)} bytes")
|
||||
audio_b64 = base64.b64encode(audio_response).decode("utf-8")
|
||||
audio_mime = "audio/mpeg" if TTS_ENGINE == "edge" else "audio/wav"
|
||||
await ws.send_text(json.dumps({
|
||||
"type": "audio_response",
|
||||
"data": audio_b64,
|
||||
"text": response_text,
|
||||
"mime": audio_mime,
|
||||
}))
|
||||
except Exception as e:
|
||||
err_msg = str(e).encode('ascii', 'replace').decode()
|
||||
print(f"[TTS] ERROR: {type(e).__name__}: {err_msg}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
await ws.send_text(json.dumps({
|
||||
"type": "text_only_response",
|
||||
"text": response_text,
|
||||
}))
|
||||
|
||||
except WebSocketDisconnect:
|
||||
print("Client disconnected")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
host = os.getenv("SERVER_HOST", "0.0.0.0")
|
||||
port = int(os.getenv("SERVER_PORT", "8000"))
|
||||
uvicorn.run(app, host=host, port=port)
|
||||
8
backend/requirements.txt
Normal file
8
backend/requirements.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
fastapi==0.115.0
|
||||
uvicorn[standard]==0.30.0
|
||||
websockets==13.0
|
||||
faster-whisper==1.0.3
|
||||
openai==1.50.0
|
||||
piper-tts==1.2.0
|
||||
python-multipart==0.0.9
|
||||
python-dotenv==1.0.1
|
||||
32
backend/stt.py
Normal file
32
backend/stt.py
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
import os
|
||||
import tempfile
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
WHISPER_MODEL = os.getenv("WHISPER_MODEL", "small")
|
||||
WHISPER_DEVICE = os.getenv("WHISPER_DEVICE", "cuda")
|
||||
WHISPER_COMPUTE_TYPE = os.getenv("WHISPER_COMPUTE_TYPE", "float16")
|
||||
WHISPER_LANGUAGE = os.getenv("WHISPER_LANGUAGE", "ro")
|
||||
|
||||
model = None
|
||||
|
||||
|
||||
def load_model():
|
||||
global model
|
||||
if model is None:
|
||||
model = WhisperModel(WHISPER_MODEL, device=WHISPER_DEVICE, compute_type=WHISPER_COMPUTE_TYPE)
|
||||
return model
|
||||
|
||||
|
||||
def transcribe(audio_bytes: bytes) -> str:
|
||||
whisper = load_model()
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
|
||||
f.write(audio_bytes)
|
||||
tmp_path = f.name
|
||||
|
||||
try:
|
||||
segments, _ = whisper.transcribe(tmp_path, language=WHISPER_LANGUAGE)
|
||||
text = " ".join(seg.text for seg in segments).strip()
|
||||
return text
|
||||
finally:
|
||||
os.unlink(tmp_path)
|
||||
80
backend/tts.py
Normal file
80
backend/tts.py
Normal file
|
|
@ -0,0 +1,80 @@
|
|||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import shutil
|
||||
import edge_tts
|
||||
|
||||
# TTS_ENGINE: "edge" (default) or "piper"
|
||||
TTS_ENGINE = os.getenv("TTS_ENGINE", "edge")
|
||||
|
||||
# --- Edge TTS config ---
|
||||
# Romanian voices: ro-RO-EmilNeural (male), ro-RO-AlinaNeural (female)
|
||||
EDGE_VOICE = os.getenv("TTS_VOICE", "ro-RO-EmilNeural")
|
||||
|
||||
# --- Piper TTS config ---
|
||||
PIPER_VOICE_MODEL = None
|
||||
|
||||
|
||||
def _find_piper_exe():
|
||||
piper_path = shutil.which("piper")
|
||||
if piper_path:
|
||||
return piper_path
|
||||
venv_piper = os.path.join(os.path.dirname(os.path.dirname(__file__)), "venv", "Scripts", "piper.exe")
|
||||
if os.path.exists(venv_piper):
|
||||
return venv_piper
|
||||
return "piper"
|
||||
|
||||
|
||||
PIPER_EXE = _find_piper_exe()
|
||||
|
||||
|
||||
def set_voice(model_path: str):
|
||||
"""Set Piper voice model path (called from main.py startup)."""
|
||||
global PIPER_VOICE_MODEL
|
||||
PIPER_VOICE_MODEL = model_path
|
||||
|
||||
|
||||
async def synthesize(text: str) -> bytes:
|
||||
if TTS_ENGINE == "piper":
|
||||
return _synthesize_piper(text)
|
||||
else:
|
||||
return await _synthesize_edge(text)
|
||||
|
||||
|
||||
# --- Edge TTS ---
|
||||
|
||||
async def _synthesize_edge(text: str) -> bytes:
|
||||
with tempfile.NamedTemporaryFile(suffix=".mp3", delete=False) as f:
|
||||
tmp_path = f.name
|
||||
try:
|
||||
communicate = edge_tts.Communicate(text, EDGE_VOICE)
|
||||
await communicate.save(tmp_path)
|
||||
with open(tmp_path, "rb") as f:
|
||||
return f.read()
|
||||
finally:
|
||||
if os.path.exists(tmp_path):
|
||||
os.unlink(tmp_path)
|
||||
|
||||
|
||||
# --- Piper TTS ---
|
||||
|
||||
def _synthesize_piper(text: str) -> bytes:
|
||||
if PIPER_VOICE_MODEL is None:
|
||||
raise RuntimeError("No Piper voice model configured. Place .onnx files in backend/voices/")
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
|
||||
out_path = f.name
|
||||
try:
|
||||
process = subprocess.run(
|
||||
[PIPER_EXE, "--model", PIPER_VOICE_MODEL, "--output_file", out_path],
|
||||
input=text.encode("utf-8"),
|
||||
capture_output=True,
|
||||
timeout=30,
|
||||
)
|
||||
if process.returncode != 0:
|
||||
raise RuntimeError(f"Piper TTS failed: {process.stderr.decode()}")
|
||||
with open(out_path, "rb") as f:
|
||||
return f.read()
|
||||
finally:
|
||||
if os.path.exists(out_path):
|
||||
os.unlink(out_path)
|
||||
0
backend/voices/.gitkeep
Normal file
0
backend/voices/.gitkeep
Normal file
388
frontend/css/style.css
Normal file
388
frontend/css/style.css
Normal file
|
|
@ -0,0 +1,388 @@
|
|||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
background: linear-gradient(135deg, #1a472a 0%, #2d5a27 50%, #1a472a 100%);
|
||||
min-height: 100vh;
|
||||
font-family: 'Segoe UI', Tahoma, sans-serif;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
color: #e0f0e0;
|
||||
}
|
||||
|
||||
#app {
|
||||
text-align: center;
|
||||
padding: 20px;
|
||||
max-width: 500px;
|
||||
width: 100%;
|
||||
position: relative;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 2.5em;
|
||||
color: #7ddf64;
|
||||
text-shadow: 0 2px 10px rgba(125, 223, 100, 0.3);
|
||||
margin-bottom: 5px;
|
||||
}
|
||||
|
||||
.subtitle {
|
||||
color: #a0c8a0;
|
||||
margin-bottom: 30px;
|
||||
font-size: 1.1em;
|
||||
}
|
||||
|
||||
#character-canvas {
|
||||
display: block;
|
||||
margin: 0 auto 20px;
|
||||
border-radius: 20px;
|
||||
background: rgba(0, 0, 0, 0.2);
|
||||
}
|
||||
|
||||
#status {
|
||||
font-size: 1.1em;
|
||||
color: #a0c8a0;
|
||||
margin-bottom: 20px;
|
||||
min-height: 1.5em;
|
||||
transition: color 0.3s;
|
||||
}
|
||||
|
||||
#status.active {
|
||||
color: #7ddf64;
|
||||
}
|
||||
|
||||
#conversation {
|
||||
margin-bottom: 20px;
|
||||
min-height: 60px;
|
||||
}
|
||||
|
||||
.bubble {
|
||||
padding: 12px 18px;
|
||||
border-radius: 16px;
|
||||
margin: 8px 0;
|
||||
max-width: 90%;
|
||||
text-align: left;
|
||||
font-size: 0.95em;
|
||||
line-height: 1.4;
|
||||
transition: opacity 0.3s;
|
||||
}
|
||||
|
||||
.bubble.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.bubble.user {
|
||||
background: rgba(125, 223, 100, 0.15);
|
||||
border: 1px solid rgba(125, 223, 100, 0.3);
|
||||
margin-left: auto;
|
||||
margin-right: 0;
|
||||
}
|
||||
|
||||
.bubble.bot {
|
||||
background: rgba(255, 255, 255, 0.1);
|
||||
border: 1px solid rgba(255, 255, 255, 0.2);
|
||||
margin-left: 0;
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
#record-btn {
|
||||
background: linear-gradient(145deg, #4CAF50, #2d7a32);
|
||||
border: none;
|
||||
color: white;
|
||||
padding: 18px 40px;
|
||||
font-size: 1.1em;
|
||||
border-radius: 50px;
|
||||
cursor: pointer;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
transition: all 0.2s;
|
||||
box-shadow: 0 4px 15px rgba(76, 175, 80, 0.3);
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
#record-btn:hover {
|
||||
transform: scale(1.05);
|
||||
box-shadow: 0 6px 20px rgba(76, 175, 80, 0.5);
|
||||
}
|
||||
|
||||
#record-btn:active, #record-btn.recording {
|
||||
background: linear-gradient(145deg, #f44336, #c62828);
|
||||
box-shadow: 0 4px 15px rgba(244, 67, 54, 0.4);
|
||||
transform: scale(0.98);
|
||||
}
|
||||
|
||||
#record-btn.recording #btn-text {
|
||||
content: "Inregistrez...";
|
||||
}
|
||||
|
||||
#btn-icon, #wake-icon {
|
||||
font-size: 1.4em;
|
||||
}
|
||||
|
||||
#controls {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
#wake-btn {
|
||||
background: linear-gradient(145deg, #1565C0, #0D47A1);
|
||||
border: none;
|
||||
color: white;
|
||||
padding: 14px 30px;
|
||||
font-size: 1em;
|
||||
border-radius: 50px;
|
||||
cursor: pointer;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 10px;
|
||||
transition: all 0.2s;
|
||||
box-shadow: 0 4px 15px rgba(21, 101, 192, 0.3);
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
#wake-btn:hover {
|
||||
transform: scale(1.05);
|
||||
box-shadow: 0 6px 20px rgba(21, 101, 192, 0.5);
|
||||
}
|
||||
|
||||
#wake-btn.listening {
|
||||
background: linear-gradient(145deg, #E65100, #BF360C);
|
||||
box-shadow: 0 4px 15px rgba(230, 81, 0, 0.4);
|
||||
animation: pulse-wake 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse-wake {
|
||||
0%, 100% { box-shadow: 0 4px 15px rgba(230, 81, 0, 0.4); }
|
||||
50% { box-shadow: 0 4px 25px rgba(230, 81, 0, 0.7); }
|
||||
}
|
||||
|
||||
/* Settings button */
|
||||
#settings-btn {
|
||||
position: absolute;
|
||||
top: 20px;
|
||||
right: 20px;
|
||||
background: rgba(255, 255, 255, 0.1);
|
||||
border: 1px solid rgba(255, 255, 255, 0.2);
|
||||
color: #a0c8a0;
|
||||
font-size: 1.5em;
|
||||
width: 45px;
|
||||
height: 45px;
|
||||
border-radius: 50%;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
#settings-btn:hover {
|
||||
background: rgba(255, 255, 255, 0.2);
|
||||
color: #7ddf64;
|
||||
transform: rotate(30deg);
|
||||
}
|
||||
|
||||
/* Modal */
|
||||
.modal {
|
||||
position: fixed;
|
||||
top: 0;
|
||||
left: 0;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
background: rgba(0, 0, 0, 0.7);
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
z-index: 1000;
|
||||
backdrop-filter: blur(5px);
|
||||
}
|
||||
|
||||
.modal.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.modal-content {
|
||||
background: linear-gradient(145deg, #1e3a1e, #2a4a2a);
|
||||
border: 1px solid rgba(125, 223, 100, 0.2);
|
||||
border-radius: 20px;
|
||||
padding: 30px;
|
||||
max-width: 450px;
|
||||
width: 90%;
|
||||
box-shadow: 0 10px 40px rgba(0, 0, 0, 0.5);
|
||||
}
|
||||
|
||||
.modal-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 25px;
|
||||
}
|
||||
|
||||
.modal-header h2 {
|
||||
color: #7ddf64;
|
||||
font-size: 1.4em;
|
||||
}
|
||||
|
||||
.modal-close {
|
||||
background: none;
|
||||
border: none;
|
||||
color: #a0c8a0;
|
||||
font-size: 1.8em;
|
||||
cursor: pointer;
|
||||
padding: 0 5px;
|
||||
line-height: 1;
|
||||
}
|
||||
|
||||
.modal-close:hover {
|
||||
color: #f44336;
|
||||
}
|
||||
|
||||
.setting-group {
|
||||
margin-bottom: 25px;
|
||||
}
|
||||
|
||||
.setting-group label {
|
||||
display: block;
|
||||
color: #a0c8a0;
|
||||
font-size: 0.9em;
|
||||
margin-bottom: 8px;
|
||||
text-transform: uppercase;
|
||||
letter-spacing: 1px;
|
||||
}
|
||||
|
||||
.setting-group select {
|
||||
width: 100%;
|
||||
padding: 10px 12px;
|
||||
background: rgba(0, 0, 0, 0.3);
|
||||
border: 1px solid rgba(125, 223, 100, 0.3);
|
||||
border-radius: 10px;
|
||||
color: #e0f0e0;
|
||||
font-size: 0.95em;
|
||||
cursor: pointer;
|
||||
outline: none;
|
||||
}
|
||||
|
||||
.setting-group select:focus {
|
||||
border-color: #7ddf64;
|
||||
}
|
||||
|
||||
.setting-group select option {
|
||||
background: #1e3a1e;
|
||||
color: #e0f0e0;
|
||||
}
|
||||
|
||||
.sub-label {
|
||||
margin-top: 12px;
|
||||
font-size: 0.85em !important;
|
||||
text-transform: none !important;
|
||||
letter-spacing: 0 !important;
|
||||
}
|
||||
|
||||
#gain-value {
|
||||
color: #7ddf64;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
.slider {
|
||||
width: 100%;
|
||||
height: 6px;
|
||||
-webkit-appearance: none;
|
||||
appearance: none;
|
||||
background: rgba(0, 0, 0, 0.3);
|
||||
border-radius: 3px;
|
||||
outline: none;
|
||||
margin: 8px 0;
|
||||
}
|
||||
|
||||
.slider::-webkit-slider-thumb {
|
||||
-webkit-appearance: none;
|
||||
appearance: none;
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border-radius: 50%;
|
||||
background: #7ddf64;
|
||||
cursor: pointer;
|
||||
box-shadow: 0 2px 6px rgba(0, 0, 0, 0.3);
|
||||
}
|
||||
|
||||
.slider::-moz-range-thumb {
|
||||
width: 20px;
|
||||
height: 20px;
|
||||
border-radius: 50%;
|
||||
background: #7ddf64;
|
||||
cursor: pointer;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.test-row {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
margin-top: 10px;
|
||||
}
|
||||
|
||||
.test-btn {
|
||||
background: rgba(76, 175, 80, 0.2);
|
||||
border: 1px solid rgba(76, 175, 80, 0.4);
|
||||
color: #7ddf64;
|
||||
padding: 8px 16px;
|
||||
border-radius: 8px;
|
||||
cursor: pointer;
|
||||
font-size: 0.85em;
|
||||
white-space: nowrap;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.test-btn:hover {
|
||||
background: rgba(76, 175, 80, 0.35);
|
||||
}
|
||||
|
||||
.test-btn.active {
|
||||
background: rgba(244, 67, 54, 0.2);
|
||||
border-color: rgba(244, 67, 54, 0.4);
|
||||
color: #f44336;
|
||||
}
|
||||
|
||||
.level-bar {
|
||||
flex: 1;
|
||||
height: 12px;
|
||||
background: rgba(0, 0, 0, 0.3);
|
||||
border-radius: 6px;
|
||||
overflow: hidden;
|
||||
border: 1px solid rgba(255, 255, 255, 0.1);
|
||||
}
|
||||
|
||||
.level-fill {
|
||||
height: 100%;
|
||||
width: 0%;
|
||||
background: linear-gradient(90deg, #4CAF50, #7ddf64, #f44336);
|
||||
border-radius: 6px;
|
||||
transition: width 0.05s;
|
||||
}
|
||||
|
||||
.test-status {
|
||||
color: #a0c8a0;
|
||||
font-size: 0.8em;
|
||||
margin-top: 6px;
|
||||
min-height: 1.2em;
|
||||
}
|
||||
|
||||
.save-btn {
|
||||
width: 100%;
|
||||
padding: 12px;
|
||||
background: linear-gradient(145deg, #4CAF50, #2d7a32);
|
||||
border: none;
|
||||
color: white;
|
||||
font-size: 1em;
|
||||
border-radius: 10px;
|
||||
cursor: pointer;
|
||||
transition: all 0.2s;
|
||||
box-shadow: 0 4px 15px rgba(76, 175, 80, 0.3);
|
||||
}
|
||||
|
||||
.save-btn:hover {
|
||||
transform: scale(1.02);
|
||||
box-shadow: 0 6px 20px rgba(76, 175, 80, 0.5);
|
||||
}
|
||||
79
frontend/index.html
Normal file
79
frontend/index.html
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
<!DOCTYPE html>
|
||||
<html lang="ro">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>EcoBot - Scoala Verde</title>
|
||||
<link rel="stylesheet" href="/static/css/style.css">
|
||||
</head>
|
||||
<body>
|
||||
<div id="app">
|
||||
<h1>EcoBot</h1>
|
||||
<p class="subtitle">Asistentul tau verde</p>
|
||||
|
||||
<button id="settings-btn" title="Setari audio">⚙</button>
|
||||
|
||||
<canvas id="character-canvas" width="400" height="400"></canvas>
|
||||
|
||||
<div id="status">Apasa butonul si vorbeste</div>
|
||||
|
||||
<div id="conversation">
|
||||
<div id="user-text" class="bubble user hidden"></div>
|
||||
<div id="bot-text" class="bubble bot hidden"></div>
|
||||
</div>
|
||||
|
||||
<div id="controls">
|
||||
<button id="wake-btn" title="Activeaza ascultarea continua">
|
||||
<span id="wake-icon">👂</span>
|
||||
<span id="wake-text">Spune "EcoBot" pentru a activa</span>
|
||||
</button>
|
||||
<button id="record-btn">
|
||||
<span id="btn-icon">🎤</span>
|
||||
<span id="btn-text">Sau tine apasat si vorbeste</span>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Settings Modal -->
|
||||
<div id="settings-modal" class="modal hidden">
|
||||
<div class="modal-content">
|
||||
<div class="modal-header">
|
||||
<h2>Setari Audio</h2>
|
||||
<button id="settings-close" class="modal-close">×</button>
|
||||
</div>
|
||||
|
||||
<div class="setting-group">
|
||||
<label for="mic-select">Microfon</label>
|
||||
<select id="mic-select"></select>
|
||||
|
||||
<label for="mic-gain" class="sub-label">Amplificare microfon: <span id="gain-value">3.0</span>x</label>
|
||||
<input type="range" id="mic-gain" min="1" max="10" step="0.5" value="3.0" class="slider">
|
||||
|
||||
<div class="test-row">
|
||||
<button id="mic-test-btn" class="test-btn">Testeaza microfonul</button>
|
||||
<div id="mic-level-bar" class="level-bar">
|
||||
<div id="mic-level" class="level-fill"></div>
|
||||
</div>
|
||||
</div>
|
||||
<p id="mic-test-status" class="test-status"></p>
|
||||
</div>
|
||||
|
||||
<div class="setting-group">
|
||||
<label for="speaker-select">Difuzor</label>
|
||||
<select id="speaker-select"></select>
|
||||
<div class="test-row">
|
||||
<button id="speaker-test-btn" class="test-btn">Testeaza difuzorul</button>
|
||||
</div>
|
||||
<p id="speaker-test-status" class="test-status"></p>
|
||||
</div>
|
||||
|
||||
<button id="settings-save" class="save-btn">Salveaza si inchide</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="/static/js/character.js"></script>
|
||||
<script src="/static/js/audio.js"></script>
|
||||
<script src="/static/js/settings.js"></script>
|
||||
<script src="/static/js/app.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
307
frontend/js/app.js
Normal file
307
frontend/js/app.js
Normal file
|
|
@ -0,0 +1,307 @@
|
|||
const character = new Character('character-canvas');
|
||||
const audioHandler = new AudioHandler();
|
||||
|
||||
const recordBtn = document.getElementById('record-btn');
|
||||
const btnText = document.getElementById('btn-text');
|
||||
const wakeBtn = document.getElementById('wake-btn');
|
||||
const wakeText = document.getElementById('wake-text');
|
||||
const statusEl = document.getElementById('status');
|
||||
const userTextEl = document.getElementById('user-text');
|
||||
const botTextEl = document.getElementById('bot-text');
|
||||
|
||||
let ws = null;
|
||||
let wakeListening = false;
|
||||
let wakeRecorder = null;
|
||||
let wakeStream = null;
|
||||
let processingAudio = false;
|
||||
let waitingForWakeResponse = false;
|
||||
|
||||
function connectWebSocket() {
|
||||
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
ws = new WebSocket(`${protocol}//${window.location.host}/ws`);
|
||||
|
||||
ws.onopen = () => {
|
||||
statusEl.textContent = 'Conectat! Spune "EcoBot" sau apasa butonul.';
|
||||
};
|
||||
|
||||
ws.onclose = () => {
|
||||
statusEl.textContent = 'Deconectat. Se reconecteaza...';
|
||||
setTimeout(connectWebSocket, 2000);
|
||||
};
|
||||
|
||||
ws.onmessage = async (event) => {
|
||||
const message = JSON.parse(event.data);
|
||||
|
||||
switch (message.type) {
|
||||
case 'status':
|
||||
statusEl.textContent = message.text;
|
||||
if (message.text.includes('Ascult')) {
|
||||
character.setState('listening');
|
||||
} else if (message.text.includes('gandesc')) {
|
||||
character.setState('thinking');
|
||||
}
|
||||
break;
|
||||
|
||||
case 'user_text':
|
||||
userTextEl.textContent = `Tu: ${message.text}`;
|
||||
userTextEl.classList.remove('hidden');
|
||||
break;
|
||||
|
||||
case 'bot_text':
|
||||
botTextEl.textContent = `EcoBot: ${message.text}`;
|
||||
botTextEl.classList.remove('hidden');
|
||||
break;
|
||||
|
||||
case 'audio_response':
|
||||
character.setState('talking');
|
||||
statusEl.textContent = 'Vorbesc...';
|
||||
await audioHandler.playAudioFromBase64(message.data, message.mime || 'audio/mpeg');
|
||||
character.setState('idle');
|
||||
processingAudio = false;
|
||||
resumeWakeIfActive();
|
||||
break;
|
||||
|
||||
case 'text_only_response':
|
||||
botTextEl.textContent = `EcoBot: ${message.text}`;
|
||||
botTextEl.classList.remove('hidden');
|
||||
character.setState('talking');
|
||||
await new Promise(r => setTimeout(r, 2000));
|
||||
character.setState('idle');
|
||||
processingAudio = false;
|
||||
resumeWakeIfActive();
|
||||
break;
|
||||
|
||||
case 'wake_detected':
|
||||
waitingForWakeResponse = false;
|
||||
statusEl.textContent = 'Te-am auzit! Vorbeste acum...';
|
||||
character.setState('listening');
|
||||
await recordAndSendQuestion();
|
||||
break;
|
||||
|
||||
case 'wake_not_detected':
|
||||
waitingForWakeResponse = false;
|
||||
// Only start next chunk after this one is done
|
||||
if (wakeListening && !processingAudio) {
|
||||
startWakeLoop();
|
||||
}
|
||||
break;
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
function resumeWakeIfActive() {
|
||||
if (wakeListening) {
|
||||
statusEl.textContent = 'Ascult... Spune "EcoBot" pentru a activa.';
|
||||
startWakeLoop();
|
||||
} else {
|
||||
statusEl.textContent = 'Spune "EcoBot" sau apasa butonul.';
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Wake Word Mode ----
|
||||
|
||||
function toggleWakeMode() {
|
||||
if (wakeListening) {
|
||||
stopWakeMode();
|
||||
} else {
|
||||
startWakeMode();
|
||||
}
|
||||
}
|
||||
|
||||
function startWakeMode() {
|
||||
wakeListening = true;
|
||||
waitingForWakeResponse = false;
|
||||
wakeBtn.classList.add('listening');
|
||||
wakeText.textContent = 'Ascult... (click pentru a opri)';
|
||||
statusEl.textContent = 'Ascult... Spune "EcoBot" pentru a activa.';
|
||||
character.setState('idle');
|
||||
startWakeLoop();
|
||||
}
|
||||
|
||||
function stopWakeMode() {
|
||||
wakeListening = false;
|
||||
waitingForWakeResponse = false;
|
||||
wakeBtn.classList.remove('listening');
|
||||
wakeText.textContent = 'Spune "EcoBot" pentru a activa';
|
||||
statusEl.textContent = 'Spune "EcoBot" sau apasa butonul.';
|
||||
stopWakeRecorder();
|
||||
}
|
||||
|
||||
async function startWakeLoop() {
|
||||
// Don't start a new recording if we're still waiting for the previous response
|
||||
if (!wakeListening || processingAudio || waitingForWakeResponse) return;
|
||||
|
||||
const micId = audioHandler.selectedMicId;
|
||||
const constraints = {
|
||||
audio: micId
|
||||
? { deviceId: { exact: micId }, autoGainControl: true, noiseSuppression: true }
|
||||
: { autoGainControl: true, noiseSuppression: true },
|
||||
};
|
||||
|
||||
try {
|
||||
wakeStream = await navigator.mediaDevices.getUserMedia(constraints);
|
||||
wakeRecorder = new MediaRecorder(wakeStream, { mimeType: 'audio/webm' });
|
||||
const chunks = [];
|
||||
|
||||
wakeRecorder.ondataavailable = (e) => chunks.push(e.data);
|
||||
|
||||
wakeRecorder.onstop = async () => {
|
||||
if (!wakeListening) return;
|
||||
|
||||
// Mark as waiting — don't record another chunk until server responds
|
||||
waitingForWakeResponse = true;
|
||||
statusEl.textContent = 'Procesez...';
|
||||
|
||||
const blob = new Blob(chunks, { type: 'audio/webm' });
|
||||
const wavBlob = await audioHandler.convertToWav(blob);
|
||||
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
const base64 = reader.result.split(',')[1];
|
||||
if (ws && ws.readyState === WebSocket.OPEN) {
|
||||
ws.send(JSON.stringify({ type: 'audio', data: base64, mode: 'wake' }));
|
||||
}
|
||||
};
|
||||
reader.readAsDataURL(wavBlob);
|
||||
};
|
||||
|
||||
wakeRecorder.start();
|
||||
statusEl.textContent = 'Ascult... Spune "EcoBot"';
|
||||
|
||||
// Record 3 second chunks
|
||||
setTimeout(() => {
|
||||
if (wakeRecorder && wakeRecorder.state === 'recording') {
|
||||
wakeRecorder.stop();
|
||||
if (wakeStream) {
|
||||
wakeStream.getTracks().forEach(t => t.stop());
|
||||
}
|
||||
}
|
||||
}, 3000);
|
||||
} catch (e) {
|
||||
console.error('Wake mode error:', e);
|
||||
statusEl.textContent = 'Eroare microfon: ' + e.message;
|
||||
stopWakeMode();
|
||||
}
|
||||
}
|
||||
|
||||
function stopWakeRecorder() {
|
||||
if (wakeRecorder && wakeRecorder.state === 'recording') {
|
||||
wakeRecorder.stop();
|
||||
}
|
||||
if (wakeStream) {
|
||||
wakeStream.getTracks().forEach(t => t.stop());
|
||||
wakeStream = null;
|
||||
}
|
||||
wakeRecorder = null;
|
||||
}
|
||||
|
||||
async function recordAndSendQuestion() {
|
||||
processingAudio = true;
|
||||
stopWakeRecorder();
|
||||
|
||||
const micId = audioHandler.selectedMicId;
|
||||
const constraints = {
|
||||
audio: micId
|
||||
? { deviceId: { exact: micId }, autoGainControl: true, noiseSuppression: true }
|
||||
: { autoGainControl: true, noiseSuppression: true },
|
||||
};
|
||||
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia(constraints);
|
||||
const recorder = new MediaRecorder(stream, { mimeType: 'audio/webm' });
|
||||
const chunks = [];
|
||||
|
||||
recorder.ondataavailable = (e) => chunks.push(e.data);
|
||||
|
||||
recorder.onstop = async () => {
|
||||
stream.getTracks().forEach(t => t.stop());
|
||||
const blob = new Blob(chunks, { type: 'audio/webm' });
|
||||
const wavBlob = await audioHandler.convertToWav(blob);
|
||||
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
const base64 = reader.result.split(',')[1];
|
||||
if (ws && ws.readyState === WebSocket.OPEN) {
|
||||
ws.send(JSON.stringify({ type: 'audio', data: base64 }));
|
||||
}
|
||||
};
|
||||
reader.readAsDataURL(wavBlob);
|
||||
};
|
||||
|
||||
statusEl.textContent = 'Te ascult... (5 secunde)';
|
||||
recorder.start();
|
||||
|
||||
setTimeout(() => {
|
||||
if (recorder.state === 'recording') {
|
||||
recorder.stop();
|
||||
statusEl.textContent = 'Procesez...';
|
||||
}
|
||||
}, 5000);
|
||||
} catch (e) {
|
||||
console.error('Record error:', e);
|
||||
processingAudio = false;
|
||||
if (wakeListening) startWakeLoop();
|
||||
}
|
||||
}
|
||||
|
||||
// ---- Manual Record (hold button) ----
|
||||
|
||||
async function startRecording() {
|
||||
stopWakeRecorder();
|
||||
await audioHandler.init();
|
||||
audioHandler.startRecording();
|
||||
recordBtn.classList.add('recording');
|
||||
btnText.textContent = 'Inregistrez... elibereaza pentru a trimite';
|
||||
character.setState('listening');
|
||||
statusEl.textContent = 'Te ascult...';
|
||||
}
|
||||
|
||||
async function stopRecording() {
|
||||
const audioBlob = await audioHandler.stopRecording();
|
||||
recordBtn.classList.remove('recording');
|
||||
btnText.textContent = 'Sau tine apasat si vorbeste';
|
||||
processingAudio = true;
|
||||
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
const base64 = reader.result.split(',')[1];
|
||||
if (ws && ws.readyState === WebSocket.OPEN) {
|
||||
ws.send(JSON.stringify({ type: 'audio', data: base64 }));
|
||||
}
|
||||
};
|
||||
reader.readAsDataURL(audioBlob);
|
||||
}
|
||||
|
||||
// ---- Event Listeners ----
|
||||
|
||||
wakeBtn.addEventListener('click', () => toggleWakeMode());
|
||||
|
||||
recordBtn.addEventListener('mousedown', (e) => {
|
||||
e.preventDefault();
|
||||
startRecording();
|
||||
});
|
||||
|
||||
recordBtn.addEventListener('mouseup', (e) => {
|
||||
e.preventDefault();
|
||||
if (audioHandler.isRecording) stopRecording();
|
||||
});
|
||||
|
||||
recordBtn.addEventListener('mouseleave', (e) => {
|
||||
if (audioHandler.isRecording) stopRecording();
|
||||
});
|
||||
|
||||
recordBtn.addEventListener('touchstart', (e) => {
|
||||
e.preventDefault();
|
||||
startRecording();
|
||||
});
|
||||
|
||||
recordBtn.addEventListener('touchend', (e) => {
|
||||
e.preventDefault();
|
||||
if (audioHandler.isRecording) stopRecording();
|
||||
});
|
||||
|
||||
// Settings
|
||||
const settings = new SettingsManager(audioHandler);
|
||||
|
||||
// Start
|
||||
connectWebSocket();
|
||||
172
frontend/js/audio.js
Normal file
172
frontend/js/audio.js
Normal file
|
|
@ -0,0 +1,172 @@
|
|||
class AudioHandler {
|
||||
constructor() {
|
||||
this.mediaRecorder = null;
|
||||
this.audioChunks = [];
|
||||
this.isRecording = false;
|
||||
this.audioContext = null;
|
||||
this.selectedMicId = localStorage.getItem('selectedMicId') || null;
|
||||
this.selectedSpeakerId = localStorage.getItem('selectedSpeakerId') || null;
|
||||
this.micGain = parseFloat(localStorage.getItem('micGain') || '3.0');
|
||||
}
|
||||
|
||||
async init() {
|
||||
const constraints = {
|
||||
audio: this.selectedMicId
|
||||
? {
|
||||
deviceId: { exact: this.selectedMicId },
|
||||
autoGainControl: true,
|
||||
noiseSuppression: true,
|
||||
echoCancellation: true,
|
||||
}
|
||||
: {
|
||||
autoGainControl: true,
|
||||
noiseSuppression: true,
|
||||
echoCancellation: true,
|
||||
},
|
||||
};
|
||||
|
||||
const stream = await navigator.mediaDevices.getUserMedia(constraints);
|
||||
|
||||
// Apply gain boost through Web Audio API
|
||||
const ctx = new AudioContext();
|
||||
const source = ctx.createMediaStreamSource(stream);
|
||||
const gainNode = ctx.createGain();
|
||||
gainNode.gain.value = this.micGain;
|
||||
const dest = ctx.createMediaStreamDestination();
|
||||
|
||||
source.connect(gainNode);
|
||||
gainNode.connect(dest);
|
||||
|
||||
this._liveAudioCtx = ctx;
|
||||
this._liveGainNode = gainNode;
|
||||
|
||||
this.mediaRecorder = new MediaRecorder(dest.stream, { mimeType: 'audio/webm' });
|
||||
|
||||
this.mediaRecorder.ondataavailable = (event) => {
|
||||
this.audioChunks.push(event.data);
|
||||
};
|
||||
}
|
||||
|
||||
setMic(deviceId) {
|
||||
this.selectedMicId = deviceId;
|
||||
localStorage.setItem('selectedMicId', deviceId);
|
||||
}
|
||||
|
||||
setSpeaker(deviceId) {
|
||||
this.selectedSpeakerId = deviceId;
|
||||
localStorage.setItem('selectedSpeakerId', deviceId);
|
||||
}
|
||||
|
||||
setGain(value) {
|
||||
this.micGain = value;
|
||||
localStorage.setItem('micGain', value.toString());
|
||||
if (this._liveGainNode) {
|
||||
this._liveGainNode.gain.value = value;
|
||||
}
|
||||
}
|
||||
|
||||
startRecording() {
|
||||
this.audioChunks = [];
|
||||
this.mediaRecorder.start();
|
||||
this.isRecording = true;
|
||||
}
|
||||
|
||||
stopRecording() {
|
||||
return new Promise((resolve) => {
|
||||
this.mediaRecorder.onstop = async () => {
|
||||
const blob = new Blob(this.audioChunks, { type: 'audio/webm' });
|
||||
const wavBlob = await this.convertToWav(blob);
|
||||
resolve(wavBlob);
|
||||
};
|
||||
this.mediaRecorder.stop();
|
||||
this.isRecording = false;
|
||||
});
|
||||
}
|
||||
|
||||
async convertToWav(blob) {
|
||||
if (!this.audioContext) {
|
||||
this.audioContext = new AudioContext({ sampleRate: 16000 });
|
||||
}
|
||||
|
||||
const arrayBuffer = await blob.arrayBuffer();
|
||||
const audioBuffer = await this.audioContext.decodeAudioData(arrayBuffer);
|
||||
|
||||
const numChannels = 1;
|
||||
const sampleRate = 16000;
|
||||
const bitsPerSample = 16;
|
||||
|
||||
const offlineCtx = new OfflineAudioContext(numChannels, audioBuffer.duration * sampleRate, sampleRate);
|
||||
const source = offlineCtx.createBufferSource();
|
||||
source.buffer = audioBuffer;
|
||||
source.connect(offlineCtx.destination);
|
||||
source.start();
|
||||
|
||||
const resampled = await offlineCtx.startRendering();
|
||||
const samples = resampled.getChannelData(0);
|
||||
|
||||
// Normalize audio - find peak and scale up
|
||||
let peak = 0;
|
||||
for (let i = 0; i < samples.length; i++) {
|
||||
const abs = Math.abs(samples[i]);
|
||||
if (abs > peak) peak = abs;
|
||||
}
|
||||
const normalizeGain = peak > 0.01 ? 0.9 / peak : 1;
|
||||
|
||||
const dataLength = samples.length * (bitsPerSample / 8);
|
||||
const buffer = new ArrayBuffer(44 + dataLength);
|
||||
const view = new DataView(buffer);
|
||||
|
||||
this.writeString(view, 0, 'RIFF');
|
||||
view.setUint32(4, 36 + dataLength, true);
|
||||
this.writeString(view, 8, 'WAVE');
|
||||
this.writeString(view, 12, 'fmt ');
|
||||
view.setUint32(16, 16, true);
|
||||
view.setUint16(20, 1, true);
|
||||
view.setUint16(22, numChannels, true);
|
||||
view.setUint32(24, sampleRate, true);
|
||||
view.setUint32(28, sampleRate * numChannels * (bitsPerSample / 8), true);
|
||||
view.setUint16(32, numChannels * (bitsPerSample / 8), true);
|
||||
view.setUint16(34, bitsPerSample, true);
|
||||
this.writeString(view, 36, 'data');
|
||||
view.setUint32(40, dataLength, true);
|
||||
|
||||
let offset = 44;
|
||||
for (let i = 0; i < samples.length; i++) {
|
||||
const amplified = samples[i] * normalizeGain;
|
||||
const s = Math.max(-1, Math.min(1, amplified));
|
||||
view.setInt16(offset, s < 0 ? s * 0x8000 : s * 0x7FFF, true);
|
||||
offset += 2;
|
||||
}
|
||||
|
||||
return new Blob([buffer], { type: 'audio/wav' });
|
||||
}
|
||||
|
||||
writeString(view, offset, string) {
|
||||
for (let i = 0; i < string.length; i++) {
|
||||
view.setUint8(offset + i, string.charCodeAt(i));
|
||||
}
|
||||
}
|
||||
|
||||
playAudioFromBase64(base64Data, mimeType = 'audio/mpeg') {
|
||||
return new Promise((resolve) => {
|
||||
const byteChars = atob(base64Data);
|
||||
const byteArray = new Uint8Array(byteChars.length);
|
||||
for (let i = 0; i < byteChars.length; i++) {
|
||||
byteArray[i] = byteChars.charCodeAt(i);
|
||||
}
|
||||
const blob = new Blob([byteArray], { type: mimeType });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const audio = new Audio(url);
|
||||
|
||||
if (this.selectedSpeakerId && audio.setSinkId) {
|
||||
audio.setSinkId(this.selectedSpeakerId).catch(() => {});
|
||||
}
|
||||
|
||||
audio.onended = () => {
|
||||
URL.revokeObjectURL(url);
|
||||
resolve();
|
||||
};
|
||||
audio.play();
|
||||
});
|
||||
}
|
||||
}
|
||||
290
frontend/js/character.js
Normal file
290
frontend/js/character.js
Normal file
|
|
@ -0,0 +1,290 @@
|
|||
class Character {
|
||||
constructor(canvasId) {
|
||||
this.canvas = document.getElementById(canvasId);
|
||||
this.ctx = this.canvas.getContext('2d');
|
||||
this.width = this.canvas.width;
|
||||
this.height = this.canvas.height;
|
||||
|
||||
// Character state
|
||||
this.state = 'idle'; // idle, listening, thinking, talking
|
||||
this.mouthOpen = 0;
|
||||
this.eyeBlink = 0;
|
||||
this.blinkTimer = 0;
|
||||
this.bounceOffset = 0;
|
||||
this.bounceSpeed = 0.02;
|
||||
this.time = 0;
|
||||
|
||||
// Leaf particles
|
||||
this.leaves = [];
|
||||
for (let i = 0; i < 5; i++) {
|
||||
this.leaves.push({
|
||||
x: Math.random() * this.width,
|
||||
y: Math.random() * this.height,
|
||||
size: 5 + Math.random() * 8,
|
||||
speed: 0.3 + Math.random() * 0.5,
|
||||
wobble: Math.random() * Math.PI * 2,
|
||||
});
|
||||
}
|
||||
|
||||
this.animate();
|
||||
}
|
||||
|
||||
setState(state) {
|
||||
this.state = state;
|
||||
}
|
||||
|
||||
setMouthOpen(value) {
|
||||
this.mouthOpen = Math.max(0, Math.min(1, value));
|
||||
}
|
||||
|
||||
animate() {
|
||||
this.time += 1;
|
||||
this.update();
|
||||
this.draw();
|
||||
requestAnimationFrame(() => this.animate());
|
||||
}
|
||||
|
||||
update() {
|
||||
// Bounce
|
||||
this.bounceOffset = Math.sin(this.time * this.bounceSpeed) * 5;
|
||||
|
||||
// Blink
|
||||
this.blinkTimer++;
|
||||
if (this.blinkTimer > 150 + Math.random() * 100) {
|
||||
this.eyeBlink = 1;
|
||||
this.blinkTimer = 0;
|
||||
}
|
||||
if (this.eyeBlink > 0) {
|
||||
this.eyeBlink -= 0.15;
|
||||
if (this.eyeBlink < 0) this.eyeBlink = 0;
|
||||
}
|
||||
|
||||
// Mouth animation when talking
|
||||
if (this.state === 'talking') {
|
||||
this.mouthOpen = 0.3 + Math.sin(this.time * 0.3) * 0.3 + Math.sin(this.time * 0.17) * 0.2;
|
||||
} else if (this.state !== 'talking') {
|
||||
this.mouthOpen *= 0.9;
|
||||
}
|
||||
|
||||
// Update leaves
|
||||
this.leaves.forEach(leaf => {
|
||||
leaf.y += leaf.speed;
|
||||
leaf.x += Math.sin(leaf.wobble + this.time * 0.02) * 0.5;
|
||||
leaf.wobble += 0.01;
|
||||
if (leaf.y > this.height + 10) {
|
||||
leaf.y = -10;
|
||||
leaf.x = Math.random() * this.width;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
draw() {
|
||||
const ctx = this.ctx;
|
||||
const cx = this.width / 2;
|
||||
const cy = this.height / 2 + 30;
|
||||
|
||||
ctx.clearRect(0, 0, this.width, this.height);
|
||||
|
||||
// Draw falling leaves
|
||||
this.drawLeaves(ctx);
|
||||
|
||||
const by = cy + this.bounceOffset;
|
||||
|
||||
// Body - a friendly tree trunk shape
|
||||
this.drawBody(ctx, cx, by);
|
||||
|
||||
// Eyes
|
||||
this.drawEyes(ctx, cx, by);
|
||||
|
||||
// Mouth
|
||||
this.drawMouth(ctx, cx, by);
|
||||
|
||||
// Leaves on top (hair)
|
||||
this.drawCrown(ctx, cx, by);
|
||||
|
||||
// Status indicator
|
||||
this.drawStatusGlow(ctx, cx, by);
|
||||
}
|
||||
|
||||
drawBody(ctx, cx, cy) {
|
||||
// Tree trunk body
|
||||
ctx.fillStyle = '#8B6914';
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(cx - 40, cy + 60);
|
||||
ctx.quadraticCurveTo(cx - 50, cy - 20, cx - 30, cy - 60);
|
||||
ctx.quadraticCurveTo(cx, cy - 75, cx + 30, cy - 60);
|
||||
ctx.quadraticCurveTo(cx + 50, cy - 20, cx + 40, cy + 60);
|
||||
ctx.closePath();
|
||||
ctx.fill();
|
||||
|
||||
// Lighter bark detail
|
||||
ctx.fillStyle = '#A07828';
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(cx - 25, cy + 60);
|
||||
ctx.quadraticCurveTo(cx - 30, cy - 10, cx - 15, cy - 50);
|
||||
ctx.quadraticCurveTo(cx, cy - 60, cx + 15, cy - 50);
|
||||
ctx.quadraticCurveTo(cx + 30, cy - 10, cx + 25, cy + 60);
|
||||
ctx.closePath();
|
||||
ctx.fill();
|
||||
|
||||
// Small arms (branches)
|
||||
ctx.strokeStyle = '#8B6914';
|
||||
ctx.lineWidth = 6;
|
||||
ctx.lineCap = 'round';
|
||||
|
||||
// Left arm
|
||||
const armWave = Math.sin(this.time * 0.05) * 5;
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(cx - 35, cy - 10);
|
||||
ctx.quadraticCurveTo(cx - 65, cy - 25 + armWave, cx - 70, cy - 40 + armWave);
|
||||
ctx.stroke();
|
||||
|
||||
// Right arm
|
||||
ctx.beginPath();
|
||||
ctx.moveTo(cx + 35, cy - 10);
|
||||
ctx.quadraticCurveTo(cx + 65, cy - 25 - armWave, cx + 70, cy - 40 - armWave);
|
||||
ctx.stroke();
|
||||
|
||||
// Small leaves on arm tips
|
||||
ctx.fillStyle = '#4CAF50';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx - 73, cy - 43 + armWave, 8, 5, -0.5, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + 73, cy - 43 - armWave, 8, 5, 0.5, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
|
||||
// Feet (roots)
|
||||
ctx.fillStyle = '#8B6914';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx - 20, cy + 65, 18, 8, -0.2, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + 20, cy + 65, 18, 8, 0.2, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
}
|
||||
|
||||
drawEyes(ctx, cx, cy) {
|
||||
const eyeY = cy - 25;
|
||||
const eyeSpacing = 18;
|
||||
const eyeHeight = this.eyeBlink > 0.5 ? 2 : 10;
|
||||
|
||||
// White of eyes
|
||||
ctx.fillStyle = '#FFFFFF';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx - eyeSpacing, eyeY, 10, eyeHeight, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + eyeSpacing, eyeY, 10, eyeHeight, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
|
||||
if (this.eyeBlink <= 0.5) {
|
||||
// Pupils
|
||||
ctx.fillStyle = '#2E7D32';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx - eyeSpacing + 1, eyeY + 1, 5, 6, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + eyeSpacing + 1, eyeY + 1, 5, 6, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
|
||||
// Pupil shine
|
||||
ctx.fillStyle = '#FFFFFF';
|
||||
ctx.beginPath();
|
||||
ctx.arc(cx - eyeSpacing + 3, eyeY - 2, 2, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.beginPath();
|
||||
ctx.arc(cx + eyeSpacing + 3, eyeY - 2, 2, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
}
|
||||
}
|
||||
|
||||
drawMouth(ctx, cx, cy) {
|
||||
const mouthY = cy + 5;
|
||||
const openAmount = this.mouthOpen * 12;
|
||||
|
||||
if (openAmount > 1) {
|
||||
// Open mouth
|
||||
ctx.fillStyle = '#5D2906';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx, mouthY + openAmount / 2, 12, openAmount, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
|
||||
// Tongue
|
||||
ctx.fillStyle = '#E57373';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx, mouthY + openAmount, 6, 3, 0, 0, Math.PI);
|
||||
ctx.fill();
|
||||
} else {
|
||||
// Smile
|
||||
ctx.strokeStyle = '#5D2906';
|
||||
ctx.lineWidth = 3;
|
||||
ctx.lineCap = 'round';
|
||||
ctx.beginPath();
|
||||
ctx.arc(cx, mouthY - 5, 12, 0.2 * Math.PI, 0.8 * Math.PI);
|
||||
ctx.stroke();
|
||||
}
|
||||
}
|
||||
|
||||
drawCrown(ctx, cx, cy) {
|
||||
const crownY = cy - 65;
|
||||
const sway = Math.sin(this.time * 0.03) * 3;
|
||||
|
||||
// Main crown
|
||||
const greens = ['#2E7D32', '#4CAF50', '#66BB6A', '#388E3C'];
|
||||
|
||||
for (let i = 0; i < 8; i++) {
|
||||
const angle = (i / 8) * Math.PI * 2;
|
||||
const r = 35 + Math.sin(this.time * 0.02 + i) * 3;
|
||||
const lx = cx + Math.cos(angle) * r + sway;
|
||||
const ly = crownY + Math.sin(angle) * 25 - 5;
|
||||
|
||||
ctx.fillStyle = greens[i % greens.length];
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(lx, ly, 20, 15, angle * 0.3, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
}
|
||||
|
||||
// Center top
|
||||
ctx.fillStyle = '#4CAF50';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + sway, crownY - 25, 18, 14, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
|
||||
ctx.fillStyle = '#66BB6A';
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(cx + sway + 5, crownY - 30, 12, 10, 0.3, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
}
|
||||
|
||||
drawLeaves(ctx) {
|
||||
ctx.fillStyle = 'rgba(76, 175, 80, 0.4)';
|
||||
this.leaves.forEach(leaf => {
|
||||
ctx.save();
|
||||
ctx.translate(leaf.x, leaf.y);
|
||||
ctx.rotate(leaf.wobble + this.time * 0.02);
|
||||
ctx.beginPath();
|
||||
ctx.ellipse(0, 0, leaf.size, leaf.size * 0.5, 0, 0, Math.PI * 2);
|
||||
ctx.fill();
|
||||
ctx.restore();
|
||||
});
|
||||
}
|
||||
|
||||
drawStatusGlow(ctx, cx, cy) {
|
||||
if (this.state === 'idle') return;
|
||||
|
||||
const colors = {
|
||||
listening: 'rgba(244, 67, 54, 0.3)',
|
||||
thinking: 'rgba(255, 193, 7, 0.3)',
|
||||
talking: 'rgba(76, 175, 80, 0.3)',
|
||||
};
|
||||
|
||||
const color = colors[this.state] || 'transparent';
|
||||
const pulse = Math.sin(this.time * 0.1) * 10 + 20;
|
||||
|
||||
ctx.beginPath();
|
||||
ctx.arc(cx, cy - 20, 80 + pulse, 0, Math.PI * 2);
|
||||
ctx.fillStyle = color;
|
||||
ctx.fill();
|
||||
}
|
||||
}
|
||||
228
frontend/js/settings.js
Normal file
228
frontend/js/settings.js
Normal file
|
|
@ -0,0 +1,228 @@
|
|||
class SettingsManager {
|
||||
constructor(audioHandler) {
|
||||
this.audioHandler = audioHandler;
|
||||
this.micTestStream = null;
|
||||
this.micTestAnalyser = null;
|
||||
this.micTestAnimFrame = null;
|
||||
this.speakerTestAudio = null;
|
||||
|
||||
this.modal = document.getElementById('settings-modal');
|
||||
this.micSelect = document.getElementById('mic-select');
|
||||
this.speakerSelect = document.getElementById('speaker-select');
|
||||
this.micTestBtn = document.getElementById('mic-test-btn');
|
||||
this.micLevel = document.getElementById('mic-level');
|
||||
this.micTestStatus = document.getElementById('mic-test-status');
|
||||
this.speakerTestBtn = document.getElementById('speaker-test-btn');
|
||||
this.speakerTestStatus = document.getElementById('speaker-test-status');
|
||||
this.micGainSlider = document.getElementById('mic-gain');
|
||||
this.gainValueEl = document.getElementById('gain-value');
|
||||
|
||||
// Set initial slider value from saved preference
|
||||
this.micGainSlider.value = audioHandler.micGain;
|
||||
this.gainValueEl.textContent = audioHandler.micGain.toFixed(1);
|
||||
|
||||
document.getElementById('settings-btn').addEventListener('click', () => this.open());
|
||||
document.getElementById('settings-close').addEventListener('click', () => this.close());
|
||||
document.getElementById('settings-save').addEventListener('click', () => this.save());
|
||||
this.modal.addEventListener('click', (e) => {
|
||||
if (e.target === this.modal) this.close();
|
||||
});
|
||||
|
||||
this.micTestBtn.addEventListener('click', () => this.toggleMicTest());
|
||||
this.speakerTestBtn.addEventListener('click', () => this.testSpeaker());
|
||||
|
||||
this.micGainSlider.addEventListener('input', (e) => {
|
||||
const val = parseFloat(e.target.value);
|
||||
this.gainValueEl.textContent = val.toFixed(1);
|
||||
this.audioHandler.setGain(val);
|
||||
// Update test gain in real-time
|
||||
if (this._micTestGainNode) {
|
||||
this._micTestGainNode.gain.value = val;
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async open() {
|
||||
this.modal.classList.remove('hidden');
|
||||
await this.loadDevices();
|
||||
}
|
||||
|
||||
close() {
|
||||
this.stopMicTest();
|
||||
this.stopSpeakerTest();
|
||||
this.modal.classList.add('hidden');
|
||||
}
|
||||
|
||||
save() {
|
||||
const micId = this.micSelect.value;
|
||||
const speakerId = this.speakerSelect.value;
|
||||
|
||||
if (micId) this.audioHandler.setMic(micId);
|
||||
if (speakerId) this.audioHandler.setSpeaker(speakerId);
|
||||
|
||||
this.close();
|
||||
}
|
||||
|
||||
async loadDevices() {
|
||||
// Request permission first to get labeled devices
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
||||
stream.getTracks().forEach(t => t.stop());
|
||||
} catch (e) {
|
||||
this.micTestStatus.textContent = 'Eroare: Nu am acces la microfon. Verifica permisiunile browserului.';
|
||||
return;
|
||||
}
|
||||
|
||||
const devices = await navigator.mediaDevices.enumerateDevices();
|
||||
|
||||
// Microphones
|
||||
this.micSelect.innerHTML = '';
|
||||
const mics = devices.filter(d => d.kind === 'audioinput');
|
||||
mics.forEach(device => {
|
||||
const opt = document.createElement('option');
|
||||
opt.value = device.deviceId;
|
||||
opt.textContent = device.label || `Microfon ${this.micSelect.options.length + 1}`;
|
||||
if (device.deviceId === this.audioHandler.selectedMicId) {
|
||||
opt.selected = true;
|
||||
}
|
||||
this.micSelect.appendChild(opt);
|
||||
});
|
||||
|
||||
// Speakers
|
||||
this.speakerSelect.innerHTML = '';
|
||||
const speakers = devices.filter(d => d.kind === 'audiooutput');
|
||||
if (speakers.length === 0) {
|
||||
const opt = document.createElement('option');
|
||||
opt.textContent = 'Difuzor implicit (browserul nu permite selectia)';
|
||||
this.speakerSelect.appendChild(opt);
|
||||
this.speakerSelect.disabled = true;
|
||||
} else {
|
||||
this.speakerSelect.disabled = false;
|
||||
speakers.forEach(device => {
|
||||
const opt = document.createElement('option');
|
||||
opt.value = device.deviceId;
|
||||
opt.textContent = device.label || `Difuzor ${this.speakerSelect.options.length + 1}`;
|
||||
if (device.deviceId === this.audioHandler.selectedSpeakerId) {
|
||||
opt.selected = true;
|
||||
}
|
||||
this.speakerSelect.appendChild(opt);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
async toggleMicTest() {
|
||||
if (this.micTestStream) {
|
||||
this.stopMicTest();
|
||||
return;
|
||||
}
|
||||
|
||||
const deviceId = this.micSelect.value;
|
||||
try {
|
||||
this.micTestStream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: { deviceId: deviceId ? { exact: deviceId } : undefined },
|
||||
});
|
||||
|
||||
const ctx = new AudioContext();
|
||||
const source = ctx.createMediaStreamSource(this.micTestStream);
|
||||
const gainNode = ctx.createGain();
|
||||
gainNode.gain.value = this.audioHandler.micGain;
|
||||
this._micTestGainNode = gainNode;
|
||||
this.micTestAnalyser = ctx.createAnalyser();
|
||||
this.micTestAnalyser.fftSize = 256;
|
||||
source.connect(gainNode);
|
||||
gainNode.connect(this.micTestAnalyser);
|
||||
|
||||
this.micTestBtn.textContent = 'Opreste testul';
|
||||
this.micTestBtn.classList.add('active');
|
||||
this.micTestStatus.textContent = 'Vorbeste in microfon - ar trebui sa vezi bara miscandu-se.';
|
||||
|
||||
this._micTestCtx = ctx;
|
||||
this.updateMicLevel();
|
||||
} catch (e) {
|
||||
this.micTestStatus.textContent = `Eroare: ${e.message}`;
|
||||
}
|
||||
}
|
||||
|
||||
updateMicLevel() {
|
||||
if (!this.micTestAnalyser) return;
|
||||
|
||||
const data = new Uint8Array(this.micTestAnalyser.frequencyBinCount);
|
||||
this.micTestAnalyser.getByteFrequencyData(data);
|
||||
|
||||
let sum = 0;
|
||||
for (let i = 0; i < data.length; i++) sum += data[i];
|
||||
const avg = sum / data.length;
|
||||
const pct = Math.min(100, (avg / 128) * 100);
|
||||
|
||||
this.micLevel.style.width = pct + '%';
|
||||
|
||||
this.micTestAnimFrame = requestAnimationFrame(() => this.updateMicLevel());
|
||||
}
|
||||
|
||||
stopMicTest() {
|
||||
if (this.micTestStream) {
|
||||
this.micTestStream.getTracks().forEach(t => t.stop());
|
||||
this.micTestStream = null;
|
||||
}
|
||||
if (this.micTestAnimFrame) {
|
||||
cancelAnimationFrame(this.micTestAnimFrame);
|
||||
this.micTestAnimFrame = null;
|
||||
}
|
||||
if (this._micTestCtx) {
|
||||
this._micTestCtx.close();
|
||||
this._micTestCtx = null;
|
||||
}
|
||||
this.micTestAnalyser = null;
|
||||
this.micLevel.style.width = '0%';
|
||||
this.micTestBtn.textContent = 'Testeaza microfonul';
|
||||
this.micTestBtn.classList.remove('active');
|
||||
this.micTestStatus.textContent = '';
|
||||
}
|
||||
|
||||
async testSpeaker() {
|
||||
this.stopSpeakerTest();
|
||||
|
||||
// Generate a short test tone
|
||||
const ctx = new AudioContext();
|
||||
const oscillator = ctx.createOscillator();
|
||||
const gainNode = ctx.createGain();
|
||||
|
||||
oscillator.type = 'sine';
|
||||
oscillator.frequency.setValueAtTime(440, ctx.currentTime);
|
||||
gainNode.gain.setValueAtTime(0.3, ctx.currentTime);
|
||||
gainNode.gain.exponentialRampToValueAtTime(0.01, ctx.currentTime + 1);
|
||||
|
||||
oscillator.connect(gainNode);
|
||||
gainNode.connect(ctx.destination);
|
||||
|
||||
// Try to set output device
|
||||
const speakerId = this.speakerSelect.value;
|
||||
if (speakerId && ctx.setSinkId) {
|
||||
try {
|
||||
await ctx.setSinkId(speakerId);
|
||||
} catch (e) {
|
||||
// setSinkId not supported in all browsers
|
||||
}
|
||||
}
|
||||
|
||||
oscillator.start();
|
||||
oscillator.stop(ctx.currentTime + 1);
|
||||
|
||||
this.speakerTestStatus.textContent = 'Se reda un sunet de test (440 Hz)...';
|
||||
this._speakerTestCtx = ctx;
|
||||
|
||||
setTimeout(() => {
|
||||
this.speakerTestStatus.textContent = 'Ai auzit sunetul? Daca nu, selecteaza alt difuzor.';
|
||||
ctx.close();
|
||||
this._speakerTestCtx = null;
|
||||
}, 1200);
|
||||
}
|
||||
|
||||
stopSpeakerTest() {
|
||||
if (this._speakerTestCtx) {
|
||||
this._speakerTestCtx.close();
|
||||
this._speakerTestCtx = null;
|
||||
}
|
||||
this.speakerTestStatus.textContent = '';
|
||||
}
|
||||
}
|
||||
18
start.bat
Normal file
18
start.bat
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
@echo off
|
||||
chcp 65001 >nul
|
||||
set PYTHONIOENCODING=utf-8
|
||||
echo ================================
|
||||
echo EcoBot - Scoala Verde
|
||||
echo ================================
|
||||
echo.
|
||||
|
||||
cd /d "%~dp0"
|
||||
|
||||
echo Activating virtual environment...
|
||||
call venv\Scripts\activate
|
||||
|
||||
echo Starting server...
|
||||
cd backend
|
||||
python main.py
|
||||
|
||||
pause
|
||||
Loading…
Add table
Reference in a new issue