Forumi
Home Pravila i pomoć Članovi Kalendar Današnji postovi


Povratak   PC Ekspert Forum > Računala > DIY i Tinkering
Osvježi stranicu Pričanje s lokalnim AI
Ime
Lozinka

Odgovori
 
Uređivanje
Staro Danas, 15:21   #1
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Pričanje s lokalnim AI
Znači cilj je nekako upogoniti lokalni LLM + Rhasspy + par mikrofona po stanu da se može kompletno pričati s lokalnim AI-jem (full offline) i istovremeno davati komande tipa "ugasi svjetlo".

No zapeo već na prvom koraku - u dnevnom je HTPC s Windowsima, na njemu je mikrofon. U drugoj sobi je komp na kojem se vrti LLM.

Treba nekako napraviti Speech to Text (STT) na način da komp u dnenvom sluša, snimi wav, pošalje kompu u drugoj sobi (OpenAI Whisper) koji onda obradi zvuk i dalje šalje Rhasspyu ili LLM-u, koji nakon responsa šalje Coqui TTS da napravi TTS i vrati wav nazad HTPC-u da ga izgovori.

Stvar je očito mega komplicirana i sad gledam koje su druge opcije.

Da li je netko slagao takav sistem u kojem bi znači bilo sve objedinjeno, STT, TTS, Home Assistant i naravno full LLM za pričat o glupostima?

Chatgpt je probao složiti neku Python skriptu, ali ništa to ne pomaže.
dakky je offline   Reply With Quote
Staro Danas, 16:29   #2
Ivo_Strojnica
PRO
Moj komp
 
Ivo_Strojnica's Avatar
 
Datum registracije: Apr 2010
Lokacija: Zagreb
Postovi: 4,624
u borati jesi ga zakomplicira.
šta fali nekom bluetooth zvučniku koji je spojen na tvoj server?
A tvoj mudrac se pali kad čuje neke ključne riči, tipa: "Hej, mudrače!".
__________________
"Who is your daddy and what does he do?"
Ivo_Strojnica je offline   Reply With Quote
Oglasni prostor
Oglas
 
Oglas
Staro Danas, 16:32   #3
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Ništa ne fali, osim što nemam BT zvučnik, ali bi ga kupio kad bi znao da ću eliminirati sve ove zavrzlame.

Zvučnik samo ispušta zvuk, problem je dovesti vlastiti glas do servera... i transcribati ga u tekst...
dakky je offline   Reply With Quote
Staro Danas, 16:38   #4
mkey
Premium
Moj komp
 
Datum registracije: Sep 2018
Lokacija: tu
Postovi: 3,114
Meni se čini kako bi se taj krajnji dio možda najlakše dalo riješiti korištenjem nekog starijeg mobitela, pa da on služio kao audio ulaz i izlaz. Nisam baš momentalno pametan kako to odraditi, ali koristio sam droidcam aplikaciju da bih mogao koristiti kameru na mobitelu kao mobilnu kameru, tako da bi taj dio trebao (po meni) biti relativno jednostavan. Redovito ima nekih starih mobitela pri ruci pa bi se reklo da je to minimalan trošak.

BTW, droidcam app od mobitela napravi web kameru, tako da bi LLM komp to trebao uredno vidjeti na mreži. YMMV. Nisam baš garant hoće li LLM komp htjeti vidjeti mob kao zvučnik, tako da bi to mogla biti dodatna začkoljica.

A sve ovo ostalo, pojma nemam bogati. Jamačno dodatni problem bude ako želite spikati po naški -> OK, ovo pp planiraš riješiti sa Rhasspy.
__________________
Citiraj:
Autor George Carlin
But there’s a reason. There’s a reason. There’s a reason for this, there’s a reason education sucks, and it’s the same reason that it will never, ever, ever be fixed. It’s never gonna get any better. Don’t look for it. Be happy with what you got. Because the owners of this country don't want that. I'm talking about the real owners now, the real owners, the big wealthy business interests that control things and make all the important decisions. Forget the politicians. The politicians are put there to give you the idea that you have freedom of choice. You don't. You have no choice. You have owners. They own you. They own everything. They own all the important land. They own and control the corporations. They’ve long since bought and paid for the senate, the congress, the state houses, the city halls, they got the judges in their back pockets and they own all the big media companies so they control just about all of the news and information you get to hear. They got you by the balls. They spend billions of dollars every year lobbying, lobbying, to get what they want. Well, we know what they want. They want more for themselves and less for everybody else, but I'll tell you what they don’t want: They don’t want a population of citizens capable of critical thinking. They don’t want well informed, well educated people capable of critical thinking. They’re not interested in that. That doesn’t help them. Thats against their interests. Thats right. They don’t want people who are smart enough to sit around a kitchen table to figure out how badly they’re getting f*cked by a system that threw them overboard 30 f*cking years ago. They don’t want that. You know what they want? They want obedient workers. Obedient workers. People who are just smart enough to run the machines and do the paperwork, and just dumb enough to passively accept all these increasingly shittier jobs with the lower pay, the longer hours, the reduced benefits, the end of overtime and the vanishing pension that disappears the minute you go to collect it, and now they’re coming for your Social Security money. They want your retirement money. They want it back so they can give it to their criminal friends on Wall Street, and you know something? They’ll get it. They’ll get it all from you, sooner or later, 'cause they own this f*cking place. It's a big club, and you ain’t in it. You and I are not in the big club. And by the way, it's the same big club they use to beat you over the head with all day long when they tell you what to believe. All day long beating you over the head in their media telling you what to believe, what to think and what to buy. The table is tilted folks. The game is rigged, and nobody seems to notice, nobody seems to care. Good honest hard-working people -- white collar, blue collar, it doesn’t matter what color shirt you have on -- good honest hard-working people continue -- these are people of modest means -- continue to elect these rich c*cksuckers who don’t give a f*ck about them. They don’t give a f*ck about you. They don’t give a f*ck about you. They don't care about you at all -- at all -- at all. And nobody seems to notice, nobody seems to care. That's what the owners count on; the fact that Americans will probably remain willfully ignorant of the big red, white and blue dick that's being jammed up their assholes everyday. Because the owners of this country know the truth: it's called the American Dream, because you have to be asleep to believe it.
mkey je offline   Reply With Quote
Staro Danas, 16:42   #5
Ivo_Strojnica
PRO
Moj komp
 
Ivo_Strojnica's Avatar
 
Datum registracije: Apr 2010
Lokacija: Zagreb
Postovi: 4,624
Citiraj:
Autor dakky Pregled postova
Ništa ne fali, osim što nemam BT zvučnik, ali bi ga kupio kad bi znao da ću eliminirati sve ove zavrzlame.

Zvučnik samo ispušta zvuk, problem je dovesti vlastiti glas do servera... i transcribati ga u tekst...
Ima onih cheap zvučnika sa mikrofonom, ali sad se postavlja pitanje kakve je kvalitete.

možda kupiti neki bluetooth mikrofon?
ili da prvo samo spojiš slušalice sa mikrofonom i složiš ulaz da to sve radi.

Kad on tebe bude čuo i razumio, onda možeš kreniti sa zamjenom tipa ulaza.
__________________
"Who is your daddy and what does he do?"
Ivo_Strojnica je offline   Reply With Quote
Staro Danas, 16:45   #6
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Ima dobrih TTS modela koji pričaju naški, ima čak i STT modela koji transcribaju naški... to je sve drugi korak kad riješim engleski

Znači chatgpt je složio Python skriptu koja:
1. sluša na lokalnom kompu s mikrofonom
2. recorda 5 sekundi kad čuje govor
3. šalje serveru na transcribe
4. dobiva tekst natrag
5. šalje dobiveni tekst LLM-u na server
6. LLM šalje odgovor natrag
7. python skripta šalje dobiveni tekst Coqui TTS na server
8. CoquiTTS šalje wav natrag na playback
9. skripta pušta snimljeni wav file

I što je najgore ovo je upravo proradilo nakon što se cijelo jutro zezam s ovim.

I komplicirano je do ibera, da. Mora biti neki bolji način.
dakky je offline   Reply With Quote
Staro Danas, 16:47   #7
mkey
Premium
Moj komp
 
Datum registracije: Sep 2018
Lokacija: tu
Postovi: 3,114
Što je kompliciranije to će otpor prema održavanju biti veći, to budi siguran.
__________________
Citiraj:
Autor George Carlin
But there’s a reason. There’s a reason. There’s a reason for this, there’s a reason education sucks, and it’s the same reason that it will never, ever, ever be fixed. It’s never gonna get any better. Don’t look for it. Be happy with what you got. Because the owners of this country don't want that. I'm talking about the real owners now, the real owners, the big wealthy business interests that control things and make all the important decisions. Forget the politicians. The politicians are put there to give you the idea that you have freedom of choice. You don't. You have no choice. You have owners. They own you. They own everything. They own all the important land. They own and control the corporations. They’ve long since bought and paid for the senate, the congress, the state houses, the city halls, they got the judges in their back pockets and they own all the big media companies so they control just about all of the news and information you get to hear. They got you by the balls. They spend billions of dollars every year lobbying, lobbying, to get what they want. Well, we know what they want. They want more for themselves and less for everybody else, but I'll tell you what they don’t want: They don’t want a population of citizens capable of critical thinking. They don’t want well informed, well educated people capable of critical thinking. They’re not interested in that. That doesn’t help them. Thats against their interests. Thats right. They don’t want people who are smart enough to sit around a kitchen table to figure out how badly they’re getting f*cked by a system that threw them overboard 30 f*cking years ago. They don’t want that. You know what they want? They want obedient workers. Obedient workers. People who are just smart enough to run the machines and do the paperwork, and just dumb enough to passively accept all these increasingly shittier jobs with the lower pay, the longer hours, the reduced benefits, the end of overtime and the vanishing pension that disappears the minute you go to collect it, and now they’re coming for your Social Security money. They want your retirement money. They want it back so they can give it to their criminal friends on Wall Street, and you know something? They’ll get it. They’ll get it all from you, sooner or later, 'cause they own this f*cking place. It's a big club, and you ain’t in it. You and I are not in the big club. And by the way, it's the same big club they use to beat you over the head with all day long when they tell you what to believe. All day long beating you over the head in their media telling you what to believe, what to think and what to buy. The table is tilted folks. The game is rigged, and nobody seems to notice, nobody seems to care. Good honest hard-working people -- white collar, blue collar, it doesn’t matter what color shirt you have on -- good honest hard-working people continue -- these are people of modest means -- continue to elect these rich c*cksuckers who don’t give a f*ck about them. They don’t give a f*ck about you. They don’t give a f*ck about you. They don't care about you at all -- at all -- at all. And nobody seems to notice, nobody seems to care. That's what the owners count on; the fact that Americans will probably remain willfully ignorant of the big red, white and blue dick that's being jammed up their assholes everyday. Because the owners of this country know the truth: it's called the American Dream, because you have to be asleep to believe it.
mkey je offline   Reply With Quote
Staro Danas, 17:00   #8
Ivo_Strojnica
PRO
Moj komp
 
Ivo_Strojnica's Avatar
 
Datum registracije: Apr 2010
Lokacija: Zagreb
Postovi: 4,624
auf, amen.
A i samo sjećanje će ispariti kako to sve radi....
__________________
"Who is your daddy and what does he do?"
Ivo_Strojnica je offline   Reply With Quote
Staro Danas, 17:05   #9
mkey
Premium
Moj komp
 
Datum registracije: Sep 2018
Lokacija: tu
Postovi: 3,114
Ma može se stvari solidno skriptirati pa to pomogne dosta kod raznih (naknadnih) instalacija. Ali ovo je jako puno koraka i sve što se bude moralo ručno podešavati bude kasnije vjerojatno tražilo n+1 koraka ručnih podešavanja. Vjerojatno bi bilo pametno što više toga utrpati u neki distrobox ili docker tako da se podesi i ne dira nikad

A meni se dobitak od toga na kraju čini minimalan. I tako bude elonov brain interface uskoro, pa neće trebati ni pričati, samo pomisliš i gasi svjetlo.
__________________
Citiraj:
Autor George Carlin
But there’s a reason. There’s a reason. There’s a reason for this, there’s a reason education sucks, and it’s the same reason that it will never, ever, ever be fixed. It’s never gonna get any better. Don’t look for it. Be happy with what you got. Because the owners of this country don't want that. I'm talking about the real owners now, the real owners, the big wealthy business interests that control things and make all the important decisions. Forget the politicians. The politicians are put there to give you the idea that you have freedom of choice. You don't. You have no choice. You have owners. They own you. They own everything. They own all the important land. They own and control the corporations. They’ve long since bought and paid for the senate, the congress, the state houses, the city halls, they got the judges in their back pockets and they own all the big media companies so they control just about all of the news and information you get to hear. They got you by the balls. They spend billions of dollars every year lobbying, lobbying, to get what they want. Well, we know what they want. They want more for themselves and less for everybody else, but I'll tell you what they don’t want: They don’t want a population of citizens capable of critical thinking. They don’t want well informed, well educated people capable of critical thinking. They’re not interested in that. That doesn’t help them. Thats against their interests. Thats right. They don’t want people who are smart enough to sit around a kitchen table to figure out how badly they’re getting f*cked by a system that threw them overboard 30 f*cking years ago. They don’t want that. You know what they want? They want obedient workers. Obedient workers. People who are just smart enough to run the machines and do the paperwork, and just dumb enough to passively accept all these increasingly shittier jobs with the lower pay, the longer hours, the reduced benefits, the end of overtime and the vanishing pension that disappears the minute you go to collect it, and now they’re coming for your Social Security money. They want your retirement money. They want it back so they can give it to their criminal friends on Wall Street, and you know something? They’ll get it. They’ll get it all from you, sooner or later, 'cause they own this f*cking place. It's a big club, and you ain’t in it. You and I are not in the big club. And by the way, it's the same big club they use to beat you over the head with all day long when they tell you what to believe. All day long beating you over the head in their media telling you what to believe, what to think and what to buy. The table is tilted folks. The game is rigged, and nobody seems to notice, nobody seems to care. Good honest hard-working people -- white collar, blue collar, it doesn’t matter what color shirt you have on -- good honest hard-working people continue -- these are people of modest means -- continue to elect these rich c*cksuckers who don’t give a f*ck about them. They don’t give a f*ck about you. They don’t give a f*ck about you. They don't care about you at all -- at all -- at all. And nobody seems to notice, nobody seems to care. That's what the owners count on; the fact that Americans will probably remain willfully ignorant of the big red, white and blue dick that's being jammed up their assholes everyday. Because the owners of this country know the truth: it's called the American Dream, because you have to be asleep to believe it.
mkey je offline   Reply With Quote
Staro Danas, 17:06   #10
Ivo_Strojnica
PRO
Moj komp
 
Ivo_Strojnica's Avatar
 
Datum registracije: Apr 2010
Lokacija: Zagreb
Postovi: 4,624
Prvo nek proradi, onda lako u Docker
__________________
"Who is your daddy and what does he do?"
Ivo_Strojnica je offline   Reply With Quote
Oglasni prostor
Oglas
 
Oglas
Staro Danas, 17:22   #11
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Ma da, izgleda da je ovo uzalud traćenje vremena.

Mikrofon lovi i što treba i što ne treba

Onda treba imati timeout, jer ne može skripta raditi konstantan send i dobivati konstantan transcript...

Znači skripta mora slušati 5-10 sekundi, poslati snimljeno, vidjeti što je snimljeno, slati opet.

Problem je što taj timeout od 5 sekundi gotovo uvijek prekine tvoje pitanje na pola i onda transkripcija bude na pola i onda LLM dobije polovični upit i evo sad mi priča o kuhanju nekog jela za koje ga nisam pitao.

Ideja je dobra da možeš "pričati sa kućom" o svemu i svačemu, ali je izgleda realizacija nemoguća...
dakky je offline   Reply With Quote
Staro Danas, 17:37   #12
mkey
Premium
Moj komp
 
Datum registracije: Sep 2018
Lokacija: tu
Postovi: 3,114
Vjerujem da biš trebao moći napraviti tako da čeka neku ključnu riječ. Dakle, Rhasspy snima stalno i kada dobije "start" krene obrađivati i koristi LLM. Da/ne?
__________________
Citiraj:
Autor George Carlin
But there’s a reason. There’s a reason. There’s a reason for this, there’s a reason education sucks, and it’s the same reason that it will never, ever, ever be fixed. It’s never gonna get any better. Don’t look for it. Be happy with what you got. Because the owners of this country don't want that. I'm talking about the real owners now, the real owners, the big wealthy business interests that control things and make all the important decisions. Forget the politicians. The politicians are put there to give you the idea that you have freedom of choice. You don't. You have no choice. You have owners. They own you. They own everything. They own all the important land. They own and control the corporations. They’ve long since bought and paid for the senate, the congress, the state houses, the city halls, they got the judges in their back pockets and they own all the big media companies so they control just about all of the news and information you get to hear. They got you by the balls. They spend billions of dollars every year lobbying, lobbying, to get what they want. Well, we know what they want. They want more for themselves and less for everybody else, but I'll tell you what they don’t want: They don’t want a population of citizens capable of critical thinking. They don’t want well informed, well educated people capable of critical thinking. They’re not interested in that. That doesn’t help them. Thats against their interests. Thats right. They don’t want people who are smart enough to sit around a kitchen table to figure out how badly they’re getting f*cked by a system that threw them overboard 30 f*cking years ago. They don’t want that. You know what they want? They want obedient workers. Obedient workers. People who are just smart enough to run the machines and do the paperwork, and just dumb enough to passively accept all these increasingly shittier jobs with the lower pay, the longer hours, the reduced benefits, the end of overtime and the vanishing pension that disappears the minute you go to collect it, and now they’re coming for your Social Security money. They want your retirement money. They want it back so they can give it to their criminal friends on Wall Street, and you know something? They’ll get it. They’ll get it all from you, sooner or later, 'cause they own this f*cking place. It's a big club, and you ain’t in it. You and I are not in the big club. And by the way, it's the same big club they use to beat you over the head with all day long when they tell you what to believe. All day long beating you over the head in their media telling you what to believe, what to think and what to buy. The table is tilted folks. The game is rigged, and nobody seems to notice, nobody seems to care. Good honest hard-working people -- white collar, blue collar, it doesn’t matter what color shirt you have on -- good honest hard-working people continue -- these are people of modest means -- continue to elect these rich c*cksuckers who don’t give a f*ck about them. They don’t give a f*ck about you. They don’t give a f*ck about you. They don't care about you at all -- at all -- at all. And nobody seems to notice, nobody seems to care. That's what the owners count on; the fact that Americans will probably remain willfully ignorant of the big red, white and blue dick that's being jammed up their assholes everyday. Because the owners of this country know the truth: it's called the American Dream, because you have to be asleep to believe it.
mkey je offline   Reply With Quote
Staro Danas, 17:41   #13
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Tako bi trebalo biti, ali ne znam kako to riješiti s Pythonom, lokalno.

Znači chatgpt je složio verziju skripte gdje čeka da stisnem enter, snima 5 sekundi, šalje na transcribe ... i ta verzija radi odlično... stisneš enter, pitaš "how are you" i dobiješ glasovni odgovor "i'm fine thanks for asking"

ali to staviti u loop ne ide nikako jer ako je tišina, u recording uđe šum i transcribe napravi neko nepostojeće pitanje pa dobiješ i nepostojeći odgovor.. probao dizati threshold ali onda uopće ne reagira ili prekasno reagira ili kad krene slušat već sam gotov s pitanjem ili uhvati samo zadnju riječ itd itd.

wake word bi bio idealan, pitanje je kako to napraviti...
dakky je offline   Reply With Quote
Staro Danas, 17:53   #14
spiderhr
Premium
 
spiderhr's Avatar
 
Datum registracije: Jul 2021
Lokacija: Sesvete
Postovi: 906
Kaj nema FireStick onu foru: Alexa play?

Ili Google?

Ti smisli Štef slušaj: i tu kažeš kaj te zanima.
Ili ako voliš ženska imena reci: Đurđa donesi pivu: i tu kažeš kaj te zanima

__________________
tomek@vz: ajd nemoj | Mali Čile SAD Češka Peru | Windows Free
spiderhr je offline   Reply With Quote
Staro Danas, 18:17   #15
OuttaControl
Premium
Moj komp
 
OuttaControl's Avatar
 
Datum registracije: Feb 2007
Lokacija: Dalmacija
Postovi: 5,744
Tebi tu treba neki lokalni algoritam koji analizira soundeave pattern i kad je dovoljno slican snimljenom "Stef Slusaj" aktivira snimanje i ide dalje, ima vjerojatno neki python library za to, pitaj LLM

Po meni ne treba niti pretvarati to u tekst nego samo pattern da bude dovoljno slican.
OuttaControl je offline   Reply With Quote
Staro Danas, 18:19   #16
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
E, taj wake word... Tko njega procesuira. Nešto treba procesuirati sav zvuk koji se konstantno sluša... dakle neki Silero VAD ili nešto treba stalno slušati i čekati glas... kad čuje glas, treba proslijediti glas na prepoznavanje...

Zvuči jednostavno ovako na papiru "Hej Štef", ali u realnosti je taj "Štef" mora isto biti transkripcija iz voice u tekst... i to konstantna... bez pauza... stalno slušanje i stalna transkripcija i stalno čekanje te jedne riječi da se upali druga faza koja sluša i čeka pravo pitanje.

Citiraj:
Autor OuttaControl Pregled postova
Tebi tu treba neki lokalni algoritam koji analizira soundeave pattern i kad je dovoljno slican snimljenom "Stef Slusaj" aktivira snimanje i ide dalje, ima vjerojatno neki python library za to, pitaj LLM

Po meni ne treba niti pretvarati to u tekst nego samo pattern da bude dovoljno slican.
E da, to... to bi trebalo biti lokalno, bez slanja na transkripciju...
dakky je offline   Reply With Quote
Staro Danas, 18:20   #17
tomek@vz
Premium
Moj komp
 
tomek@vz's Avatar
 
Datum registracije: May 2006
Lokacija: München/Varaždin
Postovi: 4,435
Vrlo komplicirano...koji je krajnji cilj?
__________________
Lenovo LOQ 15AHP9: AMD Ryzen 5 8645HS / 16GB DDR5 / Micron M.2 2230 1TB / Sandisk Extreme Pro 1TB / Radeon 760M + Geforce RTX 4050
Acer Aspire V3-574G: Intel i3 5005U / 8GB DDR3 / Seagate 1TB HDD / Geforce GT 940M
tomek@vz je offline   Reply With Quote
Staro Danas, 18:24   #18
dakky
Uptime freak
Moj komp
 
dakky's Avatar
 
Datum registracije: Dec 2002
Lokacija: Ri, Cro
Postovi: 2,932
Citiraj:
Autor tomek@vz Pregled postova
Vrlo komplicirano...koji je krajnji cilj?
Pa imat pametnu kuću... s kojom možeš pričat o svemu... i usput gasit i palit svjetla po kući, puštat muziku... pitat kako skuhat paštašutu i što misli o Plenkoviću .. i za koga glasat na lokalnim izborima... i reć da te podsjeti baciti smeće i da ćeš biti doma oko 18h i da upali klimu pol sata ranije.

To je krajnji cilj. Ali izgleda da je to SF još uvijek.
dakky je offline   Reply With Quote
Oglasni prostor
Oglas
 
Oglas
Odgovori

« Prethodna tema | Sljedeća tema »


Pravila postanja
Vi ne možete otvarati nove teme
Vi ne možete pisati odgovore
Vi ne možete uploadati priloge
Vi ne možete uređivati svoje poruke
BB code je Uključeno
Smajlići su Uključeno
[IMG] kod je Uključeno
HTML je Uključeno
Idi na



© 1999-2024 PC Ekspert - Sva prava pridržana ISSN 1334-2940
Ad Management by RedTyger
Powered by vBulletin®
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.