That's an interesting interview, thanks for sharing.
Though I draw the line with using these tools at helping me out with the drudgery of daily work. I don't want them to impersonate me, or write emails on my behalf. I cringe whenever Gmail suggests the next phrase it thinks I want to write. It's akin to someone trying to end your sentences for you. Stop putting words in my mouth!
The recent Microsoft 365 Copilot presentation, where the host had it ghost write a speech for their kid's graduation party[1]—complete with cues about where to look(!)—is unbelievably cringey. Do these people really think AI should be assisting with such personal matters? Do they really find doing these things themselves a chore?
> And I have no idea why things are like this.
Oh, I think it's pretty clear. The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people, not to mention the disk space it requires. Once the resource requirements are lowered, and our personal devices are _much_ more powerful, then self-hosting would be feasible.
Another, perhaps larger, reason, is that AI tools are still a business advantage for companies, so it's no wonder that they want to keep them to themselves. I think this will change and open source LLMs will be widespread in a few years, but proprietary services will still be more popular.
And lastly, most people just don't want/like/know how to self-host _anything_. There's a technical barrier to entry, for sure, but even if that is lowered, most people are entirely willing to give up their personal data for the convenience of using a proprietary service. You can see this today with web, mail, file servers, etc.; self-hosting is still done by a very niche group of privacy-minded tech-literate people.
Anyway, thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
> The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people
FWIW LLaMA 65B can run on a single MacBook Pro now. Things move crazy fast. (Or did, before Facebook started DMCA'ing everyone.)
I did a bad job of explaining that personal GPUs will be sufficient in the near future. Thanks for pointing that out.
> thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
Thanks for talking about the issue at all. The whole reason I got into AI was to run these myself. It'll be a shame if only massive corporations can run models anyone cares about.
Though I draw the line with using these tools at helping me out with the drudgery of daily work. I don't want them to impersonate me, or write emails on my behalf. I cringe whenever Gmail suggests the next phrase it thinks I want to write. It's akin to someone trying to end your sentences for you. Stop putting words in my mouth!
The recent Microsoft 365 Copilot presentation, where the host had it ghost write a speech for their kid's graduation party[1]—complete with cues about where to look(!)—is unbelievably cringey. Do these people really think AI should be assisting with such personal matters? Do they really find doing these things themselves a chore?
> And I have no idea why things are like this.
Oh, I think it's pretty clear. The amount of resources required to run this on personal machines is still prohibitively high. I saw in one of your posts you mentioned you use 8xA100s. That's a crazy amount of compute unreachable by most people, not to mention the disk space it requires. Once the resource requirements are lowered, and our personal devices are _much_ more powerful, then self-hosting would be feasible.
Another, perhaps larger, reason, is that AI tools are still a business advantage for companies, so it's no wonder that they want to keep them to themselves. I think this will change and open source LLMs will be widespread in a few years, but proprietary services will still be more popular.
And lastly, most people just don't want/like/know how to self-host _anything_. There's a technical barrier to entry, for sure, but even if that is lowered, most people are entirely willing to give up their personal data for the convenience of using a proprietary service. You can see this today with web, mail, file servers, etc.; self-hosting is still done by a very niche group of privacy-minded tech-literate people.
Anyway, thanks for leading the way, and spreading the word about why self-hosting these tools is important. I hope that our vision becomes a reality for many soon.
[1]: https://www.youtube.com/watch?v=ebls5x-gb0s