I think its better if their github mention the minimum token count requirement to selfhost this. I don’t think it will ever reach something usable for normal selfhost user.

Based on your statement i think most of your experience come from corporate AI usage… Which deploy multiple agent system in their AI and hosted in large data center.

Hi, i just started mine recently as well, currently i only host stablediffusion. Will explore more what i can do in near future