Forgive me, I’m no AI expert to fully compare the needed tokens per second measurement to relate to the average query Siri might handle, but I will say this:

Even in your article, only the largest model ran at 8/tps, others ran much faster, and none of these were optimized for a task, just benchmarking.

Would it be impossible for Apple to be running an optimized model specific to expected mobile tasks, and leverage their own hardware more efficiently than we can, to meet their needs?

I imagine they cut out most worldly knowledge etc/use a lightweight model, which is why there is still a need to link to ChatGPT or Apple for some requests, would this let them trim Siri down to perform well enough on phones for most requests? They also advertised launching AI on M1-2 chip devices, which are not M3-Max either…

Onboard AI chips will allow this to be local.

Phones do not have the power to ~~~

Perhaps this is why these features will only be available on iPhone 15 Pro/Max and newer? Gotta have those latest and greatest chips.

It will be fun to see how it all shakes out. If the AI can’t run most queries on the phone with all this advertising of local processing…there’ll be one hell of a lawsuit coming up.

EDIT: Finished looking for what I thought I remembered…

Additionally, Siri has been locally processed since iOS 15.

https://www.macrumors.com/how-to/use-on-device-siri-iphone-ipad/

I think there’s a larger picture at play here that is being missed.

Getting the weather is a standard feature for years now. Nothing AI about it.

What is “AI” is, Hey Siri, what is the weather at my daughter’s recital coming up?

The AI processing, calculated on-device if what they claim is true, is:

1. the determination of who your daughter is
2. What is a recital? An event? Are there any upcoming calendar events that match this concept?
3. Is the “daughter” associated with this event by description or invitation? Yes? OK, what’s the address?
4. Submit zip code of recital calendar event involving the kid to the weather API, and churn out a reply that includes all this information…

Well {Your phone contact name}, it looks like it will {remote weather response} during your {calendar event from phone} with {daughter from contacts} on {event date}.

That is the idea between on-device and cloud processing. The phone already has your contacts and calendar and does that work offline rather than educating an online server about your family, events and location, and requests the bare minimum from the internet, in this case nothing more than if you opened the weather app yourself and put in a zip code.

Between Lemmy and Reddit, I’m a long time lurker, rarely a poster, especially as toxic as Reddit was I just found what I was looking for and moved along. Don’t mind me, I’m just:

PassingThrough.

I’ll take a compromise where “3.1” is etched in each head end, and I can trust that “3.1” means something, and start with that.

The real crux of the issue is that there is no way to identify the ability of a port or cable without trying it, and even if labeled there is/was too much freedom to simply deviate and escape spec.

I grabbed a cable from my box to use with my docking station. Short length, hefty girth, firm head ends, certainly felt like a featured video/data/Dock cable…it did not work. I did work with my numpad/USB-A port bus thing though, so it had some data ability(did not test if it was 2.0 or 3.0). The cable that DID work with my docking station was actually a much thinner, weaker feeling one from a portable monitor I also had. So you can’t even judge by wiring density.

And now we have companies using the port to deviate from spec completely, like the Raspberry Pi 5 technically using USB-C, but at a power level unsupported by spec. Or my video glasses that use USB-C connections all over, with a proprietary design that ensures only their products work together.

Universal appearance, non-universal function, universal confusion.

I hate it. At least with HDMI, RCA, 3.5mm, Micro-USB…I could readily identify what a port and plug was good for, and 99/100 the unknown origin random wires I had in a box worked just fine.

Actually, that leads me to another point:

One upon a time, the concept behind a universal USB-C connector was so we could do exactly that.

Laptop? Phone? Camera? America? Germany? Japan? Power? Connect the to TV? Internet?

Wouldn’t matter anymore. USB-C to cover it all. Voltage high for the laptop, low for the camera, all available just the same in every country, universal. So yes, fill the airports and hotels with them. Use them for power and to play videos on the TV. Because we weren’t supposed to have to question the voltage or abilities of the ports and cables in use.

Did/will that future materialize?

I feel the only place for a €1 cable is met by those USB-A to C cables that you get with things for 5V charging. That’s it. And it’s very obvious what the limits on those are by the A plug end.

Anything that wants to be USB-C on both ends should be fully compatible with a marked spec, not indistinguishable from a 5V junk wire or freely cherry picking what they feel like paying for.

Simply marking on the cable itself what generation it’s good for would be a nice start, but the real issue is the cherry picking. The generation numbers don’t cover a wire that does maximum everything except video. Or a proprietary setup that does power transfer in excess of spec(Dell, Raspberry Pi 5). But they all have the same ends and lack of demarcation, leading to the confusion.

The worst part is, I could accept that as a generational flaw. The newer ones get better, the olds ones lying around do less. OK, that’s the beast of progress.

But no. They still make cables today that do power only. They still do cables that do everything except video. Why? Save a couple cents. Make dollars off multiple product lines, etc. Money.

What could have been the cable to end all cables…just continued to make USB a laughing stock of confusion.

Don’t even get me started on the device side implementations…