Yes, others should be able to do this too, and the change is larger memory chips.
The next big step is Medusa Halo, which will have a 384-bit LPDDR6 interface. Those should be able to support 256GB at release, with 512GB coming later with larger chips (But I don't know to what extent people should trust the memory vendor roadmaps.) I'm not sure if they will be out in a year. Probably will be in a year and half.
Do not base products on models that are not open-weights. Doing it is like building a product on someone else's platform, you are entirely at their mercy, and even when they don't have any reason to hurt you, you are tiny enough that if any policy they want to enact hurts you as a side effect, no-one is going to care.
You don't have to self-host the open-weights model, you just need to be able to source it from multiple providers.
Using the closed vendor models maybe made sense when open-weight models lagged so far behind, but that time is now gone.
I just have no choice. My application was tested and benchmarked against a lot of models, but no model can be in the same league as gemini in term of multi-language support and understanding.
Taalas only uses SRAM for the KV cache and the activations, the weights are in mask rom in the metal layers.
If they designed this right, it means that once they have a model, so long as they keep the hyperparameters fixed they can change the weights much faster than it takes to spin up a completely new chip, essentially at a cost of doing a minor revision.
Is the mask ROM really going to be worth it over Carmack's high-bandwidth-flash concept? I mean, sure, I could be convinced I guess, but it's not obvious.
Not because rom is better than flash, but because the critical part is distributing the memory with the compute. Mask ROM is just the densest way of embedding memory with logic. Instead of having a large pool of memory connected to the separate execution units with a bus, each execution unit locally has the rom that it uses. Data movement is >90% of energy use in modern ai accelerators, removing it as far as practical is how they get performance and silicon and energy efficiency.
It directly impacts performance when playing twitch fps games, any input lag causes measurable difference in score. This is before people even notice it happening.
I would love to see some research confirming that 1ms of addition latency that OP claims to be able to perceive impact performance of twitchy FPS players.
No. The reason was not economic, the reason was that the government polled the people living there and found that support for remaining British is ~100%.
Oil was found later, the fisheries were never worth maintaining the island for.
> No. The reason was not economic, the reason was that the government polled the people living there and found that support for remaining British is ~100%.
The Russians in Donetsk and Luhansk also voted to be part of Russia and not Ukraine, crazy stuff.
Man, you truly are done drinking that kool-aid, you finished the whole jug by the look of it. I'd recommend you to come back to this thread every couple of years and re-read your slop, maybe you'll learn one or two things about yourself while at it.
All you can respond with is outrage because your position is not internally consistent. You can see its morally reprehensible when the Ruzzians do it but when it comes to your country oh no then it's fine because you just need your emotional support colonies.
Like no shit one of your countrymen above said you couldn't give back Gibraltar because it'd set a bad precedent for Malvinas.
Well for one, neither of the referendums in Gibraltar were performed within 6 months of being invaded by the UK. I don't think any(?) government recognises the results of the votes in the donbas regions - except for the country running a 4 year special military operation in the region. Those referendums were held 12 years ago, and still haven't been recognised by anyone.
> All you can respond with is outrage because your position is not internally consistent
Anyone who thinks that the situation in Gibraltar in 2026 is equivalent to the situation in the Donbas in 2026 is either ignoring _literally every single reason as to why they are different_, or they're just trolling. And honestly, based on the rest of your reply I'm pretty sure you're in the latter camp.
The Australian grid presently curtails ~7-18% of production every single day between 11:00 and 14:00.
I believe that incentivizing people to acquire batteries is precisely the purpose of the policy. It's good for the grid for there to be a lot of storage at the edges. As I understand it, the 24kWh cap is subject to annual review, with it being reduced/the policy being soft phased out once curtailment is no longer necessary.
The core of the EU system is much more elegant. It sets a day-ahead production price per 15 minutes at auction. In EU countries with reasonable distribution cost, dynamic rates are a quite popular way to shift consumption to when cheap electricity is abundant.
Australia has that as well, the wholesale market changes price every 5 minutes, but it is a big ask for average consumers to follow that price and shift power usage in real time.
> it is a big ask for average consumers to follow that price and shift power usage in real time.
I could imagine appliances that take that into account. e.g. an airconditioner that works harder during the cheaper periods. I wonder whether any retailers pass through the 5 minute pricing variation? Should be easy enough to monitor the price and adjust my AC via the infra red remote protocol.
We have Amber Electric which pass through the 5 minute pricing, and I personally have a lot of automations on Home Assistant based on it. But TBH it doesn't make much difference and it may not even make sense. The price just doesn't move in the way you may find it useful.
For example, in an average sunny summer day, the midday price is usually close to free, and you can have you AC on for several hours. But the price starts rising from 3~4pm, but if you turn off the AC at this point, the room temperature would also start rising quickly at the hotest time in a day.
Also, in winter days (now), the price could be at a relatively high level for all day around almost every day in a week, but you still have to use it regardless.
It is the spring and autumn that most of day time has free electricity but it is also the time you need the least of power.
The problem is, we don't just have an absence of evidence, we have evidence of absence. The area has been widely excavated, and there is a clear continuity of settlement with the same pottery, culture, and religion. There is simply no trace of any large-scale population movement. As far as we can tell, the same people continued living in the area in the same way, worshipping the same gods (still plural for way longer) with the only large change being the yoke of the nearby great powers going away with the collapse.
This of course doesn't mean that there cannot be a trace of truth in the story! It just has to have been morphed substantially over time. For example, it was common in the time to kidnap and move foreign nobility and artisans, while no-one much cared about the identity of the average farmer or goatsherd. It could well be that "the people of Israel" who were kidnapped meant the people who actually mattered, ie, a fairly small upper class group, who could move from the Nile valley to the levant without leaving much trace in either society.
I'm still personally partial to the observation that the story seems to originate during the Babylonian captivity, and the situation of the story greatly mirrors the conditions they were living under, but while complaining about their Babylonian overlords was probably not allowed, writing stories about the plucky underdogs outwitting the horrible Egyptian overlords with divine assistance was fine, even if it contained themes of returning home and of liberation from foreign rule. (Note that Egypt was the main rival of Babylon in this period, and the Kingdom of Judah was on-again off-again vassal of the Egyptians. The captivity was party imposed to prevent this relationship from continuing.)
Larger rockets are inherently more efficient, which is why all the commercial providers are moving towards them. And while yes, most of the providers are targeting primarily for LEO, if you have high payload capacity to LEO you can solve your issue of getting anywhere by packing in a kick stage. And cheap third-party kick stages are available and more are in development.
It's not quite as bad as the parent made it out to be, the largest I've seen is 32kB per token (where sometimes, a token represents a byte, but usually it represents more than one.)
It's forced by the nature of how LLMs use vector embeddings for language.
Basically, a single token in a LLM is represented as a n-element vector, where n is the "hidden dimension", also known as model dimension. In order for the model to be smart, the hidden dimension needs to be large, on the order of 2^16 on top-tier models. Elements of this vector are typically quantized to 2-byte floats, or sometimes smaller. Every possible fact is embedded as a direction in this very many dimensional vector space, and a token is related to a fact if the vector representing that token points into a similar direction as that fact. You can do vector math about these things, famously for most trained models, if you find the vector embedding for king, man, woman and queen, and calculate king - man + woman, the result is very close to queen.
(Does that mean that there are 2^16 possible different kinds facts about things in this model? No, because high-dimensional geometry is very unintuitively powerful. The facts are not axis-aligned, and they don't need to be perfectly non-orthogonal. This matters, because the numbers of individual vectors you can fit into a single 2^16 dimensional space that are orthogonal with each other (all angles 90degrees) is of course 2^16. But, if you allow for almost orthogonal vectors, the number is larger than the amount of atoms in the universe. If this sounds wacky, for people with a CS background it can help to think it working a bit like a bloom filter, in that collisions are possible. Although in actuality they are theoretical, because 2^16 is a very large number.)
The next big step is Medusa Halo, which will have a 384-bit LPDDR6 interface. Those should be able to support 256GB at release, with 512GB coming later with larger chips (But I don't know to what extent people should trust the memory vendor roadmaps.) I'm not sure if they will be out in a year. Probably will be in a year and half.
reply