it's a general politician dogwhistle, don't underestimate "the left" (especially in countries like the US, where the democrats are really not that left compared to other left parties in e.g. europe; leaving you with a pointless right-wing vs right-wing fight where nobody wins)
Its worse than that. Its "think of the children" but with slightly more truth to the statement than " think of the children". Because it is definitely a national security threatwhile primarily actually being said to |imit freedoms and increase market power abuse possibilities for big tech and/or media.
I feel like hearing this is dangerous in itself - even if you overestimate yourself, surely you yourself don't overestimate yourself as much as others?
Personally, I ran against a closed glass door yesterday. All I hope for is that I was entertaining to onlookers at least.
It's when you don't have the capacity to know how dumb you are that you overestimate yourself, and you overestimate yourself (and underestimate others) far more than those able to see how dumb you actually are. It's when you are smart that you overestimate others, and underestimate yourself.
There are limits to this, plainly Einstein didn't think some guy as dumb as a bag of rocks was as smart as he was.
Funny story about that - there's a hotel in Edinburgh originally built in the mid-1800s, which was gutted and renovated with a lot of facade retention and architectural salvage to keep the look of the place. The centrepiece was the ornate brass and mahogany revolving door.
But when it was finished people kept walking into it the wrong way and getting stuck, and no-one could figure out why.
Eventually they sussed it out - until the early 1900s, no-one standardised on which way revolving doors go. Now, of course, they turn anti-clockwise - you pass people going the opposite way on your left, don't you?
> The alternative is that Netflix et. al can't be played
But that's Netflix's decision to enforce, not a foregone technical requirement. Not to mention that widevine has been cracked as far as I understand it and doesn't stop people from pirating it.
1 Gigawatt in space is about 31k acres of just solar panels, not including heat dissipation systems. Given we can't block the sky for advertising, and this would block the night sky worse, I don't see it happening.
Good introduction to the space in Python (although a lot of languages have it, and just as many disallow it). I'm still on the fence regarding how useful or confusing it is. without (or even with) good variable names, "a + b" is an absolute mystery, especially when handling objects that come from classes you have no detailed internal understanding of.
I feel like it is on you to understand these classes first though. In game engine programming, operator overloading is often just vector algebra. If you have no idea how matrix-vector multiplication works, then you have no business doing those things anyway, overloaded operator or not. But if you do understand it, it makes your code infinitely more readable than without overloading. And everyone with a math education will also understand that aspect of your code, no matter if they are familiar with your usual paradigms.
Agda has the most wonderful (terrifying?) operator syntax I know of. Any Unicode (except for a few reserved words) can be used, and you can indicate where operandi go with an underscore. For instance defining a function named _•_ would allow you to use a • b.
But there is no limit to where the underscores go: the name _? Would give you x ? postfix. To define if-statements one defines the name if_then_else_.
To piss off architects, managers and linguists all at the same time: language isn't much of a standard at all, it's more the current agreed-ish state of things, quite similar to the current state of a code base. Your inner model of what you like things to be is without direct effect to how things de facto are.
In addition: it's not wrapped around a problem, it's wrapped around an attempt at a solution - the problem space is often not even depicted in code, and often only minimally described in documentation.
It is that, and it’s the best we got. Code is downstream and strictly worse. Of course it is great for expressing you want integers of certain sizes being added, but that’s.. yeah, don’t know what to tell you if you think that is the problem domain.
It’s not a secret we use DSL’s to express our Actual Problem. The Ancients told us it is The Way. Problem is devs think stacking int64s in a struct is a proper abstraction boundary.
If you guys would have said proper DSLs are the spec I might have agreed but “code” in general without constraints is useless noise.
I've worked with these systems for four years now and they have not meaningfully improved in that time frame.
We still have:
- statistical correlation between two things will always cause one thing to lead to the other, no matter how much you prompt it to not have that connection (to be expected with a stochastic system)
- Math completely fails in longer contexts
- "thinking" token generation being on the correct track just to 'no, wait' on an already correct conclusion
- smearing of properties between logically distinct objects (a red ball and a green cube can quickly become a red cube and a green ball)
Yes! Impressive, isn't it? I see how it has improved for some minor points, that the big models can cover more finetuning ground, but my big gripes are still the same - you could do the same back then with multiple models and more targeted finetuning.
Am I missing something that my original points no longer hold for their products? Did it get meaningfully solved? Are your experiences flawless on that front?
What do you hope to achieve here? Posting absurd and ridiculous things and then continuing like if someone would take you seriously after that. Keep going I guess.
> you could do the same back then with multiple models and more targeted finetuning
I mean, come on, this is just not true. You could not achieve anything like what you can with modern agentic coding with Fable / 5.6 Sol from any combination or configuration of GPT 3.5 era models.
It's like saying that a teenager isn't an intellectually meaningful improvement over a toddler.
Sure they're both still fundamentally flawed humans prone to cognitive error, but one is clearly more likely to hit the mark than the other when assigned a task.
There’s no way you could get models as smart by fine tuning. I couldn’t throw a problem like “build a pokemon database with UI to teach my son sql” and get a working system, nice ui, tests (which it iterated on) examples and explanations in one shot.
There weren’t thinking tokens. Maths is now dramatically better, making actual contributions when before they were mostly mocked for making extremely basic errors. Smearing is also something say is very rare in frontier models.
If you think they have barely changed you’ve either forgotten what they were like or not used them more recently, or you’re just being obtuse.
My company had such a system four years ago, for internal work, somewhat more limited in scope (one language). What you are seeing as the frontier is not necessarily the best you can have - just because people don't try to push it to market as a product doesn't mean it's not there.
Edit: we do have a system that uses LLM and fixes the above issues largely (tracking of state, calculations and objects, still flawed in finer details). No, we don't sell, it's experimental fun and not really ready in terms of setup/ux/etc.
OP is delusional or deliberately optuse. I work in the space and stare down these systems 12h/day, and saying the systems haven't meaningfully improved is ludicrous.
OP is largely pissed with what OAI/Antrophic are trying to sell as meaningful improvements and the market-bending money they ask for it. I work in the space and we trained LLM models on conceptual tokens, not language tokens, for example. See Symbolic AI and all the attempts at hybrid models.
Also, uh, fame and riches are not really my thing. Middle income is fine. My mistake was speaking up here because I got carelessly annoyed because I have skin in the game, research-wise. I'm sorry for that.
There may not be "core" improvements (structural reliability) but there are "emergent" improvements (apparent intelligence). Already the IQ tests from Maxim Lott ( trackingai.org ) show a progressive sliding towards the right side of the curve - which btw translates to a very much non-secondary decline in the user's frustration (and progresses with an increase of usability).
The more they work on it, the more probable the jump becomes - e.g. to achieve the Large Conceptual Models you say you worked on.
You're welcome to speak up, but saying models haven't meaningfully improved in three years, when the most recent models are solving top-level frontier maths problems, is indeed going to strike most people as weird. I still have no idea what you mean.
Not sure what longer contexts we're talking about but didn't we have an old math problem optimized, which even the LLM itself was surprised about, just a week ago? Something which wasn't possible 6 months ago.
If they use Python to fill the gap, and the end user doesn’t have to know or care, is it unfair to assess this as progress and attribute the progress to the _system_?
OK, the core technology that is the language model still can’t math as well as you’d hope, but how about the end result users see from the system when they interface with it?
“Did you know humans are better at flying today than they were a thousand years ago?” ‘No they’re not, they need planes.’ Technically correct in a way but isn’t it kind of annoying to be so stubbornly pedantic when the context is speed of reaching Point B from Point A?
You are correct, the frameworks around it have improved. In that regard, my assessment is unfair: I only judge the underlying technology and what is sold by the sota providers, with the premise of what it's like when you start fresh. You can achieve a lot by coding around the issues, but that's kinda against the point of 'AI', is it?
Let us be more clear: there is no structural jump, no architectural overcoming of the original fault.
(Edit: and on a similar point, structural properties such as having static ntetworks, as opposed to continuously learning and improving architectures (such as us), will reveal that there is still road ahead.)
Maybe more fair then would be: “I've worked with these systems for four years now and while they _have_ meaningfully improved in that time frame, they’re not perfect and remain fundamentally flawed in various ways.”
You prompt less. You need not inject search results into the context window yourself, a window much larger than years ago. You get code that’s already been run successfully once instead of finding an obvious show stopping bug yourself.
The technology is not a brand new one that fixed everything wrong with the old one, no, but not sure I would’ve noticed your comment if it had been such a bland observation. I genuinely assume good faith here… will say am tempted to assume the standards of someone posting such a thing might be impossibly high. Glad to be having a fun conversation instead of getting your grades on my work product or something :)
If you go to a LLM without harness, GP original point in completely right.
LLMS by themselves are still shit at math, they still confuse weird correlation to causation every time (and sometimes in ways even a 9 year old would say "no, that's dumb"), and confuse original parameters very often.
I disagree with " "thinking" token generation being on the correct track just to 'no, wait' on an already correct conclusion", because i think that is an effect of the harness, not the LLMs.
80% off all the improvements since ChatGPT4 are in the harnesses, and the LLMs by themselves, while they improved in areas they already were good at (translation especially) did not fix any of they original issues (object permanence, calculusm correlation).
Just run old models in the playground and get them to play chess (maybe make a small custom harness if you feel like it), then replace it with a frontier model (i don't know if you still have API access without harness on US models, but if you don't try K3), you will see LLMs weaknesses were not at all fixed, even marginally. They're way better and not inducing bugs in the code, so that make them usable since Opus4.5 (anyone using them prior to that either had a greenfield project or like spending hours debugging).
I agree, harnesses is where everyone improved the most. Our internal experimental tool can now semi-reliably formulate small programs to assert the correctness of their theories, for example. Context length is still a weird factor that we haven't sensibly solved - if anything, the lesson learned was to keep the context as small as possible and do most of the true "thinking" in the harness and temporary generated code.
Because they're not serious about their offerings. They are an ISP and like to provide/bundle very basic additional services (hosting, mail, etc. and now 'AI')
reply