I spend a lot of time ruminating on the failure of the contemporary tech middle-management class (largely PMs, but also a lot of EMs and VPs) to actually *manage* their products, and what it means that LLMs and ML tools make it ever easier to break away from path dependence.
The penalty orgs will pay for not putting folks in positions of responsibility that have a sense of taste and an understanding of fundamentals just went WAY up.
This will seem counter-intuitive; "but I can just vibe-code this thing now..."
But that means the output is a commodity. The *input* -- what you ask a machine to produce -- is more important. And as contexts grow, refactoring becomes cheaper.
If you're a software engineer asking tools to hand you output that isn't as tuned up as it could be -- something you can only do if you understand the interactions and options -- then the failure of *your* context grows in (negative) impact.
@slightlyoff I'm rather AI skeptic, but will admit they seem pretty good at contextually scaffolding stuff, part of me thinks that more about where most our frameworks ended up.
But, I'm highly skeptical that LLMs will be good a refactoring. This is usually the most error prone for experienced engineers, so hard to get right. I just don't think that LLMs have enough ability to understand logic to be able to refactor accurately.
That's much harder than genning code based on likely tokens.
@slightlyoff last time saw a colleague try to use copilot, it couldn't translate a JSON file into some Typescript interfaces accurately, and we spent 30 mins debugging what would have taken 5 mins to write if they had bothered.
@slightlyoff it'll be interesting to see where LLMs will go. I just highly doubt any of the hype will ever transpire.
I can see them having value in some domains.
I doubt that they'll be net positive on highly complex engineering challenges.
@intrbiz I think we're saying the same thing with different words. The premium for knowing what you're doing is going to rise, if only because so many won't and the results will stand out all the more.
@slightlyoff yes, I agree with that.
@intrbiz @slightlyoff in the scenario where the agent has to refactor something, it's going to be on us to have unit tests, type definitions and as many ways to verify the work as possible. I've been trying to wrap my head around it, but the recent Fly.io blog post kind of got me interested in figuring this out
https://fly.io/blog/youre-all-nuts/
@intrbiz We are seeing good results, e.g., in asking these tools to give folks a head start on large transitions (e.g., "please rewrite this Framer animation in CSS")