• 0 posts
  • 29 comments
Joined 3 years ago
Cake day: June 12th, 2023
  • The way you guys are working is not about speed. It’s procrastination. The work needs to get done. You can either do it now or you can do it when the bug reports and change requests start coming in. There’s no speed to be gained by procrastinating, often it’s the opposite.

    If it was me, I’d focus on producing better code despite the pressure. You know you’ve got coworkers spending time watching YouTube instead of turning their work in or picking up the next ticket. There’s your time to ask Claude to refine and refactor the code before you commit it. Just don’t be the slow guy and you’ll be fine.

    Just refactor as you go. You don’t have to over engineer things. KISS and YAGNI are valuable engineering approaches. But don’t fool yourself into thinking that turning your work in an hour or two earlier is going to make a big difference in how the higher ups see you.

    Where this really starts to pay off is

    1. Your name comes up less often when assigning bug reports since you don’t own the feature that is bugged. People notice this.

    2. Less time spent fixing bugs means more time making new features. Means you own a larger part of the codebase. People also notice this.

    3. When a change request comes in and you go “Oh yeah, that’s easy. I already considered that and it’s like a 1 line config/code change.” You look like a fucking wizard when this happens.

    This has always been my approach. Even in places with little to no quality standards. Hell, I think it works even better in places with no quality standards because it makes you stand out more.

    P.S. While you already have a job is the best time to look for a new one. Because you don’t have any real stakes for failure.

  • This isn’t even a QA level thing. If you write any tests at all, which is basic software engineering practice, even if you had AI write the tests for you, the error should be very, very obvious. I mean I guess we could go down the road of “well what if the engineer doesn’t read the tests?” but at that point the article is less about insidious AI and just about bad engineers. So then just blame bad engineers.

  • More to the point, that is exactly what the people in this study were doing.

    They don’t really do into a lot of detail about what they were doing. But they have a table on limitations of the study that would indicate it is not.

    We do not provide evidence that: There are not ways of using existing AI systems more effectively to achieve positive speedup in our exact setting. Cursor does not sample many tokens from LLMs, it may not use optimal prompting/scaffolding, and domain/repository-specific training/finetuning/few-shot learning could yield positive speedup.

    Back to this:

    even if it did it’s not any easier or cheaper than teaching humans to do it.

    In my experience, the kinds of information that an AI needs to do its job effectively has a significant overlap with the info humans need when just starting on a project. The biggest problem for onboarding is typically poor or outdated internal documentation. Fix that for your humans and you have it for your LLMs at no extra cost. Use an LLM to convert your docs into rules files and to keep them up to date.

  • This lines up with my experience as well and what you’ve described is very close to how I work with LLM agents. The people bragging about 10x are either blowing smoke or producing garbage. I mean, I guess in some limited contexts I might get 10x out of taking a few seconds to write a prompt vs a couple of minutes of manual hunting and typing. But on the whole, software engineering is about so much more than just coding and those things have become no less important these days.

    But the people acting like the tech is a useless glorified Markov generator are also out of their mind. There are some real gains to be had by properly using the tech. Especially once you’ve laid the groundwork by properly documenting things like your architecture and dependencies for LLM consumption. I’m not saying this to try to sell anybody on it but I really, truly, can’t imagine that we’re ever going back to the before times. Maybe there’s a bubble burst like the dotcom bubble but, like the internet, agentic coding is here to stay.

  • The generalized learning is usually just the first step. Coding LLMs typically go through more rounds of specialized learning afterwards in order to tune and focus it towards solving those types of problems. Then there’s RAG, MCP, and simulated reasoning which are technically not training methods but do further improve the relevance of the outputs. There’s a lot of ongoing work in this space still. We haven’t seen the standard even settle yet.

  • I can read your code, learn from it, and create my own code with the knowledge gained from your code without violating an OSS license. So can an LLM.

    Not even just an OSS license. No license backed by law is any stronger than copyright. And you are allowed to learn from or statistically analyze even fully copyrighted work.

    Copyright is just a lot more permissive than I think many people realize. And there’s a lot of good that comes from that. It’s enabled things like API emulation and reverse engineering and being able to leave our programming job to go work somewhere else without getting sued.

  • Yeah I don’t think we should be pushing to have LLMs generate code unsupervised. It’s an unrealistic standard. It’s not even a standard most companies would entrust their most capable programmers with. Everything needs to be reviewed.

    But just because it’s not working alone doesn’t mean it’s useless. I wrote like 5 lines of code this week by hand. But I committed thousands of lines. And I reviewed and tweaked and tended to every one of them. That’s how it should be.

  • Kinda funny the juxtaposition between the programmers’ reaction to this compared to the “techies” reaction on the crosspost.

    Maybe we’re still early yet so I’ll write the difference right now for posterity: Programming post is generally critical of the article and has several suggestions on how to improve the quality of agent-assisted code.

    Technology post is pretty much just “REEEEEEEEEEE AI BAD”

  • I’ve thought about this many times, and I’m just not seeing a path for juniors. Given this new perspective, I’m interested to hear if you can envision something different than I can. I’m honestly looking for alternate views here, I’ve got nothing.

    I think it’ll just mean they they start their careers involved in higher level concerns. It’s not like this is the first time that’s happened. Programming (even just prior to the release of LLM agents) was completely different from programming 30 years ago. Programmers have been automating junior jobs away for decades and the industry has only grown. Because the fact of the matter is that cheaper software, at least so far, has just created more demand for it. Maybe it’ll be saturated one day. But I don’t think today’s that day.