>what "sense of rigour"? it's way too soon to put those rose-tinted glasses on.
I don't think OP is claiming that prior to LLM coding everything in the software development world was super rigorous (I assume that's effectively what you mean with the "rose-tinted glasses" comment). But rigor was actually possible and in a deterministic way too, which is fundamentally impossible with LLMs. You can build all kinds of guardrails and processes around LLMs that make it somewhat approach rigor again, but it's still fundamentally based on a bunch of statistical probabilities instead of deterministic, repeatable results.
All of the methods I see to mitigate the fundamental and inherent issues of LLMs seem roughly equivalent to the kind of crap you see in astrology groups or palm reading etc. You need Venus and Mercury to be in alignment while Mars is retrograde if you want to be able to get the right results from your token predictor.
Sure, but LLMs are non-deterministic in ways that no sane human ever would be. See the "Is it better to drive or walk to the carwash" scenario from a few months ago as one of many, many examples. Or a personal example I encountered just a week ago: I asked Claude (Opus 4.8 in case any of the "you aren't using the latest model that totally fixes that issue" types are interested) to convert a bunch of DB calls that currently use raw ADO.NET calls to use Dapper instead.
The projects in this repo were on .NET 4.8.1 and were still using the older format for the .csproj file instead of the newer (and far better) "SDK-style" format that Microsoft introduced a few years ago. It tried to use the dotnet CLI to add references to Dapper, even though the older format of .csproj doesn't work with that. The dotnet CLI returned errors about trying to add the package references for Dapper, which Claude completely ignored while continuing to try and convert the ADO.NET calls to Dapper. And at the end it tried building the project, which of course failed, and then it confidently informed me that the conversion had been completed successfully and that the build completed successfully and all tests were passing successfully, even though the output from the build it had done immediately prior clearly told the LLM otherwise.
A real human, despite being non-deterministic, would have caught the issue at multiple stages. They would have seen the error when trying to add the reference. If they ignored that then they would have seen the red squiggly lines all over the (deterministic) IDE telling them there was something wrong, along with autocomplete for Dapper calls not working. And if they continued to ignore those and managed to keep going anyways, they would have clearly seen that the build failed, with tons of errors specifically about references to Dapper failing to resolve. An LLM keeps going on its merry way in ways that effectively 0 humans would.
I think we're going to see a big scramble to pick up the pieces in a few years when a bunch of vibe-slopped houses of cards come crashing down. I imagine it will be like the demand for COBOL developers but on a much more massive scale.
A few major failures will scare the risk mitigating bejesus out of some kinds of businesses, but maybe AI will be better than us at fixing those kinds of problems by then.
COBOL was mostly outsourced to India, and it's a terrible professional path for anyone in the EU or US, and has been since the Y2K bugs got fixed at the last minute.
(And probably a bad path in India, too, but I have no data one way or the other. It's just that all the excellent Indian devs I know use almost exactly the same tech stacks I do.)
In the case of vibe-coded slop like OpenClaw it's not a question of some vague notion of "code quality", it's a case of the software shitting the bed and not working anymore, with no recourse of fixing it. (Neither humans nor LLMs have the context window to analyse and fix tens of millions of lines of code slop.)
> and Anthropic banned OpenClaw usage
If OpenClaw wasn't broken it would just use a standard token API.
But see above - as software it is fundamentally broken and unfixable.
Then why did it take Anthropic over a year just to fix the flickering issue in one of their main products when they have internal access to the latest and greatest models?
>All else being equal, and assuming you are building the right thing, being able to deliver more correct lines of code is a good thing.
Why? If you can deliver the same thing in fewer correct lines of code wouldn't that be preferable? At a bare minimum if you're still insisting on using AI to slop out your project, having it do things in fewer lines of code means you can fit more into your LLM's context window.
> If you can deliver the same thing in fewer correct lines of code
it really depends on what you're doing. If your goal is "become interoperable with the N different and incompatible network protocols that people have devised for doing task X" I'd really like to know a solution that doesn't have at least some part of the amount of code that scales with N.
Example: consider https://bitfocus.io/connections which connects to 700 different things. Right now it's written with Node.JS, with one repo per connection (example: https://github.com/bitfocus/companion-module-meyersound-gala...). Let's say you want to make a similar product but that runs on ESP32 where performance is paramount so you need C++ or Rust. How do you do that without at least as many lines of code as the existing JS implementations for every system supported by Companion?
Without looking at the details, I expect that each network protocol has a checksum of some form, and there are likely a lot less than N different checksum algorithms. Similarly I expect several will have encryption - using one of a few standard algorithms (if any doesn't use a standard algorithm you have a strong case to say not supported). I also expect that there is a lot of protocol parsing - this can be done as custom hand coded for each, or using a parsing framework (and likely there are some places of generic code in between).
This is still not an argument for more lines of code. It demonstrates that lines of code are positively correlated with number of features, yes. But that's like saying the number of nails scales with the size of a house. More nails does not create more house.
The entire Linux kernel is about 40 million LoC, and only something like 16 million LoC after you remove drivers. I have a hard time imagining whatever OpenAI was talking about there having anywhere close to 6% as much utility as the Linux kernel, despite having 6% as many lines of code. And I have a hard time imagining it's anywhere close to maintainable, regardless of how powerful their LLMs might be.
To be fair, few things of any number of LOC have as much utility as the Linux kernel, and it's also a particularly dense example of code. There's plenty of other examples that have higher LOC / utility ratio without being vibe coded. For example, Google's monorepo famously has 2 billion LOC, which is a statistic I've heard long before LLM coding took over.
Clarification: Google claimed to have 2 billion lines of code in their repo ten years ago, and a commit rate of 50,000 changelists per day, both on exponential growth trends.
That's certainly a way to look at it. And that repo contains a "third party" directory which itself contains Linux, LLVM, and much of the rest of the open source world. But I would suggest that the largest of those thousands of applications probably has a transitive closure of hundreds of millions of lines of code.
I'm aware. I worked on that specific project (assuming we are talking about the same one) back in the day. :-)
There are certainly very large applications in that repo in the hundreds of millions of lines of code. But comparing the entire repo to single applications is not an apt comparison.
The Linux kernel is not in any way at top of big projects. A kernel, as the name suggests, deals with specific issues and tries to remain small.
The world’s biggest software is usually built over endless adapters of different data and a need to reconcile endless edge cases with laws, regulations and real world complexities.
When I was young, I remember a (joke) program from a book or magazine, called something like report writer or something. It was written in basic and you typed it in.
You would run it and it would say:
how many pages? _
You would type in a number, and it would generate that many pages of a complex-sounding report.
something like "the subsystem design interface is ..." blah blah
Sure but it certainly seems that way. There were already thousands and this PR adds a fair bit more too.
It is possible to have thousands of tests and a lot of blind spots, but it's easier to get a grip on that using instrumentation like code coverage and indeed by using LLMs to prod at edge cases.
I am OK with trusting that it is likely the React team understands how to rigorously test based on how well-tested React itself is.
>There are much simpler mechanisms to making that would make the enforcement mechanism more effective without destroying the economy, like prioritizing them by salary instead of randomly.
Rigorous examinations for English fluency and for competency in their alleged field of expertise would be a good start. I have several H1B coworkers in the US who barely speak intelligible English, and who barely understand normal conversation let alone anything technical. A blatant example of this that I experienced recently being that several of them could not understand that just because a method in C# is asynchronous does not mean it executes out of order.
This is a hiring issue, not a legal one. The US has no official language, and no language tests, so requiring English in law would be dicey to put it mildly. What if I'm hiring someone specifically to work at a Spanish language news outlet?
The H-1B visa is specifically for hiring "highly specialized" workers. Lack of the supposed skills that let them across the border is in fact a legal issue.
You don’t need to be Shakespeare to do specialized job that’s first
Second most visa applicants already get tested on their English skill when they apply for Visa, for example, universities require English proficiency for F1 visa using GRE exam
And why do you think you are better than an employer in assessing required English proficiency of an employee
Congress would need to declare any official language(s). Moreover, by treaty and law (NALA of 1990) obligations to Native American tribes there must be more languages than merely English.
>What if I'm hiring someone specifically to work at a Spanish language news outlet?
Having actually worked at a Spanish language news outlet before (1 of 4 tv and radio stations in the office I was doing IT help desk work in), I can tell you that every single employee spoke English somewhere on the level of very good to near native fluency. As it turns out, knowing English (or the native language of whatever country you're in) is an incredible value-multiplier for almost every job position imaginable.
As far as language issues at my current job goes, it turns out once you hire a manager that speaks both Hindi and English (or Marathi and English, or Bengali and English, you get the picture) it doesn't matter much if the H1Bs he hires barely speak English because he can just start shouting at them in Hindi if they don't understand (even if several native English speakers are in the meeting too).
That’s just an Executive Order. Executive Orders are instructions to the executive branch, not the country itself (obviously, the president doesn’t have that ability). Congress hasn’t passed a law establishing an official language in the US.
Honestly, Executive Orders in general are dysfunctional cludge. I feel less bad about things like DACA, since that's trying to fix something broken instead of wrecking things for no useful reason (or acting to puff up a sick ego) ... but hell no, that should have been a proper law.
No, that's a silly test. If I want to bring in a world renown battery expert from China, it's ridiculous to also add on "and you must speak English well". English fluency has absolutely nothing to do with expertise.
What we actually need is a higher minimum salary for H1B employees. Right now it's something like 50k per year, which is insanely low for a "hard to find expert" it should be more like $300k per year. H1B employees should be some of the best paid employees in a company. Raise that minimum salary and you'll overnight fix almost all complaints with the H1B program. Except for from the business owners who are abusing the system to get cheap labor.
Hah. And you think a Govt agency will be able to do a rigorous enough examination to eliminate people who don't know that just because a method in C# is asynchronous does not mean it executes out of order?
Reminds me of a friend whose job application got rejected by some Govt agency in Canada due to "experience mismatch." Job required "Software Programmer" experience but he was a "Software Engineer" instead.
Many of the people I grew up with "barely speak intelligible English". Communication is important and the easiest way to fix that is to bring people from your linguistic group to be a coworker....
> I have several H1B coworkers in the US who barely speak intelligible English, and who barely understand normal conversation let alone anything technical.
English fluency is certainly not a requirement for fluency in any technical field. Perhaps you mean that they cannot understand _your_ descriptions of technical topics, though
Seeing as my Russian, Ukrainian, Belarusian, Mexican, French Canadian, and Brazilian coworkers don't seem to have these issues with me I don't think the issue is with my explanations.
Funny, when I was in the US, my Russian, Ukrainian, Chinese, Mexican, French, British, and 99% of the American coworkers had absolutely zero issues with my Indian accent, except that one American guy who would ask me to keep repeating even though the rest of the room had already understood and processed what I said.
The problem likely lies deeper than just the accents; and by the way, the English requirement (including a verbal test) is already set in place for most of the workers. The regular halfway-decent ones will likely already have TOEFL scores hovering around at minimum the high 100s, and in the non-university hiring pipelines I have seen, the English/ESL tests seem to be common if you are not from an English-speaking country, so if you are seeing people where nobody can understand what they are saying, you need to take a better look at your employer's hiring practices.
>Anti-immigration policy blocks them from being Americans.
Yes, because the citizens of a country (through their elected representatives) have absolute control over who they choose to allow into their country. Even blocking a brilliant surgeon or inventor, if they so choose. There is no moral right to come to America (or any other country).
Saying "I have absolute control" is not a justification for making bad decisions that hurt the US. Furthermore, it was never a question of the US had a right to make these decisions, of course it does.
Do you find the argument "I have the right to make any decision I want therefor it justifies bad decisions" convincing? I sure don't.
>Saying "I have absolute control" is not a justification for making bad decisions that hurt the US.
How is it a bad decision that will hurt the US? Can you make that argument on its merits? No one doubts that there isn't that one genius here or there
Last year, right here on HN I saw a headline where the "powers that be" wanted to increase Canada's population to 100 million (they currently sit at 30ish million). Is that a good decision for Canada? Where the fertility rate is so low population is shrinking? Like, do they need another 65 million people? Are there 65 million jobs going undone in Canada right now? Jobs that desperately need doing? The plan's the same for the United States, even if no one was careless enough to blare a similar headline from trumpets.
They can, which is why many companies do the bare minimum malicious compliance to claim thet they attempted to hire Americans for these jobs. Things like ads in the local newspaper that 99% of qualified Americans will never see:
Those newspaper tech job ads have been going on for at least the last... 20 years. When you see those, the company already has the role filled, they just need a justification for the visa. "We tried to find a US worker but failed!" Which honestly may or may not be true, I think the ad is just standard procedure at this point.
If you are willing to discuss and argue with arguments, I can prove that the whole critique of H1B is nothing more than racism, hatred, and bigotry, from the people who are worst and least qualified to talk about the subject
have you considered asking DOL (staffed by White Americans) and Congress (staffed by White Americans) why they have that requirement, instead of blaming foreigners ?
I'm not assuming, I am speaking from knowledge. Congress has always been mostly white, as do DOL leaders, this is especially true for current administration
the newspaper ad is not for H1B, it is for PERM process, which is different.
Second, the local newspaper requirement is created by the Dept of Labor itself, specifically to protect local workers in the area where Labor Market Test is being done!!!
It is not malicious compliance by firms, it is goodwill compliance by firms, to whatever DOL requires them to do. Dont like it? ask your DOL why.
Third, paper ads create audit trail that DOL wants, they dont recognize e-boards like linkedin/indeed as their audit trail is considered "soft"
I don't think OP is claiming that prior to LLM coding everything in the software development world was super rigorous (I assume that's effectively what you mean with the "rose-tinted glasses" comment). But rigor was actually possible and in a deterministic way too, which is fundamentally impossible with LLMs. You can build all kinds of guardrails and processes around LLMs that make it somewhat approach rigor again, but it's still fundamentally based on a bunch of statistical probabilities instead of deterministic, repeatable results.
All of the methods I see to mitigate the fundamental and inherent issues of LLMs seem roughly equivalent to the kind of crap you see in astrology groups or palm reading etc. You need Venus and Mercury to be in alignment while Mars is retrograde if you want to be able to get the right results from your token predictor.