Ruby, Rails, COVID and the LLM wave

I started from a fairly mundane irritation: each time I opened the monthly Ask HN: Who is hiring? thread, Ruby jobs seemed a little thinner on the ground. Was this actually happening, or was I remembering the good months and quietly ignoring the bad ones?

My first data point was simply my own impression, which is not much of a data point, so I pulled the threads. The scope then grew: I went back to January 2019, counted employer posts, tracked Ruby, Rails, Elixir, remote work, and AI language, and tried to separate two large events, COVID-19 and the public arrival of ChatGPT.

The revised analysis is more conservative than the first version. It compares the target stacks with other language and framework ecosystems, uses models suited to counts and shares, tests whether the decline was already underway, and asks whether January 2023 is a unique break date. The answer is less theatrical but more defensible: remote work changed sharply around COVID, Ruby, Rails, and Elixir later lost relative visibility, and this dataset does not identify LLMs as the cause.

What is being measured?

I count top-level employer comments, not replies. A company post can contain one job or ten; it can mention Ruby as the main language or as one item in a long stack. This is a measure of posting visibility, not vacancies, hires, employment, or developer productivity.

The formal panel covers 65 monthly threads from January 2019 through May 2024 and contains 40,087 top-level comments. It comes from a frozen HNTrends lexical panel. The separate July 2023-July 2026 extension contains 695 retained target-stack matches, of which 692 are marked as core matches. It uses a different extraction pipeline and remains descriptive rather than being appended to the formal regression panel.

The revision also builds two comparison groups. The language target index combines Ruby and Elixir and compares it with Python, JavaScript, TypeScript, Go, Java, Rust, PHP, C#, Kotlin, Scala, and Clojure. The framework target index combines Rails and Phoenix and compares it with Django, Spring, Laravel, React, Node.js, Express, ASP.NET, Vue, AngularJS, and Svelte. These indices are sensitivity tools for relative visibility, not employment-market portfolios.

The eleven overlapping months show high aggregate agreement between the historical and recent pipelines, with count correlations around 0.96 and mean absolute errors below one post per month. That supports directional continuity, but it is not a post-level precision or recall result. The revised package includes a blinded annotation protocol, and the human validation and final author sign-off remain pending.

COVID changed the address more than the stack

March 2020 had 655 top-level posts. April had 462, a 29.5% one-month fall. In the revised negative-binomial model, monthly posting incidence is estimated to be 24.6% lower during April-June 2020, with a 95% interval from 32.4% to 16.0% lower and an FDR-adjusted q below 0.001. The post-acute level is higher in the fitted specification, consistent with recovery rather than a permanent 2020 collapse.

Remote language is the durable part. The odds of a post mentioning remote work are 99.9% higher in the acute period and 249.3% higher at the post-acute adaptation level. A simpler twelve-month comparison moves from 29.90% before the break to 62.75% afterward, an increase of 32.85 percentage points. This is the clearest result because it is large, persistent, and visible without depending on one fitted discontinuity.

Ruby and Rails also dip during the acute interval: their posting odds are estimated to be 20.3% and 26.9% lower. Elixir's estimate is effectively zero with a wide interval. None of the three has a statistically clear post-acute adaptation-level change after multiplicity correction.

The comparator results weaken a stack-specific COVID story further. The acute language target/comparator gap changes by -9.5%, and the framework gap by -18.0%, but both uncertainty intervals span declines and increases. COVID changed where software work happened much more clearly than it changed the relative visibility of these stacks.

AI rises while the market contracts

ChatGPT was introduced publicly on 30 November 2022, so January 2023 is the first complete post-launch month. In this study that date is a chronological marker, not a treatment assignment. It coincides with layoffs, monetary tightening, financing changes, and trends that were already visible in the stack data.

The revision separates broad AI language from LLM-specific terminology. Broad AI predates ChatGPT and cannot stand in for LLM adoption. The LLM-term series, built from GPT, GPT-3, GPT-4, ChatGPT, and OpenAI mentions, is more event-specific but still measures words in job posts, not whether a firm has adopted generative AI.

The January 2023 broad-AI level estimate is positive but not statistically clear after trend, seasonality, and multiplicity correction: +19.7% in odds, with q = 0.185. The subsequent rise is much clearer, with broad-AI odds increasing by about 5.66% per month through May 2024.

At the same January 2023 marker, total hiring-post incidence is estimated to be 38.0% lower. Ruby odds are 31.6% lower, Rails odds 34.2% lower, and Elixir odds 61.7% lower; all three target-stack level differences remain statistically clear after FDR correction. The model does not find clear post-January target slopes, so these are conditional level differences within the chosen specification, not estimates of jobs displaced by ChatGPT.

The decline relative to comparator ecosystems

A falling share can reflect a shrinking market, a changing mix of employers, or a stack losing ground relative to alternatives. The comparator analysis is designed to separate some of those possibilities.

At January 2023, the Ruby/Elixir language index is 35.9% lower relative to its eleven-language comparator index, with a 95% interval from 45.8% to 24.2% lower and q below 0.001. The Rails/Phoenix framework index is estimated to be 38.0% lower relative to ten framework comparators, but its interval reaches a small positive value and the FDR-adjusted result is not statistically clear (q = 0.138).

The relative-language result makes pure denominator contraction an incomplete explanation. It still does not identify the mechanism. Employer composition, startup financing, lifecycle trends, and technology substitution unrelated to LLMs remain plausible.

The decline was already underway

The January 2023 coefficients do not begin from a stable pre-period. During 2021-2022, Ruby share declines by 0.133 percentage points per month and Rails by 0.104 points. Elixir declines by 0.024 points, with weaker statistical evidence. The language target/comparator gap declines by 0.91% per month and the framework gap by 1.18% per month; both pretrends remain statistically clear after correction.

This is the most important caution in the revision. A model can estimate an additional January 2023 level difference while the series is already moving downward. The coefficient describes deviation from one fitted continuation; it does not name the cause of the longer decline.

The date-sensitivity test moves the assumed break month from January 2021 through November 2023. For Ruby and Rails, 17.1% of candidate dates produce a fitted level change at least as negative as January 2023. November 2022 and February 2023 are more negative. The language-gap fraction is 14.3%; the framework-gap fraction is 31.4%, with its most negative dates later in 2023.

That does not make January 2023 an unreasonable marker. It shows that the series does not contain a uniquely timed break that can be assigned cleanly to ChatGPT's launch. The transition spans a wider market interval.

Do AI mentions track Ruby, Rails, or Elixir?

The consistent panel contains only seventeen post-January-2023 months. Within that short period, one can ask whether months with more broad-AI or LLM-specific language also have less target-stack language.

None of the six detrended relationships is statistically clear. A null result in seventeen months is not strong evidence that no relationship exists. It does fail to support the specific claim that months with more AI or LLM language systematically have less Ruby, Rails, or Elixir language after removing a linear time trend.

The recent extension

The cleaned post-level archive supports three complete July-June comparisons. Equal-length windows avoid the partial July 2026 month and give each period the same seasonal composition.

The denominator is almost flat at 3,822, 3,931, and 3,999 total posts, while the Ruby-or-Rails core share falls from 5.44% to 4.10%. This makes a smaller overall HN thread an incomplete explanation for the latest decline. It still does not identify AI as the cause.

Remote-only target-stack posts decline modestly from 76.1% to 71.1%, while hybrid rises from 12.1% to 20.1%. Salary disclosure is more common in the latest period, and the median midpoint among annual USD ranges rises from $165,000 to $180,000. Those salary observations are selected and mix geography, seniority, and role type, so they should not be generalized into a compensation trend.

What the evidence supports

COVID created a durable remote-work shift. Posting volume contracted acutely and later recovered. Remote terminology rose sharply and remained structurally above its prior baseline. This is the strongest result in the study.

Ruby, Rails, and Elixir lost relative HN visibility. The decline appears in direct target shares, in the Ruby/Elixir comparison with other languages, and in the recent extension after total thread volume stabilizes.

The post-ChatGPT timing does not identify LLM displacement. Broad AI is not the same as LLM adoption, the short within-era correlations are inconclusive, target and comparator gaps were already declining, and January 2023 is not a unique statistical break.

HN is a selective channel. It reflects a particular employer mix, technical culture, and geography. One post can contain several roles, and repeated posts need not represent new vacancies. These results should not be generalized to the entire software labor market without external comparison.

The working paper still has unresolved quality gates. Aggregate overlap is encouraging, but human post-level precision and recall validation, an independent rerun, citation verification, and final author sign-off remain pending.

Would this change what I learn or use?

Not dramatically. I would still use Ruby and Rails where their development speed, readability, and ecosystem fit the problem. I would also be less comfortable treating “Rails developer” as a complete professional identity. Deployment, databases, performance, search, data pipelines, and AI integration are adjacent capabilities with obvious value.

The data does not say that a programmer should throw away the old tools. It says the surrounding workshop has become larger and more crowded, while these tools occupy less of one public hiring channel.

There is a temptation to end with something dramatic: LLMs killed language X, remote work won forever, or the market will never recover. The seven-year view is less tidy. COVID produced a clear remote-work break. The post-ChatGPT period combines a lower hiring market, accelerating AI language, and declining target-stack visibility. The revised evidence makes the relative decline more credible and the causal claim less credible.

Paper and reproducibility

The revised working paper presents the formal models, comparator construction, robustness checks, ethics and privacy safeguards, and full bibliography. Marian Posaceanu is the sole named author. The paper separately discloses extensive software assistance and does not list OpenAI, ChatGPT, or another tool as an author.

The accompanying research package contains the selected aggregate panel, sanitized recent datasets, generated tables and figures, analysis and validation scripts, LaTeX source, data and provenance documentation, privacy reports, licenses, and a SHA-256 manifest. Raw HN comment text, usernames, contact details, external application URLs, private inputs, and incomplete human-annotation outputs are excluded.