AI Disclosure: AI helped turn my Scala meetup presentation, experiment reports, and notes into this post. The experiment and the argument are mine.
A funny thing happened while preparing my Scala meetup presentation. I set out to challenge a familiar claim: AI is good at Python because there is so much Python out there, but surely it struggles with a niche language like Scala.
First … a little disclosure …
I am quite far along the “let the AI cook” end of this spectrum. The red circle is me.
Image: Adapted from the Lopopolo–Zechner spectrum.
So I built a dark factory. Actually … two. One for Scala and one for Python. Then I gave them the same Advent of Code 2025 problems.
Advent of Code supplies programming puzzles with concrete inputs and checkable answers. This comparison covers Days 1–9: two parts per day, or 18 parts per language. The picture shows the full 2025 calendar; the measurements below cover those first nine days.
The language comparison was interesting. But the economics were what got my attention: 101–109 minutes of solver work for 18 parts, with verified solver token spending of less than $10 per language for this workload.
The first block is Scala; the later blocks are Python. Blue is DeepSeek V4 Pro, doing the solving. Green is DeepSeek V4 Flash, doing the supporting work such as gathering, submitting, and reviewing. The under-$10 figure covers the Pro solver in each language; the green portion is additional spending by the Flash agents.
Now assume a senior engineer needs an hour to solve and test each part: 18 hours, or two nine-hour days. At an assumed fully loaded cost of $800 a day, that is $1,600, compared with less than $10 in solver tokens. Under those assumptions, the solver does the implementation work roughly ten times faster, with a much smaller marginal token bill.
This compares solver time and token spending with assumed human labor. Factory setup, infrastructure, supervision, and the supporting agents’ work are outside the comparison. It is not an all-in cost or end-to-end delivery benchmark.
Think of each puzzle as a very well-defined bug-fix ticket with a clear test case to verify the result. That is the opportunity! A dark factory can free up a significant amount of time and money to build more product and features!
This is semi-scientific. Feel free to disagree with the assumptions and the interpretation. But there is code, there are tests, and there are measurements.
The resulting code lives in scala3-aoc-2025 and python-aoc-2025.
The underlying Scala timing report, Python per-part measurements, and quality assessment document the snapshots behind the charts. Both the reports and the company packages are publicly available.
Here is the picture that explains the setup best.
Image: Diagram by Brij Kishore Pandey, as credited in the image.
My shorthand for it: Buy the blue, build the red.
The blue boxes show the general machinery: model calls, tool loops, frameworks, long-running execution, and a control plane. Paperclip is the foundation I used to organize that machinery into a company: agents with roles, assigned tasks, scheduled wake-ups, handoffs, and a visible history of the work. It coordinates coding agents; those agents still need a runtime and tools.
Image: Paperclip AOC Solver 2025 Dashboard.
The red box is the harness: the instructions, context, skills, scripts, and stopping rules that make this particular factory useful. Paperclip cannot guess what “done” means for my Scala repository. I have to encode that.
The company packages live in pcc-aoc-solver-2025-scala and pcc-aoc-solver-2025-python. They contain the agent definitions and operating skills; the solution repositories contain the code and reusable project helpers.
For Scala, the harness knows about sbt, formatting, tests, and publishing a verified solution. For Python, it knows about Ruff, pytest, coverage, and the equivalent publication checks. Before a handoff, the solver has to commit, push, and verify that the intended code really is on the remote branch.
That is engineering work. It is also work I can reuse for the next puzzle.
Image: Paperclip AOC Solver 2025 Agent Org Chart.
The CTO coordinates the work. The Problem Gatherer gets the problem and input. The Problem Solver implements and tests the solution. The Problem Submitter checks the published handoff and obtains the site’s verdict. The Solution Reviewer reviews the evidence and captures reusable improvements.
The WatchDog observes health and progress. The Model Tester handles separate model and routing probes.
The intended workflow is:
CTO → gather → solve → submit → review → wait.
One CTO wake-up authorizes one part. The next part waits for a later wake-up and a completed review. A rejected answer goes through review before a repair is authorized. Some early runs exposed gaps in these rules; the harness gained explicit gates as a result.
The solver uses DeepSeek V4 Pro; the six supporting roles use DeepSeek V4 Flash, all through OpenRouter. The Python company inherited the Scala company’s model choices.
There are loops inside loops here. The solver develops and tests until it has a result. The workflow handles a rejected result and repair. The outer loop advances through the puzzle parts.
Means: I am spending my engineering effort on the factory’s behavior and its definition of done. The agents do the repeated implementation work.
The median solver time was 5 minutes 21 seconds for Scala and 5 minutes 3 seconds for Python. Python was faster on ten parts; Scala on eight.
Adding the measured runs gives 109 minutes for Scala and 101 minutes for Python. That is less than two hours of solver work for each language.
These clocks include reading, implementation, compilation, tests, Git, and handoff administration. Scala’s numbers even include some sbt and SemanticDB recovery work. This is more than the time it took the model to type an answer.
The charts use the selected successful solver run for each part, excluding superseded attempts. They also exclude gathering, submission, review, queues, and waiting between parts. An overnight service interruption alone made one Scala part’s end-to-end clock exceed ten hours.
That distinction matters. So does the amount of useful implementation work the solver got through in those minutes.
Across the 18 parts, fresh input plus output totaled about 2.49 million tokens for Scala and 2.34 million for Python. The two large spikes have specific explanations in the reports: Python’s Day03 part2 generated a lot of verification context; Scala’s Day09 part2 combined geometry work and extensive checks with an overlapping solver session.
Those are things to investigate in the harness and workflow. A tall bar does not automatically mean that the programming language is intrinsically harder for the model.
The chart excludes cached input, which adds another 37.7 million recorded tokens for Scala and 18.2 million for Python. Fresh input, cached input, and output have different prices, so the chart alone cannot tell us the bill.
The spending chart near the top shows the dollar costs. I verified less than $10 in solver tokens per language for this workload, excluding the supporting agents. The checked-in reports document token usage; the dollar figure comes from my spending verification.
Both languages produced working solutions for all 18 parts. The quality assessment went further: correctness robustness, readability, idiomatic use, test depth, and simplicity/efficiency, each weighted equally.
Python averaged 4.67 out of 5. Scala averaged 4.04. Python was ahead on 16 parts, with two ties. Its advantages included algorithm choices and deeper tests. I cannot honestly turn that chart into a claim that the code was equally good.
These were single-pass ratings by Muse Spark, supported by source inspection and automated evidence. They are useful feedback, with the limitations of a single AI judge. There was no human-versus-factory quality assessment.
There were other limitations, too. The Scala solver read an old solution for Day01 part1, and the runs were not repeated blind trials. I would isolate history and repeat the experiment before making stronger benchmark claims.
For me, the useful result is that both factories produced working, tested code, and the review identified concrete ways to improve it. The next version of the harness can use that feedback.
I started with Scala versus Python. I came away thinking about the cost of turning a well-defined problem into tested, working code.
Under the assumptions above, the factory was faster and dramatically cheaper at that implementation work than allocating a senior engineer to do it by hand. That is a concrete reason to build and improve these systems.
The engineer still has a job: choose the problems, specify the result, build the checks, investigate failures, and improve the factory. The quality gap and the operational problems give us plenty to work on.
Next step: give the factory another useful job.