Exploring Generative AI
Generative AI and particularly LLMs (Large Language Models) have exploded into the public consciousness. Like many software developers I am intrigued by the possibilities, but unsure what exactly it will mean for our profession in the long run. I have now taken on a role in Thoughtworks to coordinate our work on how this technology will affect software delivery practices. I'm posting various memos here to describe what my colleagues and I are learning and thinking.
TDD inside the agent loop - theater or actual value?
A small, informal evaluation of whether prompting a coding agent to follow TDD inside its own agentic loop actually improves the quality of the resulting code and tests, compared to no TDD instructions at all.
10 August 2026
The Economic Benefit of Refactoring
An experiment to see if decomposing a large function helps reduce token costs, suggesting that is may now be possible to measure the economic benefit of refactoring
30 July 2026
Experiences with local models for coding
Experiences with local LLMs for programming, compared with two standard tasks and and trying the best LLM in day-to-day use.
08 July 2026
Viability of local models for coding
The factors that influence how viable local LLMs are for programming tasks.
07 July 2026
Humans and Agents in Software Engineering Loops
There's been much talk about how AI agents affect the workflow loops of software development. Kief focuses on the goal of turning ideas into outcomes by building and managing the working loop.
04 March 2026
Harness Engineering - first thoughts
Why OpenAI's recent write-up on “Harness Engineering” is a valuable framing of a key activity in AI-enabled software development. Harness includes: context engineering, architectural constraints, and garbage collection of the code base.
17 February 2026
Context Engineering for Coding Agents
Powerful context engineering is becoming a huge part of the developer experience of modern LLM tools. Birgitta explains the current state of context configuration features, using Claude Code as an example.
05 February 2026
Assessing internal quality while coding with an agent
Assessing how using a coding agent affects internal code quality of CCMenu by adding a feature using the agent, and seeing what happens to the code.
27 January 2026
Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl
Making sense of spec-driven development by looking at 3 tools: Amazon's Kiro, GitHub's spec-kit, and the Tessl Framework
15 October 2025
Anchoring AI to a reference application
One of the challenges with service templates is that once a team instantiated a service with a template, it’s tedious to feed template updates back to those services. Does anchoring AI agents to a template or reference application make that easier?
25 September 2025
The constant little risk assessments Birgitta makes during AI-assisted coding, thinking about probability and impact if AI gets it wrong, and if she will be able to detect that.
23 September 2025
Partner with the AI, throw away the code
The common metric of AI code acceptance has big hole. LLM can be helpful even if you throw away its code.
31 July 2025
Even with LLMs, Birgitta still cares about the code: “LLMs are NOT compilers, interpreters, transpilers or assemblers of natural language, they are inferrers.
09 July 2025
Autonomous coding agents: A Codex example
An example of using OpenAI's Codex, helping us understand better what they do under the hood, and which agent category they fall into.
04 June 2025
Building Custom Tooling with LLMs
Even an experienced developer may be reluctant to build a simple tool due to the amount of plumbing required. Unmesh narrates his experience iterating with an LLM to handle the boilerplate of an extension to PlantUML.
14 May 2025
Coding Assistants Threaten the Software Supply Chain
Agentic coding assistants expand the attack surface of software supply chains. Jim and Lilly explore the measures we need to do to safeguard security when using them.
13 May 2025
Building TMT Mirror Visualization with LLM: A Step-by-Step Journey
Even experienced developers fumble around when working with a new platform. Unmesh shows how using an LLM helped him learn how to use an unfamiliar toolset.
30 April 2025
Guiding an LLM for Robust Java ByteBuffer Code
AI editors generate code with remarkable speed, but more is needed for robust systems. Unmesh describes a dialog between developer and LLM - demonstrating how expert guidance is critical.
17 April 2025
The role of developer skills in agentic coding
A list of examples of AI “missteps”, illustrating how developer skills still very much matter in agent-assisted coding, and ideas how to mitigate that.
25 March 2025
What role does LLM reasoning play for software tasks?
What role reasoning models can play or not play in coding assistance? A lot of stake is put into how reasoning models are a step change in coding assistance, especially the recent release of R1 has created additional buzz around that. But I don't see it - yet?
18 February 2025
Expanding the solution size with multi-file editing
GitHub Copilot released a new multi-file editing capability: a powerful coding assistance feature which expands the scope of AI assistance to larger changes over many files. Birgitta gave it a try to find out how useful it is and what it means for developer feedback loops.
19 November 2024
Building an AI agent application to migrate a tech stack
Exploring the potential of AI assistance for tech stack migrations. Birgitta describes building an agent that changes the testing framework used in a test. As a side effect you can also gain a better understanding of how AI agents work.
20 August 2024
Onboarding to a 'legacy' codebase with the help of AI
How can today's AI tools can assist with onboarding to existing, potentially messy codebases? Birgitta does this by trying to understand and solve an issue in a real life codebase. Read to find out about the parts of that journey where AI was useful and showed potential, and where not so much.
15 August 2024
How to tackle unreliability of coding assistants
how Birgitta determines her confidence level, giving the assistant a timebox, and using anthropomorphization to our advantage by coming up with an assistant persona.
29 November 2023
How is GenAI different from other code generators?
How code generation with GenAI is different from more traditional code generators, and on which software abstraction levels this new technology sits.
19 September 2023
At Thoughtworks, we are strong practitioners of Test Driven Development (TDD). Naturally this leads to the question of how generative AI can help with this technique. As ever, GitHub co-pilot can't be relied on to fly the plane, but can suggest some useful ideas for the red and green steps. It isn't very helpful for the all-important refactoring step.
17 August 2023
Coding assistants do not replace pair programming
Do coding assistants make the practice of Pair Programming obsolete? Spoiler alert: They don't.
10 August 2023
In-line assistance - how can it get in the way?
Two ways in which coding assistants can get in the way. 1) when they amplify bad practices or hinder refactoring by getting stuck in the past, and 2) when developers get tired of constantly reviewing code suggestions, instead of actively producing code themselves.
03 August 2023
In-line assistance - when is it more useful?
The most widely used form of Gen AI coding assistance in Thoughtworks at the moment is in-line code generation in the IDE, where an IDE extension generates suggestions for the developer as they are typing. Birgitta looks at the factors that impact the usefulness of these suggestions, indicating where they lead to safe waters, and the rocks that we need to look out for.
01 August 2023
Median - A tale in three functions
Birgitta asks GitHub Copilot to write a median function. It gave her three suggestions to choose from. The experience shows you still have to know what you're doing when asking LLMs to write code, since the LLM's programming skills are often rather flawed.
27 July 2023
If you're wondering why we use a donkey in our series image, read why I made up a persona for an eager, yet unreliable, coding assistant.