AI as a test of organizational learning

AI adoption is becoming a test of how fast a company can learn, how safely it can absorb more autonomy, and how honestly it can look at the bottlenecks that AI makes visible.

In the latest episode of the #137 Podcast, we spoke with Boaz Adato, Head of Engineering Guild at HoneyBook – a company with a forward-leaning engineering mindset – about how these shifts show up in real engineering teams.

In this blog post, we’re sharing a short set of takeaways from that conversation. And if you want the complete context, examples, and nuance, we highly recommend listening to the full episode.

[fs-toc-h2]AI adoption is not a single GPT moment

[fs-toc-omit]1. AI adoption is not a single GPT moment

Boaz describes HoneyBook’s AI journey as something that “unfolded on many separate events.” That feels closer to reality than the common story of one breakthrough moment after which the company suddenly becomes AI-first.

First, a few people notice that something has changed. Then, early adopters start experimenting. Then leadership aligns around the importance of AI. But the real work starts after that: turning interest into workflows, budgets, internal tools, security rules, metrics, and habits.

[fs-toc-h2]Leadership alignment is only the beginning

[fs-toc-omit]2. Leadership alignment is only the beginning

One of the most practical points from Boaz was that leadership can agree AI is important but “someone has to break down the vision into actionable items.”

That translation layer is often missing. A company can have executive support, tool access, and motivated engineers but still fail to create a repeatable operating model around AI.

This is where engineering and product leadership become critical. Someone needs to decide where AI should be used, how experiments are shared, what is safe, what should be measured, and how the company keeps updating its approach as the tools change.

[fs-toc-h2]The real gap is not between believers and skeptics

[fs-toc-omit]3. The real gap is not between believers and skeptics

A lot of resistance to AI is not ideological. It comes from people who tried early versions, saw hallucinations or unreliable output, and formed a reasonable opinion at the time.

The problem is that the technology keeps moving, while opinions often stay fixed. A workflow that failed six months ago may now work with a better model, better context, better tool access, or a stronger review loop.

So the real adoption gap is between people who keep refreshing their assumptions and people who are still reacting to an older version of the technology.

[fs-toc-h2]Adoption spreads through proof, not policy

[fs-toc-omit]4. Adoption spreads through proof, not policy

Boaz describes their adoption playbook as repeated communication, leading by example, and sharing success stories. That is a simple but important point.

People rarely change the way they work because of an internal AI policy. They change when they see someone close to their work getting real leverage: saving time, removing friction, building something that previously required another team, or solving an annoying operational problem.

Without a mechanism for sharing those examples, AI adoption in custom software development stays local. One person gets better, one team gets faster but the organization does not really learn.

[fs-toc-h2]The biggest impact is expanded agency

[fs-toc-omit]5. The biggest impact is expanded agency

The strongest examples in the episode were not about engineers writing code faster. They were about people outside traditional engineering becoming more capable of building first versions.

Boaz mentions finance building internal dashboards, analysts making coding changes, designers fixing small UI issues, and product or marketing teams testing ideas before pulling engineering into the loop.

His line, “all roles have more power into being more independent,” is probably the clearest summary of the shift. AI is not only speeding up existing work; it is changing who can start work in the first place.

[fs-toc-h2]Some work is not getting faster. It is becoming possible

[fs-toc-omit]6. Some work is not getting faster. It is becoming possible

This is why productivity is too narrow as a frame.

The meaningful shift is not always from 1 to 1.3. Sometimes it is from 0 to possible.

A dashboard that would never have reached the data team’s priority list can now be built by finance. A landing page that would have waited for design and frontend can now be tested by marketing. A small product fix that would have sat in the backlog can become a draft PR.

That changes the operating rhythm of the company: fewer handoffs, less waiting, more small experiments, and more work starting as a concrete artifact instead of a request.

[fs-toc-h2]Greenfield and mature products live in different AI realities

[fs-toc-omit]7. Greenfield and mature products live in different AI realities

Boaz makes a useful distinction between greenfield work and large mature products.

In greenfield projects, AI can create obvious velocity gains. The context is cleaner, the risk is lower, the codebase is smaller, and the cost of throwing work away is acceptable.

As we build R&D teams, we’ve noticed that in a mature product, especially one with real users, production constraints, legacy systems, and fintech-level security concerns, the value is different. AI may help people move more independently but it does not automatically transform system-level metrics overnight.

[fs-toc-h2]AI exposes bottlenecks before it removes them

[fs-toc-omit]8. AI exposes bottlenecks before it removes them

One of the most honest parts of the conversation was Boaz saying that PR throughput had not moved as much as expected.

That matters because AI can create local wins while the broader system remains constrained. If the bottleneck is review, AI may simply produce more work waiting for review. If the bottleneck is architecture, AI will hit unclear boundaries faster. If the bottleneck is ownership or security, autonomy will stop there.

This does not mean AI is failing. It means AI is showing where the system is actually blocked.

[fs-toc-h2]Technical debt is becoming AI adoption debt

[fs-toc-omit]9. Technical debt is becoming AI adoption debt

Boaz mentions modularization as one of the ways to make LLMs more useful in large codebases. That is a practical and underrated insight.

A messy codebase does not only slow down engineers. It also limits what AI can safely understand, change, and validate.

If the system has unclear boundaries, implicit rules, fragile dependencies, and scattered context, AI will not magically remove that complexity. It may amplify it by generating plausible changes faster than the organization can safely review them.

Technical debt used to mean that people move slower. Now it also means AI has less room to create leverage.

[fs-toc-h2]Internal AI adoption can move faster than product AI adoption

[fs-toc-omit]10. Internal AI adoption can move faster than product AI adoption

Boaz says HoneyBook has AI features in the product that create value but he expected the impact to be more meaningful.

This is a useful warning for product teams. People inside tech companies are unusually motivated to experiment with AI. They understand the context, tolerate rough edges, and recover quickly when the model gets something wrong.

Customers are different. Especially in SMB or mass-market products, they may still prefer familiar interfaces and workflows. Internal readiness for AI does not automatically mean market readiness for AI-native product experiences.

[fs-toc-h2]Chat is probably not the final interface

[fs-toc-omit]11. Chat is probably not the final interface

Chat is a strong entry point because it gives users a blank canvas. But for many real tasks, people do not want a conversation with software.

They want the right output in the right format: a dashboard, a table, a draft, a workflow, a comparison, a decision screen, or a ready-to-review change.

The stronger product opportunity may not be “add an AI chat.” It may be using AI to reshape the workflow and generate the interface that fits the task.

[fs-toc-h2]AI adoption should be measured by flow, not activity

[fs-toc-omit]12. AI adoption should be measured by flow, not activity

Boaz’s preferred metrics are grounded: cycle time, PR throughput, review bottlenecks, and eventually unique PR contributors.

That is much more useful than counting prompts, subscriptions, generated lines of code, or “AI usage.” Those numbers can rise while the company itself does not move any better.

The real question is whether work flows differently. Are changes moving faster? Are reviews less blocked? Are more people able to contribute safely? Are teams reducing handoffs, or just producing more artifacts?

[fs-toc-h2]AI is not producing exponential gains in mature engineering orgs yet

[fs-toc-omit]13. AI is not producing exponential gains in mature engineering orgs yet

Boaz is clear that the change in broader engineering metrics is “definitely not exponential.”

That is valuable because it cuts through the hype. AI can be extremely useful and still not immediately create exponential improvement across a mature engineering organization.

For that to happen, the company needs supporting changes: better tools, guardrails, modularization, review processes, internal context, and new habits. The model is only one part of the system.

[fs-toc-h2]AI spend is becoming its own operating discipline

[fs-toc-omit]14. AI spend is becoming its own operating discipline

HoneyBook allocated a significant AI token budget for engineering and expected to exceed it. The more interesting part is how they use limits.

Boaz says caps are not mainly there to block people. They help the team understand who is spending heavily, what they are doing, and which use cases are worth expanding.

That is the right framing. AI spend behaves less like a fixed SaaS subscription and more like cloud infrastructure: variable, usage-based, and highly dependent on behavior.

The goal is not to minimize usage blindly. The goal is to make usage conscious.

[fs-toc-h2]Security has to become enablement

[fs-toc-omit]15. Security has to become enablement

One of the strongest organizational framings from the episode is Boaz calling security an enablement team.

That is exactly the right mindset for AI adoption. More autonomy means more surface area for AI mistakes, data exposure, unsafe actions, and unclear accountability.

The answer cannot be to block everything. It also cannot be to let everyone do anything. The real work is defining where AI can act, where it can suggest, where human review is required, and what guardrails make autonomy safe enough to scale.

[fs-toc-h2]The future is not necessarily smaller companies

[fs-toc-omit]16. The future is not necessarily smaller companies

Boaz pushes back on the extreme idea that AI means every company becomes a two-person company.

The more interesting possibility is that we get more software, more niche tools, more internal automation, more temporary applications, and more experiments because the cost of building has dropped.

That is a better lens for founders. The question is not only “who will AI replace?” It is also “what can we now afford to build?”

[fs-toc-h2]The best people are not the ones with the strongest AI opinions

[fs-toc-omit]17. The best people are not the ones with the strongest AI opinions

Near the end of the conversation, there is a useful hiring insight: using AI only as a chatbot is already behind but claiming to have the full AI playbook figured out is also a red flag.

That tension feels right.

The best people are not the loudest AI believers. They are the ones who keep rebuilding their workflows as the tools change, while staying honest about the limits, risks, and unfinished nature of the field.

The main takeaway

AI adoption is becoming a test of organizational learning.

The companies that benefit most will not simply be the ones with the most tools or the biggest budgets. They will be the ones that can turn local experiments into shared practice, make their systems legible enough for AI to work inside them, manage spend without killing exploration, and let more people build without letting quality or security collapse.

That is harder than buying tools and much more important.

Keep up with On The Spot for more podcast episodes, engineering insights, and TechSpot community events. And if your company is currently figuring out how to integrate AI into engineering workflows or looking to build and scale high-performing AI engineering teams, feel free to reach out to us.