Applied AI Engineer

We bring startups and disruptive tech companies to the local market, offering engineers the opportunity to work on innovative, high-impact products. Join us and become a member of their R&D team while we take care of employment, HR support, a benefits package, and an enjoyable work environment.

This isn't a frontier research role. You won't need a GPU cluster, and we're not optimising for papers or leaderboards. You don't need a PhD.

‍

You'll be the first person on our team whose whole job is how good our agent is. Owners trust Handle with their customers and their business, so good means more than finishing the task. It means getting the details right, understanding what the customer needs, and knowing when to ask the owner first. You'll build the evals that measure that, and run the loop that makes it better.

‍

Our engineers have laid the groundwork for evals and own the harness, tracing and deployment they run on. You'll take the evals over and own them from here.

‍

You need to love finding out why an agent did the wrong thing, and proving your fix worked.

• Build and own our eval suite: scenarios, rubrics, graders, benchmarks, and the quality gates every change has to pass.
• Run the hill-climb loop on the agent's skills. Find where it goes wrong, form a hypothesis, change the skill or prompt, rerun the suite, and prove the fix worked without breaking something else.
• Judge conversation quality as well as task success: whether the agent understood the customer, said the right thing, and helped.
• Test new models as they come out, and separate what the model changed from what the prompts, tools and skills changed.
• Spot repeated failures that local fixes won't solve, and make the case for restructuring a skill or the agent.
• Build our view on fine-tuning: when it's worth it, what data it would need, and whether a better skill or model gets there first.

• First month: take over the existing evals, learn the agent's main workflows, and set a baseline scorecard.
• By month two: the hill-climb loop is running, every skill change goes through a regression gate, and the first improvements have shipped and been verified.
• By month three: a repeatable way to compare models, a ranked backlog of improvements, and a short written view on whether fine-tuning is worth exploring.

• You've built and run evals for an LLM product in production. You've written graders, checked an LLM judge against human ratings, and caught a regression before users did.
• You've worked on multi-step, tool-using agents, and you know how they fail.
• You write the code yourself, from graders and harness code to experiments, and you read traces without waiting on anyone.
• You're disciplined about experiments. You run things more than once, account for variance, and can tell a real gain from noise.
• You have taste in conversation. You can read a transcript and say what was wrong with it, even when the task got done.
• You're comfortable setting your own agenda. As the first person in this area, you'll decide what to measure first.
• You work comfortably in English, which is the language of our team.

• Hands-on post-training or fine-tuning of open-weight models.