Robert Stewart, CTO at Arbital Health, on transforming healthcare payments and empowering actuaries to code

Host: Lucas Hendrich, CTO at Forte Group
Guest: Robert Stewart, CTO at Arbital Health

Introduction

Lucas Hendrich: Welcome to the CTO to CTO podcast, where chief technology officers dig into the challenges and opportunities leaders face every day at the intersection of business and innovation. I'm your host, Lucas Hendrich, CTO at Forte Group.

With me today is Robert Stewart, CTO at Arbital Health. Thanks so much for joining us.

Robert Stewart: Thanks for having me, Lucas. Great to spend time with you.

What Arbital Health Does

Making value-based care work

Lucas Hendrich: Explain it to a 12-year-old: What is Arbital Health, and how does it work?

Robert Stewart: Arbital Health is a health tech startup founded about two years ago, almost exactly. At its core, we're trying to make value-based care easier to administer and even more successful in the United States.

The US healthcare system has a lot of challenges. A big one is how healthcare gets paid for, and how you set incentives. Value-based care is a model based on outcomes, which raises the challenge of how you measure outcomes. And healthcare finance, like finance in general, has a lot of complicated actuarial aspects.

So we acquired a value-based care actuarial firm, Santa Barbara Actuaries, which has absolutely amazing healthcare actuaries. The company was founded by Travis May and Brian Overstreet, with the view that data fragmentation creates an opportunity to improve how healthcare gets paid for. Travis has done that in a number of different areas.

We're building a platform around the actuaries to scale up what they do. We sit between the payers — typically health insurance plans — and the providers, or companies that enable healthcare providers and physicians. We help with negotiations, but specifically with reconciling contracts: essentially the arbitration and adjudication of value-based care contracts.

Lucas Hendrich: So in a sense, you connect both payers and providers through a platform?

Robert Stewart: Exactly. We build a modern platform — I have to say AI-powered, but it genuinely is, and that's a key element; there are lots of opportunities there, and it makes a lot of sense for us. The platform ingests all the data, cleans it, reconciles it against the actual contracts — which contain a lot of complicated actuarial concepts — and renders the data in visualizations that anyone participating can understand.

Fee-for-service vs. value-based care

Lucas Hendrich: Where does the value-based part come in? What you're describing sounds like it would apply to any payer-provider connection.

Robert Stewart: Great question. The traditional way healthcare has been paid for in the US — and arguably still is today — is fee-for-service: providers get paid a fee for providing a service. In some ways it's simple to administer, theoretically. You come up with a set of procedure codes. For example, a moderate-complexity follow-up visit with your primary care provider is 99213. There are codes for everything.

But to get reimbursed, there has to be a code. There are many things providers can do that don't have codes — or at least historically didn't — like responding to an email. Without a code, they can't be reimbursed, even though it could be very helpful. That's the traditional model.

With value-based care, providers are compensated not just for the volume of activities but for the actual outcomes. That's great and makes a lot of sense; I've been a total believer for a while. The challenge is measuring those outcomes.

That applies to a lot of things, including engineering. I always get asked how to measure engineering productivity, and measuring outcomes is challenging. So value-based care adds administrative burden. What we're trying to do is help it succeed by taking that admin burden off providers.

Lucas Hendrich: So essentially reducing operational friction.

Robert Stewart: Exactly.

Bringing AI to Subject Matter Experts

Merlin AI: chat with your data

Lucas Hendrich: One thing we discussed briefly before the podcast is how you're using AI. A challenge with any digital solution, but particularly AI, is that to be effective it relies on subject matter expertise, whether that's a trained model or an actual expert training it. How do you marry the technology with something like actuarial science, which to the uninformed seems like a dark art — extremely dense, with a lot of very complex math?

Robert Stewart: One of the initial ways we're using AI is a feature called Merlin AI. Essentially, it's chat with your data, and it's very powerful. Many people involved in care — especially clinicians, but also medical economists — may not know SQL. And personally, I'd rather my provider focus on the clinical side than on writing code to interact with data.

Merlin enables that. You can ask questions about the data in natural language and, even more powerfully, get visualizations built for you. Rather than creating lots of one-off reports, you use AI to interact with the data, which saves a ton of time.

That only works, of course, if the data is well-structured and the context is well communicated to the AI. So we spend a lot of time with our subject matter experts, the actuaries, working out how the data needs to be structured and how to communicate value-based care contract provisions.

There is information about value-based care and contracting on the internet, but it's not the most popular topic.

Lucas Hendrich: Which is kind of surprising.

Robert Stewart: Yes. And more importantly, the actual contract provisions aren't public information, of course. So we take a very common RAG architecture approach: we index all that information and inject it into the prompt as part of the user's interaction.

The actuarial engineer

Lucas Hendrich: So you're bringing the technology to your subject matter experts, rather than the other way around, where you'd have to train engineers on the actuarial side.

Robert Stewart: Right. We've created a role that I don't know exists anywhere else: the actuarial engineer. Many platforms are very subject-matter-expert-heavy, and in our case the subject matter is actuarial science.

Many actuaries write code, though often it's scripting and analysis rather than full software engineering on a platform. But it's a good step toward it. Becoming a credentialed actuary involves entire degree programs and many very hard exams. Getting a data engineer to do all that is a lot of work.

So we thought it might be easier to go the other way: Can we get our actuaries — at least some of the analysts — to the point where they can write production code? We've been super successful at that, and we're continuing to grow that team.

What's made it possible, I think, is AI, especially AI coding tools. They cover much of the extra work it would take to build up basic knowledge of how to write this code. We even have actuaries writing front-end code — the whole end-to-end stack — and it's been surprisingly successful.

Lucas Hendrich: Is that code ultimately committed to a production platform?

Robert Stewart: Yes, it's part of the production platform. It takes more oversight from software engineers, of course, but the actuaries who've made the transition into those roles have been really successful.

Guardrails

Lucas Hendrich: I think we're starting to see this at many companies: the product function, which starts with subject matter expertise around a specific problem, is now able to write code and ultimately build out the platform. But there are guardrails, and there's always a last-mile problem. What guardrails do you use so you're not creating a vibe-coding explosion? A friend of mine had to review a thousand-line, purely AI-generated commit, and it took him two weeks.

Robert Stewart: The actuarial engineers are paired with people who have traditional software engineering training, and the team's leader also has very deep software engineering experience. We also use LLM-as-a-judge: another AI coding expert reviewing the code, which helps. But humans are definitely involved: another engineer reviews the code they write. And there are a lot of automated tests as well.

From Physics and Philosophy to Code

Sonar, assembly and Occam

Lucas Hendrich: Let's talk about your origin story. You have a background in physics. What kind of physics did you do — or do you still do?

Robert Stewart: I don't do much anymore, though I still do some writing. I got my degree in physics and philosophy at Rice University. My focus was nuclear physics, then space physics, mostly on the experimental side, and it was super useful.

After Rice, I moved to Austin and worked in a research lab doing underwater acoustics, designing sonar for submarines. Back then, the joke was that a bachelor's in physics qualified you for one thing: a master's in physics, which qualified you for a PhD. There weren't many job opportunities outside academia. But that was an amazing opportunity, and I learned to write a lot of code there.

Lucas Hendrich: Interesting. What were you coding in?

Robert Stewart: Assembly language was part of it.

Lucas Hendrich: Wow.

Robert Stewart: It was a real-time embedded system — VRTX32, if I remember correctly. These systems had to go on submarines that stayed underwater for a long time, so they had to be very reliable and performant. We used C and 486 assembler.

My favorite language then — and maybe still — was Occam. It was designed for Inmos transputers, chips built specifically for parallel programming, which we used for much of the image processing. We received signals from the sonar's transducers as they pinged out into the water and off the ocean bottom, and we rendered a 3D shaded bitmap of the ocean floor in essentially real time. This was a long time ago, so it was very cool, exciting work.

Lucas Hendrich: That's interesting. Another friend of mine also worked on submarines and is now a CTO in financial markets. Everything has a marine equivalent: finding the market signal and moving toward it. In his context it was torpedoes and finding targets, with the physics involved too.

Philosophy and ethics

Lucas Hendrich: When you said Occam, I thought of Occam's razor, which ties to your philosophy background. What did you study in philosophy, and what from it still informs you today?

Robert Stewart: My main focus was ethics — though as a bachelor's degree, you don't go super deep. I did a lot of reading. If nothing else, my philosophy degree prepared me to read dense material, stick with it, go deep, and then try to simplify it.

That's probably been a big part of my career: simplifying things, not always technical ones. And in AI ethics there are many interesting questions about how AI should be deployed and trained. I think my philosophy degree helped prepare me to understand those.

Ethics and Transparency in Healthcare AI

Making sure models point to the right patients

Lucas Hendrich: In the context of Arbital Health, what ethical concerns with your use of AI have you had to address? How did you think through things like bias, or governance of AI in decision-making?

Robert Stewart: I'd think of it in a couple of ways. First, we use a lot of traditional machine learning too. We have predictive models that try to identify things like high-cost claimants, so providers can focus on them.

Often, people are high-cost claimants because they have serious health issues, and helping providers get ahead of that is great — for the provider in the value-based care program, but also for the patient. The patients who most need help get it. So one aspect of bias and ethics is examining our models to make sure they aren't keying in on the wrong things and steering providers away from the patients who need the most attention. We take that very seriously, because providers will act on this information, and we need to enable them to take the best care of their patients.

Fairness to both sides

Robert Stewart: Another aspect of ethics: regardless of who pays us, provider or payer, we want to be very transparent about how we do the calculations, compute the data, and interpret it — including how the AI interprets the data. Whoever is looking at it, payer or provider, we want to be very fair in how we interpret the contract.

Lucas Hendrich: So lineage and observability are baked into the process and the stack.

Robert Stewart: Right. We keep track of where information came from and where the claims data came from, all the way down to which file a piece of information was in, so we can confirm it was actually part of the contract.

Machine Learning in Production Before MLOps

Reverse-engineering negotiated rates at Castlight

Lucas Hendrich: Going back a bit: you mentioned machine learning. People with a short history — and short attention spans — might think generative AI is AI. But if you've been in technology a while, you know AI concepts go way back, with all kinds of applications. Could you talk about your work at Castlight Health on prediction and price transparency, and where that sat in the evolution of AI?

Robert Stewart: I joined Castlight Health in 2011, pretty early in the company's life — it had about two customers live. Even then, the engineering team was using machine learning in production, in some really interesting ways.

The main focus was predicting prices for price transparency. This was many years before the No Surprises Act and the Transparency in Coverage rules. We were trying to help employees and their families at large self-insured employers understand what things cost before they went to the doctor. It seems like an obvious concept, but it was very difficult in the United States — and still is.

Lucas Hendrich: It certainly is.

Robert Stewart: It's very difficult, given all the complexity of insurance plans. First, we had to figure out what the costs were. They weren't public then, and they're only semi-public now. Your insurance plan can also have complicated provisions, and then there's where you are on your deductible, and whether it's individual or family.

Some of that is purely algorithmic: getting real-time data on where you stand with your benefits. But there was no way we'd have enough claims data to simply look up historical prices, and there were no credible price lists at that point.

So we used machine learning to essentially reverse-engineer the negotiated rates between payers and providers, using real claims data. There are many thousands of procedures, many medical groups negotiating independently with payers, and many locations. So many variables go into the actual price, even before you calculate a person's out-of-pocket cost.

We did some very cool work. One of my favorite algorithms was quantile regression. It's somewhat like a classification problem: we were trying to identify price ranges, at least for imaging procedures. We kept building on that over the years.

Predicting conditions

Robert Stewart: In late 2014, our business team realized we had enough data to potentially build models predicting conditions, like weight issues.

Lucas Hendrich: What do you mean by enough data?

Robert Stewart: At that point, we probably had about 1.5 billion medical claims — enough data on a large enough population to predict things like future weight management issues. Those are issues on their own, but they also lead to comorbidities, like an increased chance of developing diabetes, which we predicted too.

We also identified people who might be considering elective knee or hip replacement surgery, to get ahead of it. In many cases, we wanted to let those employees know their employer offered a second-opinion program, or that there were alternatives to consider, like physical therapy, if their claims showed they hadn't tried it yet.

In 2015, we built that system. It still felt like the early days of machine learning, at least in production for a system like this. We built it out, and it was very successful and pretty transformative for the company.

Building it by hand

Lucas Hendrich: That was before many of the managed services cloud providers offer now. What was building that production system like? What was the stack, and how much duct tape and wire was involved?

Robert Stewart: A fair amount. Nobody used the term MLOps at the time, because there wasn't any, and there wasn't much information about doing this at scale. Google Cloud's Vertex AI didn't exist, and neither did Bedrock; those came much later. There were some primitive tools — not to disparage them; it was early and they were still figuring it out — that helped manage experiments, but not full pipelines or APIs around them.

So we built it all ourselves, in Python, with scikit-learn, XGBoost, and the like — a very traditional approach. We had to do a lot of monitoring of outputs, looking for data skew and prediction skew, and checking whether the underlying data was the cause, because the data changes underneath you. The data sources we got from payers might change, and data quality issues might creep in. We built a lot of tooling around that, and it was a lot of manual work I'd never want to do again.

Lucas Hendrich: If you did it now, how much faster would it be to get to production? A hundred times? Or is there still the same complexity in building the algorithms?

Robert Stewart: Probably not a hundred times — that would be nice — but at least several times faster. A lot of the building we did, we could now get from the cloud providers or other ML service providers. Tools like Weights & Biases would have made our lives much easier had they been further along then.

But much of the work is data cleaning and understanding the data you're operating on. And you make discoveries. Before this, the diagnosis codes on claims weren't very important to us; they didn't affect what we did in the application. For these models, the diagnosis codes were key. Then we discovered data issues that, before, there had been no reason to investigate.

So a lot of work went into that, with a lot of back-and-forth with data providers to get them to clean up their data. When you start doing very different, more advanced things with data, you often discover your data quality needs are much higher than before.

Lucas Hendrich: Right. It's like a very sophisticated fighter jet needing a runway with no pebbles, versus a crop duster that can land anywhere.

Why Healthcare Interoperability Is Still Hard

Lucas Hendrich: Those data problems still persist, don't they? It's not something that has been, or can easily be, solved for?

Robert Stewart: I'd say it hasn't been solved. We talked earlier about telecommunications. An interesting thing about telecom is that its data is more constrained and higher quality, and interoperability is really high. It has to be. If I were on T-Mobile and could only talk to other T-Mobile customers, that would be terrible, and it wouldn't take off. In healthcare, interoperability is hugely important, but it's still a major, major problem.

Lucas Hendrich: So where did it fail? The digitization of healthcare has been a topic for, what, three decades? There's still a lack of standardization. Why does that persist? If it were purely an engineering problem, it seems like it would have been solved long ago.

Incentives and a long tail of systems

Robert Stewart: Financial incentives and pseudo-monopolies. Money is often what drives these things. But, not to focus entirely on that, it's also a highly distributed problem. There are so many individual physician groups, each with its own electronic health record (EHR) system, and many EHR vendors. The large hospital and health system space is fairly concentrated, but among smaller clinics there's a long tail of EHR vendors.

The Meaningful Use program made a big difference in incentivizing and encouraging EHR adoption. But how those systems are configured, and the semantics of the data in them, can be interpreted differently. That's one aspect.

On the incentive side, health plans sometimes lack the financial incentive or motivation to share data. And you have to be very careful: this is incredibly sensitive data, so they rightly limit its distribution.

Many of these systems were built in the '70s and '80s. There are jokes about things running on COBOL, and many claims adjudication systems at the older, more traditional payers really do. There have also been a lot of roll-ups combining them, so even within one health system there may be multiple claims adjudication systems, and you have to know which one actually processed a given claim.

So I think much of it comes down to the sheer number of provider groups storing this data. There are intermediaries, like health information exchanges, that solve part of the problem; many are nonprofits and not well funded. There's still a lot of work to do on interoperability.

Can AI Actually Lower the Cost of Care?

Lucas Hendrich: We could spend hours on that topic, so I'll shift gears — though there's one more question I'll try to cap at about two minutes. It may sound naive, but we keep hearing about foundation and frontier models breaking benchmarks, and that measure of progress seems very far removed from things like GDP or the cost of healthcare. Is there a line from those benchmarks — or from LLMs plus compute plus traditional AI — to actual reductions in healthcare costs? Is it a naive fantasy that we'll cross that bridge, or is it more a human question of untangling incentive structures that clearly add friction, or at least point in different directions?

Ambient scribes

Robert Stewart: That's a really interesting question, because there are a couple of ways it could help. One is simply a better experience. Some technologies are really taking off right now, like ambient scribes. At my last primary care visit, my provider was using one from Nabla. For the first time, he was actually engaging with me instead of being heads-down on his keyboard, taking notes.

I think there's a lot of opportunity there. We've hit a tipping point in the last year or two: these tools can not only transcribe but also extract codes. The diagnosis codes I mentioned earlier — providers don't have to look them up as much, because AI can do a lot of that.

That's nice for the patient experience. I can't imagine most providers love typing everything and spending so much time revising notes, typically in the evening — what they call "pajama time." It could also reduce the administrative burden of a different kind of coding: medical coding, entering the appropriate codes.

Lucas Hendrich: Right. And it removes the human from the loop there. Adding a code to something shouldn't cause so much friction, and yet it does.

Smoother referrals and simpler contracts

Robert Stewart: And as more information is captured, if you're referred to a specialist, the more the referring provider can pass along, the more efficient the specialist visit can probably be. Specialty care is where a lot of healthcare costs are, more than primary care.

So making those transitions smoother is a big opportunity: passing information to the referral, and getting it back too. That's a big challenge — providers refer you to a specialist and then never hear back. If AI can facilitate that, it could make a big difference.

And as I described with what Arbital does in value-based care, there are many complex ways to administer these contracts. If we can use AI to cut out much of the manual effort spent interpreting them, I think it can have a pretty material impact on the cost of care in the United States.

How to Run a Great Hackathon

Open the aperture beyond engineering

Lucas Hendrich: Absolutely. Again, we could spend a lot of time there. Shifting gears: you run hackathons. Most people know what one is, but broadly, it's a group of engineers, product people, and designers coming together to jam on problems and, through that, generate potential product ideas and solutions. That's the spirit.

I've found that running a hackathon can actually be complex. It sounds like a great idea, but then you have to figure out who judges and what the prizes are. There are details you need to be sensitive to in order to get the engagement you want — if engagement is the goal. Sometimes the goal is new ideas to bake into your products. How do you run hackathons, and what advice would you give people who want to run one?

Robert Stewart: At Castlight, I led or co-led 27 hackathons.

Lucas Hendrich: That's why I'm asking.

Robert Stewart: We did a lot of them, and I've done one at Arbital. They're a lot of fun. At Castlight, much of our focus was engagement, and one thing I did a little differently was focus heavily on inclusivity across all departments.

Hackathons often focus on engineering and product, and people think, "To hack, I have to be able to code." AI coding tools have lowered that barrier, making it easier to participate, but it can still be intimidating. So I went out of my way to reach out to the legal team, the finance team, and others. On average, about half or more of our hackathon participants weren't from traditional engineering or product roles. We got great participation, especially from customer support.

Lucas Hendrich: They know all the pain points.

Robert Stewart: They know the pain points because they're talking to customers. They have a lot of ideas, but maybe aren't sure whom to ask or how to pursue them. Our team was very excited to pursue them, went all in, and it was fantastic. So I'd definitely recommend widening the aperture of whom you market to internally and encouraging them. It does take some effort to convince them it's not that hard and they don't have to code.

Teams and incentives

Lucas Hendrich: Do you pair people from customer support with a tester or a DevOps person? How do you configure the teams? Are they self-selected?

Robert Stewart: They were mostly self-selected. I did encourage engineers to reach out to people in other groups, rather than forming a team of five engineers, and to try to include someone from support. After a while it takes off, and people realize it's the norm.

As for your question about rewards and incentives: I typically had a couple of awards, and one gave more points to teams with people from more departments. So there was an actual incentive to build broader teams rather than very concentrated ones. And we did a lot of very cool stuff.

Swag helps a lot — T-shirts, of course.

Lucas Hendrich: It's funny how people respond to it.

Robert Stewart: Yes. Another thing we did was donations. Winning teams chose a charity, and the company made a fairly sizable donation on their behalf. That was really motivating for many teams, because they got to choose. It's just a great thing. We also did gift cards and the like. I still have a little "Hack Day #9" microfiber yoga towel that my cat uses as a laptop blanket. So that's still around.

Lucas Hendrich: So two takeaways. One: bring in people from areas of the organization outside engineering. That may be easier at a startup building a SaaS platform than at a larger services or manufacturing company, but maybe not. I love that idea; I can imagine people from sales bringing a lot of insight into what the market wants. Two: incentivize with things like donations. That's a great idea. Swag is fun too, and it's funny how people compete for the smallest things. I was just at a basketball game where they shoot T-shirts into the crowd. That shirt is worth maybe $10 — you'd never buy it on the street — but you want to win it.

A hackathon is not free backlog work

Lucas Hendrich: One phenomenon I've observed — I don't know if it's a problem with hackathons — is the temptation to use them to solve actual problems faster. A cynical take is management saying, "We've got all this stuff in the backlog we can't deliver, so let's do a weekend hackathon and tackle all these bugs." But often you want a hackathon to generate ideas that eventually become part of the product. How do you draw the line between a hackathon and just getting free velocity?

Robert Stewart: Really good point. For judging, there was a popular vote, and then I'd weigh in — sometimes putting my thumb on the scale. Projects that came straight from the backlog got pushed down. It's great to do that work, but free backlog work isn't the point of a hackathon. That's a temptation you have to actively work through with teams.

The pandemic hackathon and vaccines.gov

Robert Stewart: I also want to mention that sometimes events can make a really big difference. When COVID hit and the shutdown began in early March, everyone was asking, "What do we do?" Suddenly we were all remote. Our CEO reached out to me and said, "We need to do a hackathon immediately, this month." OK, that's possible — let's do it.

About two or three weeks after the shutdown began, we held a hackathon. Of all 27, it had the most participants. The company was probably about 500 people at the time, and about 150 actively worked on projects — 20 to 25 projects in all. Eleven went into production, because people wanted to build things. They were frustrated and stuck at home.

I'm not saying wait for the next pandemic to run a hackathon, but there are opportunities like that, and themes to rally around. Sometimes we focused on reducing user friction, and gave people context, because otherwise people wonder, "What do I do?"

Lucas Hendrich: It helps them come up with an idea. Too much of a blank slate can be paralyzing.

Robert Stewart: Yes. And some amazing things came out of it. Later that year, we got connected to building the back end for vaccines.gov. If you went to vaccines.gov and ran a search, we were the search back end. We built that in six weeks, in November and December, and it was amazing.

It came out of the hackathon. Some of the early work there was about finding places to get a COVID vaccine. We built one of those and had it in production very quickly, and we ended up tracking the actual COVID vaccines. It was incredible, from home in December, to think, "It's real. I can see these vaccines." A lot of that came from the results of that hackathon. You never know where these things will lead or how they'll transform your business.

Lucas Hendrich: That's great. I hope it helps anyone in the audience thinking about running a hackathon. That's great advice.

Competing for Talent

A West Coast team and a flood of applications

Lucas Hendrich: A question about talent. We're in the Bay Area, and the labor market has fluctuated a lot over the last couple of years: hiring sprees, mass layoffs, more hiring, more layoffs, in different areas. Are your teams mostly local or distributed? And how do you compete for the best talent in an area where, it goes without saying, competition is fierce?

Robert Stewart: The engineering team is heavily concentrated on the West Coast, mostly in the Bay Area. We have an office in San Francisco that we go into two days a week, and probably close to half the engineering team comes in very regularly. That's a really nice balance. Having most of the team on the West Coast also helps; time zones make things simpler.

Much of our work involves sensitive healthcare data, with a lot of regulations, and some of our contracts with other companies prevent us from doing any of that work offshore. That's why we focus on the US. The actuaries and the rest of the company are a bit more distributed.

Competing is tough. And on an AI note, there's the deluge of applications. When we open a position, it's not unusual to get 250 to 300 applications a day. Some positions have drawn 1,000 applications in a single day. Sorting through all of that is one of the big challenges of recruiting — or do you skip it and focus on sourcing with a recruiter?

Mission matters

Robert Stewart: Competition is very difficult, especially as the Bay Area has taken off again, particularly for AI-focused businesses. Not to be clichéd, but the mission-driven aspect really does matter. It certainly matters to me, and to many of my coworkers. You want to work on something meaningful.

Fixing the US healthcare system — or at least doing a significant part to transform how healthcare gets paid for, reduce administrative costs, and shift spending toward the patient instead of the underlying systems — being part of that is, I think, a big advantage for us.

Lucas Hendrich: Absolutely. And it's not just removing friction; it's changing the model itself, moving to value-based care.

In-person hackathons

Lucas Hendrich: Since most people are local but some actuaries are distributed, when you run a hackathon, does everyone join from everywhere, or do you do it in a local office?

Robert Stewart: We did ours as part of a company off-site. Remote hackathons add an extra challenge. I wouldn't say don't do them, but if you have the opportunity, in person is a good way to start, especially for a first one. It's easier to explain, easier for people to engage, and more fun to get in a room or a section of the office, whatever space you have.

To start, even allocate just two or three hours — a mini-hackathon. That's what we did as part of the off-site, and there was a huge amount of enthusiasm. It's easier to keep things moving, help people when they struggle, match up teams, and give them really quick feedback.

Lucas Hendrich: And this goes beyond engineering, so you're face-to-face with customer support people who deal with pain points every day.

Robert Stewart: Exactly. It makes it much easier for people who may not work together — and might not feel comfortable reaching out over Slack — to team up on a project. In person, it's a lot easier.

The Junior Engineer Question

Lucas Hendrich: My last talent question for now. If you went purely by the news cycle, you'd think no one hires junior engineers anymore: agents write all the code at the junior and mid levels, and senior engineers are 10x reviewers of everyone else's code. Reality is obviously more complex, especially in very complex domains with a lot of institutional, tribal knowledge. I think there's still an argument for investing in developing talent, especially junior talent.

There's pressure to say, "We can do so much more with AI, so we don't need to hire; we'll keep headcount flat." But people eventually age out, and you don't want to lose that knowledge, so you have to keep cultivating it. It depends on the company, the domain, and the problems you're solving. How do you balance that? With all these open positions, are you hiring at all levels, or only for people with a certain depth?

Robert Stewart: It's definitely a hard challenge, and something I worry about a lot: the hollowing out of the junior layer. At first, the optimism about these tools was that they'd level up someone with almost no experience to the equivalent of two or three years. I didn't think that was really the case.

And the tools were less appealing to senior engineers, because they weren't good enough. Seniors spent so much time correcting suggestions that there was no value. That's changed. Now they're really powerful tools for senior engineers, and in some cases even more leveraged, because seniors quickly know when the output is wrong and can guide it. They've spent years thinking through the questions you'd ask if you were pair coding.

Lucas Hendrich: Or if you're an actuarial engineer.

Robert Stewart: Yes. When you pair-code with someone, what do you talk about? Now you have those conversations with the tools, and seniors have been having them for years. People right out of school have to some degree, but not as much.

We're currently not doing as much junior hiring as I'd like. It's a tough trade-off when you're building tools and a platform in a complicated area, and also trying to bring someone along from the beginning. I think it will get easier as we grow. When you're a smaller organization with fewer people to mentor, it's tough.

We've been scaling the teams significantly, and I'm hoping we soon hit the inflection point where adding a person doesn't grow engineering by 20%, but by 5% or less. Then it's easier to integrate more junior engineers.

Mentorship

Onboarding, pair coding and skip-levels

Lucas Hendrich: Do you do any formal mentoring in your organization, to upskill people or just engage them more?

Robert Stewart: We don't have a formal mentorship program, but our onboarding process is quite good, and more structured than at some much larger companies I've worked at. There's an emphasis on pair coding and on taking new people through the basics of what we do. It's not formal, but mentoring happens organically.

Lucas Hendrich: Do you mentor people?

Robert Stewart: I've scaled up the team to have a group of managers who work with me, and I do my best to mentor them. I also do skip-level one-on-ones with all the individual contributors in our group.

That's always a challenge as you grow from a small startup: one-on-ones often start out very task-focused, about getting things done. Now that the engineering org is around 20 people, we're big enough to shift toward career development and how I can help them, rather than project management.

Richard, and Things That Make Us Smart

Lucas Hendrich: Did you have a mentor who made a mark on you and helped your career?

Robert Stewart: Yes. I had a mentor at my last company whom I'd also worked with two companies earlier, straight out of grad school. He was fantastic, with really broad experience not just in engineering but in finance, and he wanted to be involved in every aspect of the company.

Early on, he sparked my interest in not just focusing on my engineering work, but trying to understand marketing and finance, talking to those people, and spending a lot of time with them. He is a great manager, and I was very lucky to be an individual contributor on his team for a few years. He was very empathetic and focused on very productive one-on-ones. He also learned about individuals personally, getting to know people's motivations and incentives, and spent a lot of time understanding what someone actually wants to do. That's my experience with him — Richard.

Then there's something that's not quite a mentor: a book. Things That Make Us Smart by Donald Norman had a big impact on me. There's an intuition that everyone likes what you like: "I think this work is really cool, so everyone on my team will too."

The book is about how we learn. Some people learn through practice and tuning — getting really good at one thing — and they're very comfortable with that. Others want to soak things up like a sponge and want every day to be different. Those are very different learning styles.

My insight from reading it was that I need to understand how people want to learn and what kind of work they want to do. I shouldn't assume that because I find it really cool to learn something new every day, that's comfortable for everyone else. Watching Richard interact with other team members, and with me, reinforced that. I've learned a lot from books, but having a mentor model it firsthand changed my career.

People come to work for different reasons

Lucas Hendrich: That's great. I've had a couple of mentors who made a similar impact. I've said before that organizations can be like an operating system, where everything is intentional: interviewing, structured one-on-ones, meeting cadences driven by true necessity rather than routine or comfort.

One COO in particular had a big impact on me, shaking me out of a purely technologist's view of the world and helping me see organizations almost as organisms, and really understand individual motivation. My boss at my very first job out of college once told me, "People come to work for different reasons. Don't assume they all come for the same one." Understanding what that reason is matters enormously. And I can see how having a mission attracts the kind of talent that will really commit to building something interesting.

Closing

Lucas Hendrich: This has been a wonderful discussion, and I really appreciate it. Any other thoughts before we wrap up?

Robert Stewart: It's been fantastic — wonderful questions, and great to spend more time with you. One thing to wrap up with: always be open to opportunities, and look for companies with a mission that can really change things, especially at a startup. It's tough, as I remember from Ben Horowitz's The Hard Thing About Hard Things.

Lucas Hendrich: I was thinking about that.

Robert Stewart: You wake up and eat glass every day. There's a lot of romanticism about startup life, but it can be grueling and mentally tough. So it helps to have that focus and understanding of why you're doing what you're doing — not just financial incentives, because you never know how those will work out.

You want to be in a role where what you do matters and you like the people you're with. Then you get the added bonus of a promising idea that can really take off. Those opportunities are out there, and I was lucky that this role found me.

Lucas Hendrich: When you're in a sector with big problems to solve, you couldn't be in a better spot. Thanks so much. It's been a lot of fun.

Robert Stewart: Thanks very much, Lucas.

Lucas Hendrich: That's it for this episode. Join us next time for more real-world conversations between technology leaders. And if you liked what you heard, subscribe to CTO to CTO. See you soon.

‍