John Higginson, CTO at Curriculum Associates, on AI literacy, privacy, and building resilient teams

‍Host: Lucas Hendrich, CTO at Forte Group
Guest: John Higginson, CTO at Curriculum Associates

Introduction

Lucas Hendrich: Welcome to the CTO to CTO podcast, where chief technology officers dig into the challenges and opportunities leaders face every day at the intersection of business and innovation. I'm your host, Lucas Hendrich, CTO at Forte Group. In this episode, I'm joined by John Higginson, Chief Technology Officer at Curriculum Associates, a company focused on helping teachers and students achieve better outcomes through data and personalized planning. We talk about real-world applications of AI in education, what AI literacy means for modern engineering teams, and how leaders can create space for experimentation. Here's the conversation.

Welcome to the CTO to CTO podcast. I'm your host, Lucas Hendrich, CTO of Forte Group, and today I'm with John Higginson, CTO of Curriculum Associates.

John Higginson: Hey, Lucas. Good to be here.

AI in Education Products

Starting from the mission

Lucas Hendrich: To kick off, we're going to start with a topic that's been, if not beaten to death — apologies to anyone who's heard too much AI hype — still very relevant, especially for the products you're working on. I'm really curious about how you're integrating AI into the product stack and into your ways of working, and anything else you want to say about it.

John Higginson: I laugh, because I feel like as an industry we were infatuated with crypto, and then about three years ago we just crossed out "crypto" and wrote "AI." It's obviously a lively topic, and not just for internal product development. Customers ask us about it, our board asks us, and so on.

Let me start with our mission: to help teachers teach and help students achieve better outcomes. Our chairman, who used to be our CEO, always says internally that a kid never gets to take third grade again. We take that very seriously.

We also take privacy very seriously. One challenge with AI — with all its promise — is that it requires data to deliver. It has to be trained on data you have and on what it learns, and that raises real privacy concerns. So whatever we do with AI in our products starts from three questions. Is it good for teachers and does it help them teach? Is it good for student outcomes? And, above all, does it protect student privacy?

Speech AI and student privacy

John Higginson: To that end, about a year and a half ago we acquired a company called SoapBox Labs, which did pioneering work in speech AI. The software listens to a child read a passage and analyzes it: here's the rhythm of the student's speech, here are the phonemes they pronounced correctly or incorrectly. It gives the teacher a roadmap for helping them become better speakers and readers.

All of that has to be trained, so we have a process to collect real children's voices while anonymizing them and protecting their privacy, including protecting and anonymizing the recordings themselves. We're the first company in our space to receive a Digital Promise certification for protecting student privacy. That drives everything we do.

Assessment, resources and engaging content

John Higginson: AI shows up in our products in a few forms. One falls under the broad heading of assessment: how do we help teachers understand how students are doing relative to grade level? Speech and reading is one example, and there are other applications.

Another is helping teachers find the right resources. One way you and I use generative AI is to look things up. We used to Google it — and we've both been around long enough to have used encyclopedias.

Lucas Hendrich: Which is crazy to think about.

John Higginson: I know. I actually feel nostalgic for the old books. I want to go find them; I remember them on the shelf. But now you use AI to do research, so AI tailored to and trained on the materials teachers need would be really helpful.

We also create a lot of content. We've all seen the demos of AI-generated images, and we want lessons to be engaging. We think AI can be a powerful way to do that — creating images and other things that make a ten-year-old sit there and enjoy the lesson.

Inside engineering

John Higginson: Internally, like many companies, our engineering team has embraced tools like Copilot. We're looking at how AI can help us build better tests when developing software. And in general, we want it to take busywork off people's plates so they can do more impactful and interesting things.

AI Literacy as a Prerequisite

Lucas Hendrich: If you were designing an organization from scratch today — say, on the order of 20 to 30 people — what would it look like now? How would you staff it, how would you think about capabilities, and how would it change the software development lifecycle for you?

John Higginson: I think a bit differently from what's been said at companies like Salesforce — that AI will replace a whole class of engineers. Our engineering team is about 400 people, and in our experience it's actually engineers with some level of experience and discernment who get the most out of AI.

So I'd approach it as: AI literacy is a prerequisite for the people we bring in. An engineer in 2025 who says, "I don't need AI — I don't need Copilot, Claude, Cursor, whatever the tool is," is like an engineer ten years ago saying, "I don't need Stack Overflow, I don't Google anything," or ten years before that, "I don't need the developer network or the reference books." AI literacy is really important.

Then it's about figuring out where AI fits. It's a different way of using technology, but you still have software engineers and UX designers. UX designers are probably refining first drafts generated for them. Testers are probably starting from first drafts of tests, and product folks from first drafts of stories. The team is the same; they just have faster ways to get there.

Disruption is real

Lucas Hendrich: We see it very similarly: it's becoming a way of working more than anything, extending capability more than replacing it. The noise tends to provoke one of two views. The more common, fear-driven one is that it will replace jobs — a kind of Malthusian view. Whereas I think the real change, which we haven't seen yet, is how it will create new opportunities, new job descriptions, and new ways of working.

John Higginson: I agree. Not to get too philosophical — though that's what we do here — but if you go back to the Industrial Revolution, that has always been the evolution of technology. What we have to recognize as leaders and companies is that it is disruptive. We don't do anyone a service by saying, "Don't worry, it won't feel disruptive." It will.

My grandfather worked in a steel mill. In his lifetime, it went from a bunch of people with hammers to automation. That's been the march of technology, and his job changed from swinging the hammer to running the automation and the factory line. What's incumbent on us is to invest in training and education, and to make it safe for people to experiment, because things will change. That's a good thing, as long as we bring people along.

Creating Space to Experiment

Hackathons and champions

Lucas Hendrich: I feel we're on a razor's edge as leaders of engineering organizations. On one hand, you want to institutionalize it: when people start working for you, you want them trained on the right tools so they're enabled. On the other hand, the landscape changes almost every day. How do you reconcile that? What's your buffer for the risk you manage?

John Higginson: I think it depends on organization size. One luxury of being bigger is that we can carve out time for tech talks on a particular technology, or a hackathon — we run one every year — and give people space to experiment.

That serves two purposes. First, it feels safer for people to try things. I'm not writing the Jeff Bezos API mandate memo for AI — "Everything has to be a service; if you don't comply, you're fired; have a nice day." It's more: we'll try these things, and what works, we'll scale. Second, it lets us learn as an organization.

In smaller organizations the same thing happens, but usually through champions. In a 30-person engineering org, someone says, "I'm going to go play around with this and come back and talk to you about it." That works because the organization is small enough for everyone to get in the same virtual room.

Having been through technology shifts in engineering organizations, I think much of it comes down to encouraging people: finding champions, letting them run with it, and creating some incentives to push people in that direction. I know some organizations have talked about mandating it: "You must do this, you must do that." I'd take a different approach. We'll give you the tools, show you good examples of how people are using them, and then set a team goal that measures how effective we've been. We succeed or fail together. Mandates are a really hard, painful way to drive change.

Where the gains are

Lucas Hendrich: And I think we're still very much in the measurement phase. We're trying to model this. Some of our initial findings: using generative AI for unit tests is low-hanging fruit and a 10x productivity multiplier. Same for spikes in a sprint: a throwaway prototype that would have taken two or three days can now take two hours or less. That's a huge gain. But apply the same approach to something very domain-specific or algorithmic, and you might get 85% of the way in one shot, then lose ground trying to refine it. Are you finding that?

John Higginson: Yes. It's great for first drafts and prototyping. The example I've used recently: if you want to call an external service or endpoint, like the Google Places API, why write it yourself? Sure, you could figure it out. But just ask one of the LLMs to generate a code stub in your language of choice — Ruby, Java, whatever — and play around with it.

When you get into more complicated work at the core of the software you're writing, that's where the experienced engineer gets more value than the junior engineer, who doesn't know the domain or the context. I think that's true. But I'd caveat it: AI is evolving rapidly and giving better answers. It's still early days for agentic AI, where AIs link together to perform complicated tasks and hand them off. There are some cool demos and use cases, and they'll get more sophisticated.

So, going back to the root of your question, we have to maintain curiosity as engineering organizations. I still think we all got into this because we saw a future that didn't exist yet: "I can see where that's going." That's why we got excited about computers and the internet. We can't fall prey to thinking we've figured everything out and there will never be another invention.

Lucas Hendrich: The end of history.

John Higginson: Exactly.

The Human Part of Building Software

Specialized models and practical intelligence

Lucas Hendrich: I agree. It's more about changing how we work and produce things than about wholesale replacement of humans. It makes me wonder whether the pursuit of AGI will even be that useful, given all the practical uses for smaller, specialized models. Take SoapBox: the model you use for voice recognition isn't the one you'd use for general language tasks.

As models become more specific to different areas, it makes me think of the human body and intelligence. The "model" your eyes use isn't the one your ears or your gut use, yet they're all part of your experience of the world. I find that encouraging, because there's so much room to grow, rather than just racing to the bottom to commoditize intelligence.

John Higginson: I agree. What the product is and does matters. It's not just about cranking out more plastic rubber ducks. We're creating complicated software to solve complicated challenges — in schools, in running a business, in running a website. The human part — the design, the creativity — is where we want people to focus. The real promise of AI is freeing people from everything else.

If I could get an AI to do budgeting and forecasting for me — here's my credit card.

Lucas Hendrich: And have the conversation about it too?

John Higginson: Exactly — reach the conclusion. I'd rather focus on other things. That's the promise. We have to be curious, learn, adapt, figure out where it works, and recognize that it will be disruptive for people. It's a big shift, but there have been others. There was a time when, as a software engineer, you had to build every UI by hand in code. You don't anymore. There was a time when, if you couldn't figure something out, you were stuck. Not anymore. If we'd stayed frozen — "this is the way it is" — we wouldn't have AI or many things we take for granted.

A more efficient use of human cognition

Lucas Hendrich: I totally agree. Shifting what was human cognition to machines actually makes cognition more efficient. Think of writing code like data entry: do I need all these keystrokes for something whose result is a pattern reflecting a broader pattern, which I can now get from an LLM?

John Higginson: Yes.

Lucas Hendrich: But for a hard cognitive task — which usually involves human attributes like intent and curiosity — you're creating more space for it. It's a more efficient use of human cognition. Maybe I'll be proven wrong, and I probably will, but I don't know that you can model human curiosity the same way.

John Higginson: It's really hard to see. LLMs feel magical because they're trained on so much data that they're really good at knowing what follows what. That's how many LLMs work.

I do think design won't leave human hands for AGI. Fundamentally, creating software for people to use, whether adults or children, is something humans do. But a lot goes into it. From a DevOps perspective: I need a dev environment, a database, servers, a web server. People can do that, but you'd rather have them focus on designing and building the product.

Lucas Hendrich: Precisely. I could make candles and light my house with them. It's quaint, and a fun hobby.

John Higginson: Yes.

From Liberal Arts and Law to CTO

A computer kid who studied literature

Lucas Hendrich: One thing I love about this podcast — and I think it reflects our generation — is that guests come from such diverse backgrounds. I don't think anyone could model "the" CTO background. When my kids ask me what I do, how do I explain it? It's all part of managing multiple disciplines, and different organizations index more on each one. You come from a legal background.

John Higginson: And even before that, a liberal arts background.

Lucas Hendrich: I'm curious how that informs you. What perspective does it give you that might differ from your peers?

John Higginson: My educational background is highly unusual for what I've been doing for the last several years, so it's worth explaining.

When I was 11 or 12, I was a computer kid. It was the dawn of the PC age. My grade school had TRS-80s — anyone younger than about 45 will have to look that up — and then I had a Commodore 64. I taught myself to code and took classes in high school. I actually badgered my high school principal into letting me create my own independent study year for computer science.

When I got to college, a lot of it was repetition of things I thought — in hindsight, with a lot of hubris — that I already knew, so I didn't pursue it. I'd always been interested in storytelling and literature, so I focused on that for my bachelor's degree. But I never stopped coding. I had friends in college who were CS students, and I'd help them with their computer assignments, even though I wasn't one.

My first job out of college was as an engineer, and I worked on lots of different things. At one point I was even a network engineer, because in that era we did everything. After we sold Applied, I went to what's now UIC Law School — it was John Marshall back then — and focused my legal studies on privacy law.

Leadership is about people and communication

John Higginson: Stepping back, here's how that changes the way I think. What I've learned as a CTO is that leadership is far more about people and communication than about technology choices.

Don't get me wrong, I love to nerd out. If you want to talk about Rust, we'll talk about Rust. When containerization became more accessible on AWS and GCP, I was like, "Yes, let's talk about that. I want to see how it works." But getting a team of 30, 300, or 400 people to work together is about talking with them about where we're going and about the mission. And customers don't want to talk about technology. They want to talk about the outcome they need. You want a story — one that ends with them getting what they need to run their business.

So those communication skills, and an appreciation for all the different ways ideas come into software development, have been really helpful. Years ago, Steve Jobs talked about how the Mac wouldn't have had beautiful fonts if he hadn't taken a calligraphy course in college. I think about that a lot. A lot of the software I've helped build echoes things I studied that have nothing to do with computer science.

Why privacy law matters now

John Higginson: Then there's the legal background in privacy, which has obviously become huge. It's a key part of our obligations at Curriculum Associates, and of roles I've had before. We're entering a world where all our data is everywhere, and everyone is worried about malicious uses of it, and about hacking and phishing attacks to get it. Being able to think through not only the obligations, but also the broader industry thinking on privacy, is really helpful.

Lucas Hendrich: It's also interesting, from the outside looking in, how people talk about legal frameworks.

John Higginson: Yes.

Lucas Hendrich: Frameworks are the most interesting part of technology, in my view. When you define an architecture, it's incredibly important to do it in a way that persists over time and doesn't degrade — though it always does anyway. Using frameworks and patterns, and seeing how things become widely adopted in a scalable way, is very similar to creating a legal framework to address an issue in society, or privacy. I see a ton of parallels. Being a CTO is much more multidisciplinary; it has to be.

John Higginson: It has to be, because you're asked to do so many different things. You're the liaison between technology and the rest of the business. You sit with engineers and talk about how they're building things, then go to the board and explain what we're doing, why we're doing it that way, and how we can adopt something like AI. In broader product development, you need a sense of marketing and customer storytelling.

And overall, pick your metaphor — the fun uncle, the team dad, the soccer coach — you have to get people from very different backgrounds, doing different jobs, to understand each other and work together. It's like any sports team: different personalities, people driven by different things, and you have to smooth that out. That's why I say it's about so much more than technique.

I've done mentoring here in Chicago with a few groups, and I always tell newly minted engineering leaders: the decision you think is most important — "Should we build our services in Go? How hard should we lean into Lambdas?" — won't be as impactful, at least as a CTO, as these: Do I have the right people? Have we collectively decided on the right goals? Are we aligned on what we're delivering? The best technology in the world rests on people to deliver it. Without them, it won't happen.

Tech Debt, COBOL and Evolution

Good debt and bad debt

Lucas Hendrich: Absolutely. I'm curious about your thoughts on this too. I always think that if you arrived on this planet and your only source of information was LinkedIn, you'd think every company had already implemented agentic AI for everything.

John Higginson: Right.

Lucas Hendrich: In reality, many companies we work with and help are facing technical debt that goes back 20 or 30 years. How do you balance that? And I won't ask how much exists at your company…

John Higginson: We have none, Lucas.

Lucas Hendrich: Right — you're unique, you're all done.

John Higginson: Totally agentic. We can all go home now.

No, it exists in every company, especially the longer you've been around, and it exists for good reasons. I grew up in the insurance industry — the first industry I worked in. Although we were building modern software, we still rested on the industry's entire infrastructure, which then and now is largely COBOL. It has stayed COBOL because a lot of work went into building a scalable, long-lived, well-vetted, quality system. Much of that processing remains because it works.

With tech debt, or with embracing new things, it's always a balance. In all these years, there's never been a year where all we had to worry about was retiring tech debt. It never happens. You make targeted improvements or retirements. Every once in a while, you do have to kill off an entire system because it no longer works or doesn't scale. But usually it's evolutionary.

What people learn over time is that there's good tech debt and bad tech debt. Some things just work, and it isn't worth changing them — like the insurance COBOL example. Other things don't work, and you do want to change them and apply new technology. But in my experience it's always an evolution, never a big bang.

AI as a bridge to legacy code

Lucas Hendrich: I think technology is also saving us from the crisis of who knows COBOL. Just train the models.

John Higginson: I've done this. I've gone back and asked Claude or ChatGPT to write some app, or do some number sort, in COBOL. Or in Forth — Forth was actually the first language I learned after BASIC, and as far as I know almost nobody used it — and then C, which lots of people use. I'll think, "I can't remember how you'd do that. Give me this in Forth," and it spits it out.

That helps a new generation maintain a code base. I don't think anyone is seriously suggesting you can just ask AI to rewrite the millions of lines of COBOL out there and it'll all be fine. But for synthesizing and understanding it — "I want it to do this thing," or "there's this defect" — it can help.

Lucas Hendrich: It will future-proof us in a way, because that code will persist as long as nobody touches it.

John Higginson: And it will help bridge the generational divide. We're not really turning out CS graduates who know COBOL or some of these other languages, but AI will let people with good engineering skills appreciate and understand that code, because many of the concepts are the same.

Agile and the Experimentation Cycle

Lucas Hendrich: It's probably safe to say that for the past 20 years we've been following Agile methodologies for developing software. When you start running data experiments and incorporating them into your product stack, it changes the time cycle a bit. Have you found that? Is it disrupting your development cycle?

John Higginson: What we're still grappling with is how to create enough space for that experimentation. There's always timeframe pressure, no matter what you're working on: you need to get a certain number of things done. We run on a school calendar, so we have pretty hard dates for delivering things. But we also want to try things and learn, and things happen in development cycles.

So whether it's the hackathon I mentioned, or a spike in a sprint where we say, "We're going to look at this or do this data research," that's the approach we've taken. I don't know that it gets us all the way there. Like many companies, we're still figuring out how to balance experimentation with roadmap pressures. And Agile is also a framework, not a set of hard-and-fast rules. The question is how to adapt the way you use it to make room for this.

Lucas Hendrich: Maybe it's also about when decisions are made. I think a reasonable future scenario is that you go into planning — maybe for one sprint, maybe five or seven — and put the research up front to resolve some uncertainty. If you can resolve that uncertainty with 15 minutes of vibe coding, maybe you can do it alongside leadership and shrink the whole planning cycle. That's incredible. It's almost like looking at five different scenarios at once.

It's like creating a canvas. The other day I built something in a language I don't know, deployed it on my local machine with an interface, and could start building a product in about two hours. Not very sophisticated, but it did something another platform I could pay for does. Once you have that canvas, you iterate on improving it much faster. That research is what I find really fascinating.

John Higginson: It's an interesting idea — just being intentional about it.

Lucas Hendrich: Something it might otherwise take six months to reach.

John Higginson: Yes. "We're doing Agile," you know? I like that: being deliberate about carving out time to look at something and play around with it.

AI and Personalized Learning

"Tell it to me a different way"

Lucas Hendrich: This is a provocative question, so answer as much as you like, and please don't take it as a gotcha. I have three children who are, let's say, actively engaged with LLMs when doing their homework. Teachers probably don't want to hear this, but I think it's a fact: it's out there in the wild.

One thing I particularly like: a teacher might explain a concept really well for 80% of the kids, but 20% just don't get it. That creates frustration, and they lose interest. With an LLM, you can keep asking, "Tell it to me a different way. Tell it to me a different way."

John Higginson: Yes.

Lucas Hendrich: At no cost.

John Higginson: Yes.

Lucas Hendrich: How does that fit into your product vision — that kind of interactivity, letting the student really shape their own learning path?

Meeting students where they are

John Higginson: That's interesting. We agree that what we call personalized instruction is really important, and AI can play a role in it. But a couple of other things have to go into it for it to succeed.

First, it's really important to understand where a child is relative to grade level and relative to the curriculum's grade level. That's something off-the-shelf LLMs don't always do well. In school districts across the country, you can have a fifth grader who reads at a fifth-grade level but understands math at a second-grade level. That changes how you teach them, even in the same classroom. That's much of what our software has always done, pre-generative AI, and where generative AI plays a supporting role.

The second is helping teachers understand how to individualize instruction, which is what your kids are using AI for: "Give me other examples." Our vision is to help teachers teach in more and different ways and give them the tools to do that.

There's also the digital divide. It's still very real that students don't universally have the same level of access. With i-Ready, our software suite, we provide those tools to schools so they can use them in the classroom. Students may not have them at home, but they'll always have them in the classroom.

Through that, I definitely think generative AI — using LLMs to give examples that help kids understand things, or to help teachers — is important, and I can certainly see it flowing into future generations of education software. It's just really important that it's teacher-centric, so teachers can guide it. It also has to be aligned to where the student is, so we don't make mistakes about the level they're being taught at. And it should help every student grow, not just the students who already have great access.

Lucas Hendrich: That makes total sense. At the margin, you could imagine an extreme where students create their own bubbles of thought. One hallucination could take you down a path, and it's hard enough to figure out what's true anyway.

John Higginson: There are lots of examples. There used to be a podcast about things you learned in history class that are all wrong — none of it actually happened that way.

Lucas Hendrich: I find that especially with early iterations — less so now — the story is so credible. That's what LLMs are really good at, even when it's absolute nonsense.

Advice for the Next Generation

Curiosity, critical thinking and collaboration

Lucas Hendrich: Speaking of mentorship and the next generation: will we call them engineers? Product designers? This is an impossible question, but I've got to tell my kids something, because one of them starts college next year. Let me preface it: I feel there's a shift back toward critical thinking and away from pure technology, as technology — at least the core stack — becomes easier to implement. What are your thoughts?

John Higginson: I think that's true, and it's always been important. What will most benefit someone who's 22 and just out of college is curiosity and critical thinking. I'm willing to bet that you and I could write down the operating systems, programming languages, frameworks, and technology stack we used in our first year out of college, and none of them are what anyone is learning now. But we learned and adapted, because that was part of the job.

Understanding how to analyze a problem — whatever it is — and how to break it down is a critical skill in technology, whatever the role. So is being able to collaborate. When I was a kid in my room with my Commodore 64, I loved the Infocom text-based games. Some of those were developed largely by one person, writing both the game engine and the content. There's no game on Xbox written by one person, and probably no game on iOS or Android either.

So working with people is the other really important part. Going back to what we said earlier, I think in tech we sometimes over-index on technology choices rather than on building resilient people who can learn and adapt as technology changes.

As for what they'll be called, I have no idea. When I started, "programmer" was still the term of art. Then it was software developer, then software engineer, and now everyone's an engineer. Who knows — "wizard"? You can't predict it.

Toward a natural-language interface

Lucas Hendrich: I totally agree. When I started working in technology, there was a gap between what was taught in traditional computer science and the landscape of commercial products — the platforms you'd work with and the languages you'd write in. Now it's becoming a natural-language interface. Not across the board, but it's moving that way.

I wonder how that will change our language itself. LLMs reflect who we are and how we speak. We want LLMs to help us synthesize and be more concise. Maybe we'll become less verbose as humans. I don't know.

John Higginson: Maybe. Or maybe we'll win out over the machines and make them more verbose.

But I think that's true. Prompt engineering is yet another step in the abstraction away from low-level coding toward human language: telling the machine what you're thinking and having it do the hard work of assembling the code. You can definitely see that trend continuing.

Data as the Next Big Problem

John Higginson: It's been happening with access to data too. It used to be that any custom query required someone who knew SQL or had an engineering background. Not anymore. Look at the new version of Excel with Copilot: it can interrogate spreadsheets the way two people would talk.

Lucas Hendrich: So I agree that access to data will become a bigger problem to solve than building transactional applications. Wrangling data in different formats, integrating it, getting insights, and ensuring it's current and accurate will all be really critical.

John Higginson: It'll be interesting to see. Since the mid-'90s and the rise of the web, data has been as free as it's ever been. It'll be interesting to see whether that gets pulled back — whether companies start seeing themselves not just in whatever business they're in, but in the data business: "I have all this data that's really important to your large language model. What will you pay me for it?" We may see a class of companies built entirely around data accuracy, aggregation, and cleanliness, reselling it.

Lucas Hendrich: I agree. And the structure of that data matters too. It's already informing how we create internal documentation: you want it to be LLM-readable. In a broad sense, maybe SEO goes away, or becomes less keyword-focused and more structure-focused, since structure will determine how easily information can be retrieved through LLMs.

John Higginson: That's interesting, and it makes sense. You've already seen web traffic to Google decline as people use AI as a search tool. The idea that all the SEO black arts we've practiced for years in web development might just go away is fascinating.

New Frameworks: MCP and Agent-to-Agent

Lucas Hendrich: Are you excited about the new frameworks coming out? This is day-to-day stuff for me too: Model Context Protocol, A2A. Is that making its way into your work?

John Higginson: Yes, we've started talking about MCP. Everything's different, but it's still the same. This is where we say, "OK, now we have all these tools. How do we build middleware to make them work together?" That's a key part of agentic AI.

The frameworks now being developed for getting different LLMs to talk to one another, pass tasks back and forth, and make that work in your deployment environment — and how you test it — that's what's been most interesting to me over the last couple of months.

Lucas Hendrich: Funny that we say "two months" now. That cycle is so short. I think framework adoption will matter more over time. Right now people index heavily on benchmarks — this model just hit this benchmark or that one. For the general population, I don't think that means much. People don't really know how to interpret it. What does good look like?

John Higginson: It's changing so rapidly that it's hard to understand the context. One thing we've discussed internally: we built some RAG chatbots for internal use, but RAG may go away for some of those cases. When prompts can hold millions of tokens, why build a retrieval pipeline? For a chatbot like that, you can put everything in the prompt, and the prompt becomes a kind of living thing.

Lucas Hendrich: And that's what makes it so hard to build architecture. Two years ago, you might have built your own vector database with some Python in the middle. Now it's just a service, built in.

John Higginson: For people in data science, the disc is spinning fastest right now. Infrastructure is changing, but not as much. Software engineering is changing a bit faster. But in data science, things are moving really fast.

Lucas Hendrich: Absolutely. If you were following this 20 years ago and reading white papers — today the compute power makes everything possible.

From ELIZA to today

John Higginson: Remember ELIZA, the first computer "therapist"?

Lucas Hendrich: Yes, exactly. When you talked about one person writing those text games, I was thinking of The Oregon Trail — "You have died of dysentery."

John Higginson: I never made it to the end. But with ELIZA — people using LLMs for therapy is a real thing now. I don't think psychologists or psychiatrists recommend it, but it gets much closer to the advice a human therapist would give, whereas ELIZA was basically a game. Compute power and access to training data make that much more of a reality.

Lucas Hendrich: And as humans, we love to anthropomorphize things. We love to believe in the magic.

John Higginson: Yes.

Closing

Lucas Hendrich: John, it's been wonderful talking to you. Thanks for joining us, and I hope everyone out there enjoyed this conversation as much as we did.

John Higginson: Awesome. Thanks, Lucas.

Lucas Hendrich: Thank you. That's it for this episode. Join us next time for more real-world conversations between technology leaders. And if you liked what you heard, subscribe to CTO to CTO. See you soon.

‍