Interview For all the talk of the “agentic era” from AI vendors like OpenAI, Google, Anthropic, Microsoft, and just about everyone else in the space, corporate use of the technology is still tentative. Virgin Atlantic has been conducting flight tests of its website with an AI agent called Operator, and early results are promising, pointing the way toward how agents might actually be used to help customers book flights.
OpenAI announced Operator in January. It’s a research preview of an AI agent based on a model called Computer-Using Agent (CUA) – basically, Operator is the brand name through which CUA, the actual model, is made available to OpenAI Pro subscribers at operator.chatgpt.com.
CUA combines GPT-4o’s vision capability – the ability to “understand” images and “reason” about them – with training that covers graphic user interfaces. The model can look at a webpage and have some idea which elements represent buttons and how to apply click events. It’s essentially the LLM version of a browser automation framework like Selenium or Playwright.
Shortly after its debut, Virgin Atlantic began testing Operator to see how the AI agent navigates the booking flow of its website, whether it can retrieve useful information, and whether it can monitor website usage and provide business-relevant feedback on user interactions. The early results are showing promise in helping Virgin tune its website, according to digital engineering VP Neil Letchford, and his team is now exploring how Operator could be used in actual customer-facing interactions.
The Register spoke with Letchford to better understand the extent to which AI agents are actually being deployed.
Letchford: I work in the customer experience function at Virgin Atlantic. I head up the digital engineering group. So I’m VP of digital engineering. And my team is accountable for building all the customer-facing digital experiences for Virgin Atlantic. So everything that appears across the customer journey, that’s what my team is accountable for.
So when we partnered with OpenAI – and by the way, we’re not a massive team, right? We’re a relatively small team as a lot of software engineering teams are – but when we partnered with OpenAI and we spoke about Operator, we saw a real opportunity to scale and automate some of the functions we have within my team – internally looking.
But then also in doing that, we have started exploring how Operator or these Operator-type systems might impact customer experiences of the future. How our customers might be using them. So it’s kind of led us down two paths during this sort of period, which have been really interesting.
The Register: So what sort of things have you found?
Letchford: We’ve been using it for some of our internal use cases. So initially, when we think about our website, for example, we talk a lot about walking the customer journey and understanding the customer journey.
Our initial use cases were around how might we very basically utilize Operator to search and book for a flight, up to the point where maybe you have to put in your credit card details, right?
In doing that, we kind of figured out that, actually, Operator understood UX [user experience]. So it’s effectively understanding the UX and the structure of your website, which was quite interesting, right? Because it has to learn how to navigate and how to move through your customer journey to book a flight, for example.
And in doing that, we kind of extended the use case to act as a product owner. [In other words,] could Operator give us useful feedback on how our website might be improved or is performing?
So we gave it some tasks like some optimization tasks around text and things like that. We’d ask it to find that information maybe about New York or Miami. It would go away and then it would come back and come up with ways we could maybe improve our context or improve our thinking.
We then gave Operator – I don’t know if we’re allowed to do this – but we gave Operator access to an email, a Gmail account. And then what it would do is email us the results. It would put its results in the Google Docs and then email us back the results.
So it’s really interesting and valuable automation of these everyday tasks that you do need people to do. And you could kind of see a world where we could build and build and build upon this to kind of have these different personas of Operators carrying out different tasks across the website.
And then in doing that, it also opened up thoughts about how Operator understands our UX – the context structure or the navigational structure. But because Operator doesn’t have the context of what the web is – simple things like a hamburger menu, Operator in some cases doesn’t – it has to learn these things.
That opened up some really interesting conversations with our UX team about whether we’re being clear on how our website is structured and other UX questions – questions about date pickers and things like that [offered] really valuable insight really into how our user experience works.
The Register: So you’ve been using Operator for internal web optimization. Are there ways in which customers interact with the technology?
Letchford: Yeah, so we’re using it for those internal use cases at the moment, but we’re also part of the OpenAI Operator pilot where we’ve opened it up to basically people who use Operator as part of ChatGPT.
The Register: Among the customers who have done so, what have you learned about their experience?
Letchford: Usually, Operator is really good at understanding the tasks. Usually, Operator doesn’t really break out [unless] there’s security or personal information you need to enter. It will stop and hand control back to the customer to finish off their task.
The Register: And how do you evaluate Operator in terms of your organizational goals?
Letchford: I wouldn’t want to share any internal metrics, but obviously we have a bunch of KPIs [key performance indicators] we use in terms of performance and technical debt and things like that on how we internally manage our website. We basically treat it as another member of our team effectively and we’re going to be building on top of it from there.
And in terms of the external Operator piece, at the moment it’s very much a proof of concept that we’re working on OpenAI with. We’re testing it really to see what’s happening in the market because we believe that these experiences and agents having the ability to use the internet could very much become a key channel for us in the future.
So understanding the technology and how agents might interact with our site is really important for us. One of the key learnings for us there is not only this UX idea, but also we’re investigating, you know, how we might give Operator more context about our website. So we’re thinking about how we might provide Operator-specific context to the types of systems in the future so that they can become more efficient.
Because for someone like you, going to our website, you could quite easily find help content about baggage. It’s very natural. But we could accelerate those things for these Operator-type systems by giving it context of how to navigate our website and things like that.
The Register: Are the infrastructure and computational costs of running this about what you expected?
Letchford: No, for us, right, the costs are all borne effectively by OpenAI. The only cost for us is traffic to our website, which we have anyway.
The Register: Do you have a sense of how many different sorts of steps or systems you can sort of put into an automation flow before it becomes too complex to reason about?
Letchford: Well, really, the way we think about Operator is you give it a task and then you give it the tools to carry out its task. So we don’t really limit it. You could obviously say, “What are the five cheapest flights to New York?” and it would go off and just carry out that task. And it’s also really good at understanding when it can’t complete a task or when it needs more information, it will ask.
The Register: Have you had any incidents where you realized you had to guardrail something, where it went ahead and pulled data from, say, someone’s email without expecting it?
Letchford: No, it’s really strong around personal data and stuff like that.
The Register: What would you like to see Operator do that it doesn’t do yet?
Letchford: The ability to give it your test data and information that it can use to get through like our internal test environments would be really useful.
We’re also really interested in personas and accessibility. So for example, the ability to give Operator a persona – say, for example, you’re a family of four organizing a trip to Orlando.
Giving [the agent] a persona and seeing how that performs across our journeys and scoring it is really interesting for us. Because then we could understand how different cohorts of users go through our customer journey.
The other thing we’re really interested in that’s related to personas is accessibility – how might Operator help us improve the accessibility of our website by having the persona of maybe a partially sighted user or someone like that is really interesting for us. ®
