January 16, 2024

Models and Beauty Contests

Organisations of all shapes and sizes are racing to build new applications and services, based on latest generation AI models. In November, OpenAI claimed that 92% of Fortune 500 companies, the largest businesses in the US, are using their tools already. As with other Keynesian Beauty Contests there are benefits to picking the same option as everyone else, particularly access to resources, tooling and developers. Closer to home, many of the UK-based companies we work with are developing prototypes and full products, often using OpenAI’s GPT-4. Amongst everyone we speak to, though, a question lingers: How can I build (or buy) when the tech is changing so fast we might be out of date before the project is halfway finished? 

Whilst the media often treats ChatGPT as the totality of AI developments, the real transformation has come from proprietary systems which integrate the underlying models directly. And there are lots of them out there. At the very high end, OpenAI’s GPT-4 has topped general performance charts, but can’t be run on an organisation's own hardware, raising all manner of data & privacy challenges. Once unmatched, Google has now announced a rival product launching in the coming weeks. Beyond these two, there are hundreds of proprietary and Open Source models, many of them as good as the version of ChatGPT which made headlines a year ago. 

Nobody can tell you what the best choice will be twelve months from now, because the rate of change and potential for specialisation makes such predictions idle speculation. This leaves organisations with three choices: 

  • Wait until the dust settles 
  • Pick an option and hope that it wasn’t the wrong one down the line 
  • Design and build for flexibility, allowing models to change later. 

Unsurprisingly we recommend the latter. Whilst more costly in the short term, the flexibility is essential to insuring projects against sunk costs and redundancy a few months down the line. 

Reasons to go with a model agnostic, modular & flexible design

  1. Model capabilities and costs vary. Whilst the most general systems achieve performance on a wide variety of tasks, smaller specialist models will be cheaper to run and may provide as good, if not  better performance. That is, if you can integrate them as they emerge. 
  2. Open Source models are improving rapidly. The best models are currently pay-per-use and costs can quickly rack up. Open Source models can be hosted and run by anyone, which may prove significantly cheaper. Whilst they aren’t good enough yet to rival state of the art models, they are already sufficient for many tasks and improving quickly. 
  3. OpenAI is still a start up. The longest lasting impact of November’s leadership coup, in which 90% of employees threatened to resign, is the realisation that, unlike industry stalwarts such as Google or Microsoft, OpenAI is still a rapidly evolving company. Many organisations are wondering quite how much reliance to place on such a young company, not to mention one with a highly atypical board structure. Sometimes, slow and steady pays dividends in the long run. 
  4. The best is still to come. As technological capabilities shift and best practices emerge, avoiding getting stuck with technical debt (i.e. the negative impacts of previous tech decisions) is imperative. Imagine being forced to keep using fax machines as email emerged because all communications were designed to be paper based... It’s always preferable to design a system that can take advantage of technological advances readily. 

In the face of a changing world, refusing to act until you can be sure of the outcome is a decision in and of itself. Sometimes wait and see is the correct decision. Other times, it is an invitation to competitors to steal a march. Uncertainty needn’t mean committing to an action and hoping for the best. A range of strategies, including scenario planning and the use of other Futures & Foresight tools, allow organisations to make decisions now which maximise the chance of getting it right in the future. For technical builds, avoiding being locked in to any one choice is a necessary starting point. 

Paradigm Junction help organisations take decisions now, to prepare for uncertain futures and navigate technology-caused change. To find out more get in touch

Related posts

Computers aren't supposed to be able to do that

18 months ago he would have been right. To me, it had stopped being remarkable they now can.

Introduction to Futures and Foresight for Emerging Technologies

No one is sure about how the capabilities of AI will develop, let alone how business, government and society will respond to them. But this doesn’t mean that you should stand still. Tools exist for helping decision makers make smart choices in the face of uncertainty about the future, where traditional forecasts are liable to falter.

Apple Vision Pro - Seeing The World Through A New Lens

The Vision Pro isn’t only an AR/VR play from Apple - it’s a first bid to equip us with the tools we will want to be using in the future world of work.

Lightning Rounds - New Technologies That Could Change the Built Environment

We gave a presentation to a group of real estate investors. The question: What technology might be interesting if you are imagining how our commercial buildings will be radically different in a decade. Think internet shopping and fulfilment centres.