Artificial Intelligence (AI) continues to impact the way organisations operate and make decisions. The UK government recognises the immense potential of AI, evidenced by its allocation of £2.3 billion to AI initiatives since 2014. This commitment continued in the 2023 budget with an additional £1 billion earmarked for AI research.
As AI gains momentum in today's business landscape, the importance of strategic planning and a comprehensive understanding becomes paramount. Organisations have various factors to consider when beginning AI and machine learning projects, from defining the processes, people and data that fall within the scope, to choosing the methods and technology to implement.
AI analysis is important in guiding organisations through the journey of integrating AI into their operations. The initial phase of this journey — the discovery phase — holds significance in setting the course for successful AI implementation.
Here we outline three key considerations in detail:
- What type of data are you working with?
- Are you gaining insights from data or generating data?
- What model should you use?
1. Data Types: What type of data are you working with?
Are you working with financial data, user activity, volumes of text, images or something else? Is your data structured or unstructured?
For example, your organisation may want to analyse online customer behaviour to inform marketing strategies. The data involved would consist of structured data such as user demographics, browsing preferences and purchase records. In this scenario a model could be used to capture preferences in future behaviour.
Alternatively, if you want to visually identify stock, then your data will be images. Many image classifiers are pre-trained, where a model that has already been trained on a dataset. Using pre-trained models can allow organisations to begin quickly leveraging AI technology without having to invest in training data or building models from scratch.
Pre-trained models like those offered in Azure OpenAI Service and AWS Rekognition provide a strong foundation for these scenarios, with pre-trained models for image classification and object detection, specifically.
Also consider the data that you would receive from your solution; how will you evaluate the output? If you decide to use a language model to process and generate text (e.g. a chatbot), then it is important to consider the challenges that come with evaluating its responses. Large Language Models (LLMs) can be difficult to test because their outputs are subjective; how would you define an ideal response?
There are different strategies for evaluating generative language models and each one will likely be suited to a different use case. You may want to evaluate the truthfulness of the model’s responses (i.e. how accurate are its responses by real-world factual comparisons) or how grammatically correct its responses are. For translation solutions, you are more likely to measure metrics such as the Translation Edit Rate (TER), that is, how many edits must be made to get the generated output in line with the reference translation.
Language libraries like LangChain provide features for evaluating the responses according to relevance, accuracy, fluency and specificity, as well as giving you the flexibility to define your own criteria for evaluation via the LangChain API.
2. Data Outputs: Are you gaining insights from data or generating data?
Clarify whether your intended solution would process and analyse existing data or generate new content. For cases where you want to identify patterns or predict future behaviour, a model that processes data will be well-suited.
A WhatsApp travel chatbot for customers is an example of data processing. In this scenario, the chatbot is interacting with customers, processing the data provided by customers' inquiries and responses to provide relevant information and assistance related to travel.
Data generation solutions, on the other hand, are used to create data that did not previously exist. This new data could take the form of synthetic data that can then be used to train and test machine learning models, or even new creative content, such as text or images.
There is also the option of using a solution that is capable of both processing and generating data. This type of solution can be advantageous in cases where you want your model to learn from its experiences and the data that it is processing.
An e-commerce organisation may train a model on a large data set of user behaviour to learn about customers interests. Once this training is completed, the model could then be used to generate new recommendations for users.
3. Model Selection: What model should you use?
The core component at the centre of a machine learning project is a trained model, which in the simplest terms is a software program that, once given sufficient training data, can identify patterns and make predictions. Your final consideration, therefore, should be how you will access a model for your AI/ML project.
There are two popular choices when it comes to selecting a machine learning model approach. On one hand, AI cloud services offered by Azure, Google Cloud, and AWS provide pre-built, pre-trained models for tasks like sentiment analysis, image detection and anomaly detection, among others. These services empower organisations to quickly adopt AI technology by leveraging pre-built models, APIs, and infrastructure. This means that organisations can accelerate time-to-market and prototype validation, without the need for an extensive business case.
On the other hand, for those seeking more control over model development and training, using machine learning frameworks such as TensorFlow or PyTorch to build or define a model can be advantageous. These frameworks offer libraries and tools that facilitate model development.
Building a machine learning model encompasses the entire process, including selecting algorithms, defining structure and implementation. In contrast, defining a model typically involves utilising a library or framework with pre-defined architectures.
In our complete guide to AI and machine learning, we look at these two popular approaches for accessing a machine learning model in more detail.
The choice between these approaches depends on an organisation's use case, resources and desired level of model customisation. Building a model from scratch, especially for advanced uses cases such as deep learning, offers exceptional control, but does come with higher financial and computational requirements.
The role of Discovery in AI and Machine Learning Projects
AI discovery services play a vital role in guiding businesses through the intricate process of incorporating artificial intelligence into their operations. Ultimately, as organisations begin machine learning projects, it's crucial to assess how AI aligns with existing business processes. AI is not merely an add-on; it's a powerful technology that can reshape the way your organisation operates.
With expertise provided by an AI/machine learning consultant, your organisation can navigate the complexities of AI integration to ensure that the technology not only aligns with your processes but also empowers you to create even better customer experiences.
Audacia is a leading technology consultancy with experience delivering AI and machine learning solutions for leading organisations. We collaborate with organisations to leverage artificial intelligence and machine learning to improve services and make decisions better, faster and at scale. We provide end-to AI and machine learning services – from consultancy, process analysis and data valuation, to prototyping, development, QA and support.
To find out more about the AI services we offer, contact us today on 0113 543 1300 or at info@audacia.co.uk
