Large Graphical Models
A Crystal Ball into your Business' Future

Introduction

Generative AI became a household technology in 2023, with enterprises in verticals as diverse as retail, healthcare, and education beginning to embrace the technology. McKinsey released a recent report finding that three-quarters of respondents think generative AI will cause significant or disruptive change in their industry’s competitive landscape within the next three years.

The generative AI landscape has been dominated by Large Language Models (LLMs) like ChatGPT and Bard. And that’s for good reason – LLMs have demonstrated their value in supporting all sorts of exciting use cases, such as summarizing documents, generating content, and analyzing text-based data.  

There’s value for LLMs in just about every organization. However, they are limited in their ability to connect disparate enterprise data sources and generate accurate forecasts and scenario-based plans. Ikigai Labs identified that gap and introduced a new generative AI technology designed for predicting business outcomes leveraging tabular and time series data: Large Graphical Models (LGMs).

Use LGMs to predict what's coming next

The founders of Ikigai Labs created Large Graphical Models based on years of AI research and development at MIT. An LGM is a model that uses a graph to represent the conditional dependence structure between a set of random variables. LGMs are probabilistic in nature, meaning their purpose is to describe and explain the entire joint distribution between all variables of interest. From an end user’s perspective, LGMs produce interactive graphs that visualize the complex relationship between different data points. 

Leveraging time-series data, LGMs can predict how relationships may change in the future as the data changes. For example, an LGM could enable a retailer to precisely pinpoint how last month’s promotions impacted sales and how a new set of promotions might impact sales next month.   

Unlike LLMs, LGMs excel in modeling tabular data, which includes data typically organized in spreadsheets, tables, and databases. As previously mentioned, they prove particularly advantageous in analyzing time-series data, representing a sequence showcasing the evolution of a single variable over time. For instance, this might entail hourly sales data for a retail store. By approaching time-series data from the unique perspective of tabular data, LGMs possess the capability to predict vital business trends, including revenue, sales growth, and time to market. These insights serve as valuable compass points for enterprises, aiding in more informed decision-making processes.

LGMs possess the capability to predict vital business trends, including revenue, sales growth, and time to market.

How do Ikigai's LGMs differ from LLMs?

With LLMs rising fast in adoption, organizations might wonder what sets LGMs apart from them. Ikigai LGMs are foundationally different from LLMs. Each model has its own specific purpose. LLMs primarily work by analyzing text. At their core, text documents are linear (in the English language, text is written and read from left to right). LLMs employ very high-dimensional encoding and a multi-resolution structure to discern the relationship across different parts of a document. 

But tabular, time-series data is very different. This data is multi-dimensional – it can be analyzed in any direction. It doesn’t have to be read in a linear, left-to-right or top-to-bottom direction to make sense of it. Think of that table of hourly sales data. You could analyze it chronologically, reverse-chronologically, in order of largest dollar amount to smallest, etc., and draw accurate insights. And that’s a simple two column (variable) table. Add in a hundred more variables and it could be analyzed in exponentially more ways. 

Large Graphical Models can easily go beyond text’s linear structure and can capture relationships across many dimensions. Because of that, LGMs are much more suitable for modelling tabular data. 

Large Graphical Models can easily go beyond text’s linear structure and can capture relationships across many dimensions.

Another important distinction between the two is data volume. LLMs need to train on massive internet-scale data sets – often stretching over significant periods of time – to learn and deliver quality answers. In contrast, by combining the spatial and temporal structure of the data, LGMs can deliver precise insights even when analyzing small, sparse data sets. Aside from the obvious gains in resource efficiency, this helps organizations make accurate predictions even in situations with limited historical data, such as new product introductions, suppliers, or channels.  

LGMs and LLMs can work together. While LGMs extract patterns from tabular data, they can naturally interface with LLMs so that an end user or analyst can interact with LGMs by asking questions. They could feed it a prompt like “How will our new promotions impact sales next month?” and the model will produce an interactive graph quantifying the impact.

LGMs in action

Ikigai’s LGMs empower enterprises in many industries to forecast critical trends and make decisions accordingly. A few examples are:

• Supply chain (labor planning, sales and operations planning)
• Retail (demand forecasting, new product launch)
• Insurance (auditing, ratemaking)
• Financial services (compliance, KYC)
• Banking (customer entity matching, transaction reconciliation)
• Manufacturing (predictive maintenance, quality assurance) 

Let’s look closer at an example of how LGMs can be used to forecast labor needs. During the holidays, retailers in the US hire hundreds of thousands of seasonal workers to accommodate increased demand. For individual companies, it’s difficult to determine exactly how many workers they’ll need and where they’ll be needed in the supply chain. If a retailer hires too few workers, they won’t be able to fulfill orders. If they hire too many, they’ll waste money on redundant labor.  

LGMs can analyze a diverse web of variables, such as sales trends, social media mentions, advertising spending, and unemployment rates – with a specific focus on how these variables are changing over time – to create a detailed map of expected consumer behavior over the holiday period. With that insight, retailers can anticipate demand and make staffing decisions accordingly. Even for a massive business, these insights can be highly granular. For instance, maybe A, B, and C store locations will need to increase seasonal headcount by 50%, while locations X, Y, and Z only need 10% more workers.  

Solving the computational challenge

AI has become more cost efficient over the past few years, but organizations looking to deploy AI at scale still find it to be cost prohibitive. The issue is the price of computing resources. To operationalize large graphical models for broad applications, computationally efficient algorithms and a scalable infrastructure are required. Efficient computing infrastructure is essential to enable graphical models to learn from data and to then use those trained graphical models to make predictions and inferences.  

Over the past decade, the distributed, iterative (also called message-passing) algorithms have become the architecture of choice for computing at scale. To implement them at speed and scale in a cost-effective manner, Ikigai has developed an innovative computational architecture using PubSub infrastructure. In effect, it brings computation to data rather than the traditional approach of bringing data to compute. The result is that enterprises can perform both learning and inference at scale. An MIT thesis found that Ikigai was able to deliver over 13X faster scientific computation on a commodity laptop compared to modern infrastructure using 68 parallel machines.

Read Next: To (use) LLM or not to LLM: A Case-Study with Tabular Data

Ikigai LGMs can transform your business

Enterprises have vast amounts of valuable tabular, time-series data which often goes untapped because it's too difficult for spreadsheets and traditional AI models to make sense of. With Ikigai Labs’ Large Graphical Model technology, including aiMatch for data reconciliation, aiCast for forecasting, and aiPlan for what-if scenario planning, organizations can now unlock the value within enterprise data and transform the way they forecast and make key business decisions.

To learn more about the Ikigai platform, visit here.

To download the eBook as a PDF, click here.

‍