How Much Data Do You Need for Predictive Analytics in Sales?

 
How much data does Predictive Analytics require? Find out when ERP sales data is sufficient, what matters when it comes to data quality, and when Excel may be the better choice.

Wholesale companies often have more relevant sales data than they realise. Large numbers of customers, products, orders and individual transactions generate extensive data sets over the years. This information is usually already available in the ERP system.

Nevertheless, many companies assume that they first need to collect additional data, carry out extensive CRM data cleansing or purchase external market information before they can use Predictive Analytics. In many cases, however, this is not necessary.

For sales forecasting, cross-selling analysis, the identification of churn risks and data-driven pricing analysis, historical sales data from the ERP system is often the best starting point. It is reliable, closely linked to the actual business objective, available in large volumes and comparatively easy to access.

The key question is therefore: Which data can help us improve a specific sales decision at a reasonable cost?

Why ERP Sales Data Is Particularly Valuable

ERP systems usually contain a company’s actual business transactions. This includes customers, products, quantities, prices, order dates, discounts and revenue.

ERP data therefore does not merely reflect intentions or assessments. It documents what customers actually purchased, when they purchased it and under which conditions a transaction took place. This makes ERP data particularly suitable for Predictive Sales Analytics for several reasons.

ERP Data Is Reliable

Sales data is required for business-critical processes. Invoices, deliveries, orders and payments must be processed correctly. ERP data is therefore usually recorded more consistently and reliably than optional sales information.

Of course, ERP data can also contain errors or gaps. Compared with manually recorded notes or inconsistently maintained CRM fields, however, it often has a higher level of reliability.

ERP Data Is Closely Linked to the Business Objective

Actual purchasing behaviour is particularly informative for many sales predictions. A company that wants to identify customer churn should, for example, analyse whether order frequency, revenue, product selection or purchasing intervals are changing. A company looking for cross-selling opportunities needs information about which products similar customers purchase together. Pricing analysis requires historical prices, quantities, products and customer relationships.

This information is usually available in the ERP system. It is therefore closely related to the results that the company ultimately wants to predict.

ERP Data Is Available in Large Volumes

Wholesale companies often work with thousands of customers and tens of thousands of products. Even over a relatively short period, this generates large numbers of sales order items and transactions.

This volume of data offers considerable potential. Many relationships remain invisible in conventional reports or Excel analyses because they only become apparent when large numbers of customers, products and time periods are analysed together.

For a typical medium-sized or large wholesaler, insufficient data is therefore often not the main problem. Numerous customers, products, order items and several years of sales history quickly create very large transaction data sets.

ERP Data Is Comparatively Easy to Obtain

Another advantage is availability. The data is already present within the company and is continuously updated.
Additional data sources, by contrast, often need to be identified, purchased, combined and maintained over time. Before developing new data sources, companies should therefore assess how much value they can already obtain from the sales data available in their ERP system.

The Number of ERP Records Alone Is Not Decisive

One million sales order items can provide a highly valuable data basis. However, they may also be highly redundant, for example if they mainly come from a small number of major customers, a limited range of products or very similar ordering processes.

A model does not only need a large volume of data. It also needs enough different observations that are relevant to the specific objective. Thousands of almost identical orders may be less valuable for some applications than a smaller number of diverse transactions involving different customers, products and behavioural patterns.

The decisive factor is whether the data contains recurring and generalisable relationships. A model should not merely describe the past. It should identify patterns that can also be applied to previously unseen cases.

Too little data or data that is too homogeneous can contribute to what is known as overfitting. This means that a model may achieve good results with the training data but produce unreliable predictions when applied to new data.

Revenue Is Only a Rough Indicator

It may be tempting to determine whether Predictive Analytics is suitable for a company based on a specific level of revenue. However, a threshold such as EUR 1 million, EUR 5 million or EUR 10 million in annual revenue would not be technically reliable.

Revenue says little about how many different customers, products and transactions a company actually has. A company with annual revenue of EUR 3 million could, for example, serve 2,000 customers and process a large number of smaller orders. Another company with the same revenue might serve only twelve customers and handle a small number of major projects.

In the first case, there may be enough recurring patterns to use Predictive Analytics effectively. In the second case, an individual or rule-based approach would probably be more economical.

The number of products alone is not a sufficient indicator either. What matters is the combination of customers, products, transactions, time periods and relevant target events.

A company with only a few products may still benefit from churn prediction if it has a large number of customers with regular purchasing behaviour. For a cross-selling model, however, a limited product range would be a significant restriction because only a small number of product combinations could be analysed.

How Much Data Does Each Use Case Require?

The required data basis varies depending on the Predictive Analytics application. The question of data volume should therefore never be considered separately from the underlying business objective.

1. Cross-Selling

Cross-selling analyses require a sufficient number of recurring purchase combinations. If many customers regularly order different products, a model can identify which items are frequently purchased together and which product combinations are missing for comparable customers.

If there are only a small number of customers, a limited number of products or mainly one-off project transactions, it is difficult to identify reliable relationships between products and customer groups.

A large number of transaction records may therefore still be insufficient if those records only represent a small number of different purchasing patterns.

2. Customer Churn

Predicting churn risk requires a sufficient number of customers with recurring purchasing behaviour. There must also be enough cases that meet a commercially meaningful definition of churn.

Churn does not necessarily have to be stored as a dedicated field in the ERP system. It can, for example, be defined as a customer not placing an order for significantly longer than usual or substantially reducing their purchasing volume.

However, if only a very small number of customers have churned, the model may be unable to learn which behavioural changes typically occur before churn. Ten churned customers among several thousand active customers may be too few to identify reliable patterns, despite a large number of transaction records.

3. Pricing Analysis

Data-driven pricing analyses require comparable transactions. A model can, for example, examine how prices differ between similar customers, products, quantities and time periods. This can help identify anomalies and potential pricing opportunities.
However, if products are sold only rarely or if almost every order is individually configured, there may not be enough comparable cases. High revenue alone does not automatically guarantee that sufficient comparable pricing data is available.

4. Sales Forecasting

Sales forecasting requires sufficiently long and reasonably regular time series. A large number of transaction records within a few months does not automatically replace several years of sales history. Seasonal patterns, long-term changes and recurring peaks in demand can only be identified if the period under consideration is sufficiently long.

For products with highly irregular demand or extremely infrequent sales, simple planning rules or experience-based estimates may sometimes be more reliable.

Predictive Analytics Identifies More Than Simple Correlations

Predictive Analytics is often described in simplified terms as a method for finding correlations in data. While this explanation points in the right direction, it does not tell the whole story. Predictive Analytics models learn recurring and generalisable relationships between different features and a target variable. These relationships may be linear or non-linear. They may also depend on the interaction of several variables.

In customer churn analysis, for example, declining revenue may not be the only relevant indicator. The risk may only increase significantly if order frequency also declines, important product groups are no longer purchased and purchasing intervals become longer.

Such complex patterns are difficult to capture completely using simple individual rules. However, the decisive point is that an identified pattern must also work when applied to new data. The model’s performance must therefore be tested using data that was not already used during training.

Not Every Available Piece of Information Is Automatically Useful

In addition to ERP data, many companies have CRM data, call notes, emails, visit reports, market information and competitor data. Such information can supplement a prediction. However, the fact that it is available does not automatically mean that it can be used economically.

Predictive Analytics requires data that is as standardised and consistently available as possible. Structured CRM fields can therefore be useful. Examples include consistently maintained industry classifications, customer segments, sales regions and documented sales opportunities.

Unstructured information such as free-text meeting notes or individually written comments is more difficult to use. This data must first be prepared, standardised and placed into a usable context. Inconsistent wording, missing entries and subjective assessments can further increase the effort involved.

CRM systems are also not maintained consistently in many companies. Some sales employees document their activities in great detail, while others do so only occasionally. This can result in data gaps and bias.

This does not mean that CRM data is generally unsuitable. However, companies should realistically assess how reliable, complete and standardised the available data is.

External Data Must Also Provide Measurable Additional Value

The same applies to external data sources. Market information, industry data, company information and competitor data can provide additional context. However, they also create additional work and may involve ongoing costs.

Consistency is often a particular concern with competitor data. Extensive information may be available about certain competitors, markets or customer groups, while little or no comparable data is available for others.

A data source that only provides information for individual cases may create an uneven data basis. External data can also become outdated quickly or may need to be purchased again on a regular basis.

The question should therefore not simply be whether external data is available. The decisive issue is whether it improves the quality of the predictions enough to justify the additional costs and maintenance effort.

Costs and Benefits Must Be Considered Together

The selection of data sources should always include a cost-benefit analysis. How much time does it take to clean CRM data and maintain it consistently over time? How difficult is it to obtain external information? Is this data available for all relevant customers and products? How frequently is it updated? Most importantly, how much does including it actually improve the prediction?

An additional data source is not automatically valuable simply because it theoretically provides more information. If an expensive or labour-intensive data source only improves prediction quality slightly, its economic value may be limited. In such cases, it is often more practical to begin with the available ERP data and improve the results gradually.

Predictive Analytics should create a positive business case. The most technically extensive solution is not necessarily the most economically sensible one.

When Are Rules or Excel Sufficient?

Predictive Analytics is not automatically the best solution for every company and every use case.
Rule-based approaches may be more appropriate when there are only a small number of customers, products or relevant business cases. The same applies when decisions are based on a few transparent criteria, relevant events occur very rarely or there are hardly any recurring purchasing patterns.

A strongly project-based business model is a typical example. If a company sells only a small number of large industrial systems each year to a limited number of long-standing customers, sales expertise, fixed thresholds and structured Excel analyses may be more informative than a custom-trained predictive model. In such a case, the business processes are manageable. The employees responsible often know the customers personally and can assess individual circumstances directly. A complex model may only provide limited additional value.

However, fixed rules quickly reach their limits when a company has thousands of customers, tens of thousands of products and millions of possible customer-product relationships. Attempting to represent all relevant relationships manually in Excel would require considerable effort and would be difficult to keep up to date.
The strength of Predictive Analytics is therefore particularly apparent in situations where large numbers of complex relationships need to be prioritised.

At What Point Does Predictive Analytics Become Worthwhile?

There is no reliable general minimum amount of data. Statements such as “from 10,000 transactions” or “from EUR 5 million in revenue” would be too simplistic. The amount of data required depends on the specific use case, the complexity of the model, the number of features analysed, the frequency of relevant events and the diversity of the observations.

Instead of using a fixed threshold, companies should consider several questions. Is there a clearly defined business objective? Are there enough different customers, products or comparable cases? Does the sales history cover a sufficiently long period? Are there enough examples of the event that needs to be predicted? Does the model perform better with new, chronologically separated data than a simple rule or baseline? Is the additional economic value greater than the implementation and operating costs?

The suitability of the data can best be evaluated empirically. A model is trained using part of the data and then tested using data that has not previously been used. Learning curves can also show whether additional training data continues to improve prediction quality or whether performance has already reached a plateau. If performance increases significantly with more data, a larger data basis may be helpful. If performance changes very little, the limiting factor may instead be data quality, the selected features or the model itself.

 
CALCULATE NOW THE ROI OF QYMATIX PREDICTIVE SALES SOFTWARE
 

Conclusion: For Wholesalers, Data Volume Is Often Not the Main Problem

The general statement that companies only need a small amount of data for Predictive Analytics is not technically convincing. At the same time, there is no universal minimum amount of data.

The decisive factors are the specific use case, the diversity of the observations, the length of the sales history and the number of relevant target events.

For typical medium-sized and large wholesale companies, insufficient data is often not the central challenge. Numerous customers, products and order items mean that these companies usually have extensive transaction data in their ERP systems.

This data is particularly valuable for many sales predictions because it is reliable, structured and closely linked to actual purchasing behaviour.

However, the number of data records alone is not a sufficient measure. If a company has only a small number of customers, products or recurring business cases, even high revenue or a large number of similar transaction records may provide too little diversity for a reliable predictive model.

In such situations, rule-based systems, experience and Excel analyses may be more economical. Predictive Analytics is particularly valuable when the number of customers, products, transactions and possible relationships becomes too large for manual analysis. CRM data and external information can supplement the existing data basis. However, their value should always be compared with the costs of obtaining, cleaning, standardising and maintaining them.

The pragmatic approach is therefore to start with the existing ERP data, assess its suitability for the specific use case and only integrate additional data sources if they demonstrably improve both the predictions and the economic value.

 
CONTACT US TODAY FOR A LIVE DEMO