Setting the right product prices is crucial for companies and is part of their marketing mix and image. For instance, deviations from "optimal" sales prices can lead to considerable losses in revenue and margin. However, a huge amount of data affect the "optimal" price and the pricing process requires extensive manual resources.
Advanced algorithms like machine learning might have the potential to overcome the aforementioned challenges with almost no manual interactions. Pricing algorithms constantly automate and optimize pricing decisions based on the available data. Besides positive one-time effects of price optimizations, algorithmic pricing enables companies to implement new pricing strategies like dynamic pricing, price personalization, and markdown pricing.
This master thesis combines the results of a literature review and expert interviews to solve three questions: What is the research gap between the current state of the literature and business practice regarding the use of advanced algorithms based on big data for algorithmic pricing? What progress and insights have companies made in using algorithmic pricing? And how can algorithmic pricing be enhanced for future application?
The master thesis starts by explaining the basic concepts of algorithmic pricing and relevant technologies. Therefore, the results and takeaways are useful for business managers without prior experience in this area. This master thesis then provides corporate decision makers with recommendations on what to consider for new pricing algorithms and on opportunities for future development of existing pricing algorithms.
Table of content
Table of figures
Table of tables
Table of abbreviations
1 Introduction
1.1 Relevance and Problem
1.2 Research Questions and Approach
2 Fundamentals of Algorithmic Pricing
2.1 Overview and Definition of Algorithmic Pricing
2.2 Algorithmic Pricing Advancement and Automation Model
2.3 Big Data and Related Techniques in Algorithmic Pricing
3 Literature Review on the Use of Algorithmic Pricing
3.1 Aim and Methodology
3.2 Results
3.3 Discussion
4 Expert Interviews on the Use of Algorithmic Pricing
4.1 Aim and Methodology
4.2 Results
4.3 Discussion
5 Comparison and Synthesis of Literature Review and Expert Interviews
6 Conclusion
Appendix A – The Level of Automation Taxonomy by Save and Feuerberg
Appendix B – Overview of Results of the Literature Review
Appendix C – Questionnaires for Expert Interviews
C.1 Applying companies
C.2 Pricing Consultancies
C.3 Solution Providers
Appendix D – Overview of Results of the Expert Interviews
References
Table of figures
Figure 1: Exemplary Positions in the Algorithmic Pricing Advancement and Automation Model
Figure 2: Assessment of the Pricing Algorithms from the Literature Review and Expert Interviews
Table of tables
Table 1: Number of Results of the Literature Review
Table of abbreviations
B2B Business-to-Business
B2C Business-to-Consumer
CEO Chief Executive Officer
CRM Customer Relationship Management
ed. edition
eds. editors
e.g. for example (Latin: exempli gratia)
ERP Enterprise Resource Planning
et al. et alii
i.e. that is (Latin: id est)
IT Information Technology
n.d. no date
p. page
pp. pages
vol. volume
1 Introduction
1.1 Relevance and Problem
Setting the right product prices is crucial for companies and is part of their marketing mix and image. Deviations downwards or upwards from the right price can lead to missed profits (Mohammed, 2012). Yet, the median frequency for product price changes is 7.2 months (Klenow & Kryvtsov, 2008). However, many changes may take place in the meantime, making an earlier price adjustment necessary. Therefore, software algorithms “are increasingly being adopted by firms to price their goods and services, and this tendency is likely to continue” (Calvano, Calzolari, Denicolo, & Pastorello, 2019, p. 2). Pricing algorithms constantly automate and optimize pricing decisions based on the available data.
Dynamic pricing is an illustrative example of the valuable application of algorithmic pricing. Dynamic pricing is a method by which sales prices are frequently changed in order to achieve and maintain an optimal price for a product or service (den Boer, 2015). It enables companies to react to changes of all kinds in the environment of a product while pursuing certain goals like profit or revenue maximization. This is essential as a price increase of 1% at the same sales volume can lead to an 8.7% increase in profits (Baker, Kiewell, & Winkler, 2014). Dynamic pricing, therefore, plays an important role in the competitive retail industry as the right pricing decision offers significant profit improvements for companies.
For companies without pricing algorithms, it might be challenging to maintain the right price for each product over time. The required manual effort would likely diminish the profit improvements and it would take a significant amount of time to adjust all prices. In times of e-commerce where companies like Amazon use dynamic pricing systems that check competitor prices every 15 seconds (Akter & Wamba, 2016), speed and flexibility can be crucial to stay competitive.
Additionally, competitor prices are not the only factor that should be taken into account when adjusting prices. In fact, Wal-Mart filed a patent for a dynamic pricing algorithm in 2015 stating that “in reality, there are numerous factors that might affect pricing in real-time” (U.S. Patent 10,068,241 B2, 2018, background section). The ever-increasing volume of data for these factors and the increasing capacity to analyze big data sets offer the potential to make highly accurate pricing decisions: “Exploiting the knowledge contained in the data and applying this to dynamic pricing policies may provide key competitive advantages” (den Boer, 2015, p. 2).
Besides dynamic pricing, algorithmic pricing can also be used for scenarios like regular price optimization, initial pricing, price personalization, and markdown pricing. Price personalization using advanced algorithms and big data, for example, can consider the individual willingness to pay much more accurately than in the past in order to maximize profits (Bar-Gill, 2019).
In sum, retail companies face the challenge to keep prices at an optimal level or personalize prices based on big data to stay competitive. Advanced pricing algorithms might have the potential to overcome the aforementioned challenge with almost no manual interactions. These algorithms can permanently optimize prices for each product considering a variety of data sources in order to reach the company’s pricing goal (Rana & Oliveira, 2014). Even beyond pricing, the importance of advanced algorithms for marketing strategies is expected to increase further in the future (Davenport, Guha, Grewal, & Bressgott, 2020).
1.2 Research Questions and Approach
The aim of this master thesis is to critically reflect on pricing algorithms based on big data. This contributes to the research needed on advanced algorithms in various retail decisions as suggested by Shankar (2018).
In business practice, there are pricing algorithms with different degrees of automation and technological approaches. Fast technological advancement facilitates further improvements of these pricing algorithms. Part of the critical reflection is the assessment of whether research has been conducted on the latest pricing algorithms which are used by retail companies. Therefore, the first research question is as follows:
1. What is the research gap between the current state of the literature and business practice regarding the use of advanced algorithms based on big data for algorithmic pricing?
The answer to the first research question allows researchers to focus their attention either on the possible research gap or on the further development of innovative pricing algorithms in line with business needs. Furthermore, the comparison of results and explanation of differences between literature and business practice might yield insights about pricing algorithms for both researchers and business managers.
Another part of the critical reflection is to understand why and how companies started with algorithmic pricing and what they have learned until now. The second research question is therefore:
2. What progress and insights have companies made in using algorithmic pricing?
The answer to the second research question is beneficial for future field experiments by researchers and for future implementations of algorithmic pricing in business practice. Additionally, business managers can compare their own progress regarding algorithmic pricing and learn about different approaches.
As the first two research questions cover the past and present of algorithmic pricing, the question remains how the future vision of algorithmic pricing might look like. Further advancements of the algorithms could lead to even better pricing decisions and allow for a higher degree of automation. Both outcomes are expected to increase the profitability of such algorithms. Therefore, the third research question is as follows:
3. How can algorithmic pricing be enhanced for future application?
The answer to the third research question directs future development efforts of researchers and companies regarding algorithmic pricing.
To answer these research questions, the following approach is pursued. In chapter 2, the fundamentals of algorithmic pricing, big data analytics, and selected related techniques are briefly explained. This chapter creates a common understanding of the most relevant concepts of this master thesis. Additionally, a model for the assessment of the advancement and automation of pricing algorithms is proposed. In chapter 3, field experiments of researchers for the use of advanced pricing algorithms are examined in a literature review. This chapter presents the current state of literature. In chapter 4, the results of expert interviews on the use of pricing algorithms in business practice are presented and discussed. Based on these findings, the second research question regarding the progress and insights of companies is answered. In chapter 5, the results from the literature review and the expert interviews are compared to find similarities and explanations for differences. This comparison allows answering the first research question regarding the research gap. The synthesis of the results regarding the future vision of algorithmic pricing leads to the answer to the third research question. In chapter 6, the main findings and takeaways are summarized and limitations are stated.
The scope of this master thesis does not include deep dives in how to obtain access to data, which information technology (IT) infrastructure is needed, how to develop a demand model with an algorithm or whether an algorithm complies with ethical norms. Rather, this master thesis uses the insights and takeaways from other researchers and practitioners to critically reflect on algorithmic pricing on a decision-maker level. The target groups of this thesis are researchers and corporate decision-makers. Furthermore, the literature review does only cover the years from 2015 to 2020 as the current state of literature is sought. Additionally, the results from the expert interviews are not intended to claim completeness regarding pricing algorithms in business practice. Rather, expert interviews should give exemplary and deep insights from the expert’s point of view.
The focus of this master thesis is mainly but not exclusively on companies whose pricing decisions are automatable and applicable to many customers (e.g. Business-to-Consumer (B2C) retailers and Business-to-Business (B2B) companies with a high number of small deals). Companies with B2B key account sales structures, in contrast, are only covered partly. A limitation of the product range will not be given. Generally, the insights generated in this master thesis might also be relevant to other sectors.
2 Fundamentals of Algorithmic Pricing
Advanced algorithms and big data analytics are regularly part of technology top-lists like the “Top 10 Strategic Technology Trends for 2019” report by Gartner (2018). Consequently, many different wordings and understandings tend to emerge for advanced algorithms and big data analytics. Therefore, this chapter creates a common understanding of the most important terms and concepts in this thesis.
2.1 Overview and Definition of Algorithmic Pricing
Algorithms have been used in mathematics for centuries. An algorithm is basically a set of rules and well-defined operation steps to solve a specific type of problem (Knuth, 1997). A common basic computer algorithm, for example, takes over the task of sorting data in a predefined way. Nowadays, almost every computer-related technology works with algorithms (Cormen, Leiserson, Rivest, & Stein, 2001). Advanced algorithms are also the foundation for artificial intelligence and its sub-disciplines like machine learning.
One field of application for algorithms is pricing; then also referred to as “algorithmic pricing”. Algorithmic pricing means that companies use computer algorithms to determine selling prices (Chen, Mislove, & Wilson, 2016). Dynamic pricing, for example, often relies on pricing algorithms due to the need of automation for frequent price changes.
Pricing algorithms differ regarding their level of advancement and degree of automation. While the first pricing algorithms carried out rule-based computation steps, recent technological advancements have led to learning algorithms that are “much more autonomous than their precursors” (Calvano et al., 2019, p. 2).
Simple pricing algorithms follow clearly defined rules, such as monitoring competitor prices to copy the lowest price. These algorithms only need connections to specific real-time data like competitor prices (Competition & Markets Authority, 2018).
Medium-advanced pricing algorithms such as regression analysis algorithms derive their recommendations based on historical data. The developer predefines the pricing model with its parameters, weights, and data sources. Medium-advanced algorithms then use past observations to adjust the pricing model to reach a defined goal (Competition & Markets Authority, 2018).
Highly advanced pricing algorithms make use of historical data sets as well as real-time data to achieve the best possible price recommendation. With self-learning techniques and big data, these algorithms can build pricing models for individual products themselves (Calvano et al., 2019). Companies only have to define a pricing goal like profit maximization or market penetration.
The notion of “advanced algorithms” in this thesis stands for algorithms with a medium to high level of advancement. Such algorithms require computational power to be solved for a given data set within a reasonable amount of time.1
The price recommendations of pricing algorithms can be integrated into business processes according to different levels of automation. Save and Feuerberg (2012) proposed an automation taxonomy regarding interactions between humans and computer systems. It builds on the framework of Parasuraman, Sheridan, and Wickens (2000). The ‘Level of Automation Taxonomy’ defines automation levels ranging from none to full automation for four tasks: (A) information acquisition, (B) information analysis, (C) decision and action selection, and (D) action implementation (Save & Feuerberg, 2012). The combination A0-B0-C0-D0 describes the fulfillment of these tasks without any involvement of computers or tools while the combination A5-B5-C6-D8 stands for the highest levels of automation and autonomy.2 The second level in each task is the first to include a digital system or software and therefore marks the lower end of automation for computer algorithms.
2.2 Algorithmic Pricing Advancement and Automation Model
As no model could be found to compare the level of advancement and the degree of automation for pricing algorithms, the ‘Algorithmic Pricing Advancement and Automation Model’ in figure 1 is proposed. This two-dimensional classification of the algorithms visualizes the scope and organizational integration of a pricing algorithm in comparison to other algorithms.
Abbildung in dieser Leseprobe nicht enthalten
Figure 1: Exemplary Positions in the Algorithmic Pricing Advancement and Automation Model. (Source: Own illustration)
The model is composed of the level of advancement of the algorithm on the y-axis and the degree of automation on the x-axis. The level of advancement ranges from low to high.3 The degree of automation also ranges from low to high and is based on combinations from the ‘Level of Automation Taxonomy’ by Save and Feuerberg (2012).
In figure 1, six exemplary positions for pricing algorithms are shown. Different positions are possible depending on the level of advancement and degree of automation of an algorithm. Positions in the shaded area are not possible as algorithms become able to adjust or develop their pricing models with increasing advancement. This conflicts with a low degree of automation where systems only support humans in individual activities and on direct request.
In the following, the six exemplary positions for pricing algorithms in the proposed model are described:
- AP1 is a simple pricing algorithm with a low degree of automation. The algorithm carries out rule-based steps such as monitoring competitor prices to copy the lowest price. The combination of automation is A2-B2-C2-D2. On the request of a (human) pricing specialist, the algorithm collects information from predefined data sources and combines and sorts the information. The algorithm then gives a price recommendation based on predefined rulesets. The pricing specialist is able to accept, change or reject the price recommendation. The algorithm then helps to update the product price in the company’s systems on the initiative and under control of the pricing specialist.
- AP2 is a medium-advanced algorithm with a medium degree of automation. The algorithm is able to adjust the predefined pricing model to generate better price recommendations. Data sources must be specified. The combination of automation is A4-B4-C3-D4. The algorithm collects data from specified data sources and identifies the most important data based on predefined criteria. Then, the data are analyzed according to the algorithm’s design. If necessary, a pricing specialist is notified for further guidance. The algorithm presents a price recommendation to the pricing specialist. The pricing specialist can accept this recommendation or request another recommendation. After the selection by the pricing specialist, the algorithm communicates automatically with the connected systems to adjust the price. The pricing specialist can interrupt the process.
- AP3 is an advanced algorithm with a medium degree of automation. The algorithm applies learning techniques on big historical data sets to derive an appropriate pricing model for every individual product. Real-time data are used to adjust the pricing model. The combination of automation is A5-B5-C3-D4. The algorithm collects data from available data sources and identifies the most important data. The criteria for the prioritization of the data are not known. The algorithm analyzes the data and notifies the pricing specialist if the results are outside the usual range. The algorithm presents a price recommendation to the pricing specialist. The pricing specialist can accept this recommendation or request another recommendation. After the selection by the pricing specialist, the algorithm communicates automatically with the connected systems to adjust the price. The pricing specialist can interrupt the process.
- AP4 is a simple algorithm with a high degree of automation. The algorithm carries out rule-based steps such as monitoring competitor prices to copy the lowest price. The combination of automation is A4-B4-C6-D8. The algorithm collects data from specified data sources and identifies the most important data based on predefined criteria. The data are analyzed and processed according to the pricing goal. The algorithm decides on the new price without notifying the pricing specialist. The algorithm then communicates automatically with the connected systems to adjust the price. The pricing specialist can only intervene once the price change has been implemented.
- AP5 is a medium-advanced algorithm with a high degree of automation. The algorithm is able to adjust the predefined pricing model to generate better price recommendations. Data sources must be specified. The combination of automation is A4-B5-C6-D8. The algorithm collects data from specified data sources and identifies the most important data based on predefined criteria. The data are analyzed according to the algorithm’s design. The algorithm is able to optimize the criteria for data prioritization and its analysis design based on past results. The algorithm decides autonomously on the new price without notifying the pricing specialist. The algorithm then communicates automatically with the connected systems to adjust the price. The pricing specialist can only intervene once the price change has been implemented.
- AP6 is an advanced algorithm with a high degree of automation. The algorithm applies learning techniques on big historical data sets to derive an appropriate pricing model for each individual product. Real-time data are used to adjust the pricing model. The combination of automation is A5-B5-C6-D8. The algorithm collects data from available data sources and identifies the most important data. The criteria for the prioritization of the data are not known. The algorithm analyzes the data and notifies the pricing specialist if the results are outside the usual range. The algorithm decides autonomously on the new price without notifying the pricing specialist. The algorithm then communicates automatically with the connected systems to adjust the price. The pricing specialist can only intervene once the price change has been implemented.
2.3 Big Data and Related Techniques in Algorithmic Pricing
A necessary input factor for pricing algorithms is data. At the same time, the speed of data generation and collection is rising steadily. This leads to the concept of “big data” which is often specified using four “V”s to differentiate big data from traditional data (Akter & Wamba, 2016): The “volume” of big data emphasizes the vast amount of data that can hardly be analyzed using traditional methods. The “velocity” emphasizes the high speed of data generation and collection as well as the possibility to analyze these data in real-time. The “variety” emphasizes that lots of different data sources and types of data are available. The “veracity” emphasizes the need for reliability and quality of the data sources to achieve meaningful analyses. Due to the volume of big data, the data cannot be verified manually in a reasonable amount of time.
More “V”s to describe big data keep emerging like “variability”, “visualization”, and “value” (Sivarajah, Kamal, Irani, & Weerakkody, 2017). The variety of data includes structured and unstructured data. Structured data can be stored in a tabular layout. Structured data are often referred to as traditional data as this has been the first type of data that has been stored and analyzed by companies. Unstructured data cannot be stored in a tabular layout in its raw format and require advanced techniques for analysis. Examples of unstructured data are text, images, videos, and audio (Erevelles, Fukawa, & Swayne, 2016). Due to the characteristics of big data, special processes and technologies are required to store, clean, and process the different types of data (Gandomi & Haider, 2015).
A simplified profit function can be used to derive some of the most important data for pricing decisions. It can be stated as in equation 1 (Eilon & Cosmetatos, 1979):
Abbildung in dieser Leseprobe nicht enthalten
While the cost is usually known and the price is to be selected, sales correspond to demand for a company’s product given that there is sufficient supply. A main challenge of pricing is, thus, to estimate a demand model under consideration of price elasticities. Based on this demand model, the optimal price can be identified. Due to the high number of parameters and their unknown influence on demand, companies can use big data for algorithmic pricing to estimate the most accurate demand model as possible. For example, a patented dynamic pricing algorithm of Wal-Mart adjusts a product price based on data including store location, social media activity regarding a specific product, number of product views on their website, and competitor prices (U.S. Patent 10,068,241 B2, 2018). The American Major League Baseball, as another example, uses weather data, performance trends of teams, and the likelihood for a record-setting event to dynamically adjust ticket prices (Erevelles et al., 2016).
There is a variety of techniques and methods to derive valuable information and predictions out of big data. Before selecting a specific algorithm, it is advisable to compare and test different approaches as there is no “one-size-fits-all” method (Landset, Khoshgoftaar, Richter, & Hasanin, 2015). The selection of a specific approach is accompanied by a trade-off between performance, interpretability, and efficiency. In the following, selected relevant techniques and methods for algorithmic pricing are briefly described.
While there are numerous definitions for big data analytics, it is often understood as a collection of techniques, technologies, and processes to derive insights from big data (Mikalef, Pappas, Krogstie, & Giannakos, 2018). It can be used to “find hidden patterns in data” (Erevelles et al., 2016, p. 897) and helps to understand connections and relations in the data to derive actionable measures for businesses. Big data analytics thereby supports decision-making for businesses (Akter & Wamba, 2016). While patterns can also be found in traditional transaction data, the information value of analytics increases when applied to big data.
One technique to analyze the data in order to derive a demand estimate is statistical modeling. In statistical modeling, traditional statistical methods are applied manually to find a mathematical equation that approximates the real demand model. The mathematical equation is based on assumptions about the demand model (Dangeti, 2017). Archer (1980) argued to combine common statistical models like regression analysis with a qualitative method like the Delphi technique to build a demand model with assumptions as unbiased as possible.
In contrast to that, there is the field of artificial intelligence as another technique of big data analytics. Despite the multitude of definitions, artificial intelligence generally “refers to programs, algorithms, systems or machines that demonstrate intelligence” (Shankar, 2018, p. vi). The application of pricing algorithms from the field of artificial intelligence is promising as they are able to define demand models and pricing strategies with little to no manual influence and can adjust themselves to changes in influencing factors (Calvano et al., 2019).
Especially the developments over the last years in machine learning, as one sub-discipline of artificial intelligence, allow for a wide range of application possibilities for algorithmic pricing (Simchi-Levi, 2017). Machine learning algorithms learn from historical and real-time data and build a model to predict future events and outcomes (Landset et al., 2015). Common techniques of machine learning for algorithmic pricing and demand forecasting are regression tree ensembles (Simchi-Levi, 2017), k-means clustering (Cheung, Simchi-Levi, & Wang, 2017), and neural networks (AmalNick & Qorbanian, 2017).
The lack of interpretability of the outcomes of many artificial intelligence algorithms has led to the recent advent of explainable artificial intelligence. While the best performing algorithms are often only poorly interpretable, users need at least a partial or abstract explanation to understand and trust the outcomes (Gunning et al., 2019). The interpretability of advanced algorithms is considered essential for the initial investment decision by companies and for the acceptance of the employees (Lee & Shin, 2020). Consequently, the degree of interpretability can have a significant influence on the success of advanced algorithms in companies.
3 Literature Review on the Use of Algorithmic Pricing
3.1 Aim and Methodology
The aim of the literature review is to get an overview of the current state of the literature regarding the use of advanced algorithms for algorithmic pricing and the assessment of these algorithms. This overview serves as a basis to evaluate whether there is a research gap to the current state of business practice. Furthermore, the results of the literature review will be synthesized with the results of expert interviews to evaluate how algorithmic pricing can be enhanced for future application.
The literature review is based on the meta-ethnographic approach of Noblit and Hare (1988) for interpretive reviews. This approach acknowledges the importance of the context and a non-uniform use of terms and keywords in the selected literature. As there is a great variance in the definitions of important terms of algorithmic pricing and as there are numerous possibilities to approach and apply algorithmic pricing, the meta-ethnographic approach appears suitable.
The literature review is limited to field experiments by researchers with a publication date between 2015 and 2020. The reason for the timely boundary is the fast advancement of algorithms. Since there is a trade-off between the availability of literature and timeliness of the results, 2015 is considered as an appropriate lower boundary. The limitation to field experiments is due to the nature of advanced algorithms. Advanced algorithms and data analytics models have become too complex to analyze thoroughly and partially work in a “black box”. Therefore, these algorithms must be applied to data to assess their performance. Artificial tests under laboratory conditions, however, do not allow for a real assessment of the algorithms due to the risk of overfitting. Overfitting in the context of data science means that a data model is trained too narrowly to a specific training data set and cannot generalize to other deviating data sets. Algorithms must be exposed to changing data in real market environments to offer reliable insights regarding their performance (Provost & Fawcett, 2013).
The aspect of field experiments is therefore of high importance for the literature review and will be required as a keyword in the abstract of potential publications. Consequently, “field experiment” and “field study” are used as mandatory keywords for the abstract. Further synonyms like “field test”, “field trial”, and “real market experiment” have not yielded relevant results.
Further keywords for the literature search center around the topic of algorithmic pricing, variations of its term, and related techniques. These keywords must be included in the title of a potential publication. Keywords in this group are:
- Algorithmic pricing
- Pricing algorithm
- Machine learning + pricing
- Analytics + pricing
“Pricing” is used as a full-text keyword unless the title keyword does not already include this term.
Due to the frequent price changes that become possible through algorithms, algorithmic pricing is often referred to as dynamic pricing. “Dynamic pricing” is therefore used as an additional title keyword. All publications are screened for the use of relevant techniques to satisfy the research aim.
The literature review had been conducted in the e-library of HHL Leipzig Graduate School of Management and uses the EBSCOhost research platform. Smart search expanders had not been applied and publications had not been limited to HHL resources. The literature review was conducted in November 2019. The search terms have been observed for new publications until January 2020.
Under consideration of time boundaries from 2015 to 2020 and the abstract keywords “field experiment” and “field study”, table 1 shows the number of total results and relevant results for each title keyword. The total number of distinct results for all title keywords amounts to 24 whereas the number of relevant results adds up to 4.
Abbildung in dieser Leseprobe nicht enthalten
Table 1: Number of Results of the Literature Review. (Source: Own illustration)
The results will be analyzed according to five categories:
- Background (e.g. industry, company, type of business, time of the experiment)
- Level of solution advancement (e.g. aim, technique, and data sources)
- Degree of solution automation (e.g. automated tasks and human interaction)
- Assessment of algorithmic pricing solution (e.g. qualitative and quantitative)
- Outlook (e.g. research need, directions for development)
The background is needed to evaluate the comparability of the field experiments among each other and to the expert interviews. Especially the year in which the experiment was conducted is important to reflect potential technological differences. The level of solution advancement is needed to identify the specific algorithmic approach and the considered types of data. The degree of solution automation gives insights on how the interaction between the algorithms and employees is designed. Furthermore, the publications are analyzed regarding their assessment of the pricing algorithm. The assessment can consist of quantitative performance indicators or qualitative insights regarding risks and opportunities. The results of the outlook category might be used to develop a vision for the further development of algorithmic pricing solutions in chapter 5.
The solutions that are presented in the academic field experiments will also be compared to the pricing solutions from the expert interviews in chapter 5 to evaluate whether there is a research gap.
3.2 Results
The results of the literature review are presented in the following.4
In 2018, Fisher, Gallino, and Li published their article “Competition-Based Dynamic Pricing in Online Retailing: A Methodology Validated with Field Experiments” in the journal ‘Management Science’. The article was submitted to the journal on 9 January 2015. As there is no exact indication when the field experiment took place, the date of the field experiment is set at 2014 for the analysis.
Due to the high price transparency caused by the internet and price comparison platforms, Fisher et al. aimed at determining the best-response price to certain competitor price changes and availability changes. Therefore, the best-response price had to consider price elasticities, the degree to which consumers are aware of and compare competitor prices, and the selection of relevant competitors.
Fisher et al. conducted the field experiment with a Chinese online hypermarket. Based on the company specifications, it can be concluded that Yihaodian was the partnering Chinese online retailer. Yihaodian had been owned by Wal-Mart to a share of 51% when the article was submitted. Later in 2015, Wal-Mart acquired all outstanding shares of Yihaodian. At this time, Yihaodian offered about 8,000,000 products online (Wal-Mart Stores, 2015).
The observed product category was baby-feeding bottles. The main field experiment lasted for 5 weeks. The aim was to maximize the total revenue of the product category within certain margin boundaries under consideration of price restrictions from manufacturers.
First, Fisher et al. developed a consumer choice model to decide which price response to a competitor price change would convince a consumer. They enhanced a widely accepted consumer choice model framework with an identification strategy to estimate non-observable sales information for the four biggest competitors.
Second, Fisher et al. estimated unbiased own-price elasticities and cross-price elasticities through a first field experiment. Over a period of four weeks, Yihaodian’s 15 best-selling baby-feeding bottles were assigned random prices with certain upper and lower boundaries. The results were compared with different data sets to define the estimation parameter.
Third, Fisher et al. solved an optimization problem to derive best-response pricing reactions. Pricing recommendations changed when either product costs, Yihaodian’s or competitors’ product availability, or competitor prices changed.
Finally, they assessed the performance of the pricing algorithm through the main field experiment. The application of the best-response pricing algorithm resulted in a revenue improvement of 11%. Fisher et al. also noted that they applied their best-response pricing algorithm to Yihaodian’s kettle product category and achieved a revenue improvement of 19%. Fisher et al. suggested that due to the increasingly rapid changes in market conditions, retailers should update their consumer choice models periodically to “implement an effective dynamic-pricing strategy” (Fisher et al., 2018, p. 2513).
Fisher et al. used statistical modeling to develop their algorithm. Competitor prices, product availability, and competitors’ price reactions were retrieved daily from the competitor websites. Then, the computationally calculated price recommendations were sent to a designated employee of Yihaodian to execute the price changes.
In 2016, Ferreira et al. published their article “Analytics for an Online Retailer: Demand Forecasting and Price Optimization” in the journal ‘Manufacturing & Service Operations Management’.
Ferreira et al. partnered with Rue La La, which offered online flash sales on designer fashion products with high discounts. The flash sales lasted usually for between one and four days. Customers had to create an account to see offers online; Rue La La did not have physical stores. Before the field experiment, Rue La La had usually set prices according to a fixed percentage markup on costs.
Ferreira et al. wanted to create an algorithm for Rue La La to predict demand appropriately and to optimize pricing decisions so that revenue was maximized and inventory was sold out exactly when the deals ended. Neither should the items be out of stock before the end of the deal nor should stock remain after the end of the deal. The lack of historical transaction data for a specific product while considering simultaneous flash sales posed a key challenge to Ferreira et al.
First, Ferreira et al. built a demand prediction model based on data of historic deals. They applied machine learning techniques (i.e. an ensemble of regression models) to estimate historic lost sales due to out of stock situations, and to predict demand for upcoming deals. Regression models were used as they allow for interpretation of the predicted outcomes. A comparison of the performance of all applied regression models showed that the weighted average prediction of 100 regression trees performed the best.
Ferreira et al. used historic product-specific and deal-specific data as well as product data specific to a single deal to build the demand prediction model. Product-specific data included the affiliation of a product to a product group within the product hierarchy and the popularity of the size, color, and brand. Deal-specific data included the type and length of a deal as well as its time, weekday, month, and year. Combined product-deal-specific data included the final price of the product, the percentage markdown from the manufacturer’s price recommendation, the number and sales volume of competing deals, and the relative price to competing products.
Second, Ferreira et al. developed a price optimization model. As the demand prediction served as input for the prize optimization model and as the demand prediction model took the prices of competing products into account, the price optimization had to be applied to all competing deals at the same time. This required a particularly efficient price optimization algorithm.
Third, Ferreira et al. combined the aforementioned models into the final pricing algorithm. The lower price boundary was defined by the previously used markup on cost pricing. The upper price boundary was defined by a category-specific minimum percentage discount on manufacturer’s price recommendations as long as the resulting upper boundary was not greater than $15 or 15% than the lower boundary. Thereby, the low-cost image for designer fashion products should be maintained. After an initial hypercare phase, the pricing algorithm had been integrated fully automated into the IT architecture of Rue La La. The algorithm had run on a daily basis and sent price recommendations to employees. Once accepted, the price recommendations were directly updated in the enterprise resource planning (ERP) system. Ferreira et al. also automated the periodical update of the regression tree ensemble to maintain high levels of accuracy.
Finally, Ferreira et al. conducted a field experiment to measure the impact of their pricing algorithm on revenue and sales volume. The field experiment lasted from January to May 2014. The algorithm achieved to increase prices without decreasing sales for products with medium and high price levels relative to Rue La La’s typical price range. This resulted in an overall increase in revenue of 9,7% across all price levels. To make use of the pricing algorithm also for products with a low price level, Rue La La set the upper boundary at $5 above the lower boundary for these products after the field experiment. According to Ferreira et al., Rue La La has continued to use the pricing algorithm in their daily operations and began a project with Ferreira et al. to develop a dynamic pricing algorithm for their use case. Ferreira et al. further noted the following: “We hope […] that researchers and practitioners will use a combination of machine learning and optimization to harness their data and use it to improve business processes” (Ferreira et al., 2016, p. 86).
In 2017, Bradlow, Gangwar, Kopalle, and Voleti published their article “The Role of Big Data and Predictive Analytics in Retailing” in the ‘Journal of Retailing’.
Bradlow et al. partnered with a US-based retail chain that sold groceries and related non-food products. The field-experiment involved 42 stores and 788 products across 14 product categories. The field experiment lasted for 13 weeks. Since the exact date of the field experiment is not given and since the article was published in March 2017, the year of the experiment is set at 2016 for the analysis. The goal of the field experiment was to evaluate whether the application of price optimization through predictive analytics increased overall store profitability.
First, Bradlow et al. selected an econometric model – a standard logit-type attraction model – to estimate demand. The model parameters were modified to consider psychological effects of pricing like reference prices. Then, the model parameters were estimated through a maximum likelihood estimation using 102 weeks of data. Bradlow et al. included data such as historic transaction data, product specifications, seasonality, market share and competitor data, and special events in the past.
Second, Bradlow et al. solved an optimization problem to calculate a price recommendation on a weekly basis. The demand estimation model was also updated on a weekly basis. Several price constraints like margin boundaries, limits for price changes as well as product families and hierarchies were considered.
Finally, Bradlow et al. used a difference model to calculate the financial impact of their algorithm on profitability. The field experiment resulted in an average increase in margin of $0.407 per product per week per store. This margin improvement was extrapolated to an average increase in margin of $78,306.80 per year per store with 10,000 products.
Bradlow et al. used statistical modeling to calculate price recommendation. It is not specified whether the update of the demand model was conducted manually or automatically. Further, it is not specified whether the price recommendations were executed manually or automatically in the price leading systems. A flowchart of the algorithm indicates, however, that “price approval” and “price execution” were distinct steps in the algorithm (see Bradlow et al., 2017, p. 91).
Bradlow et al. concluded that despite the increasing popularity of data-driven approaches using big data and predictive analytics in the retail sector, the consideration of marketing theory and the thoughtful use of statistical methods is not losing importance.
In 2017, Cheung et al. published their article “Technical Note - Dynamic Pricing and Demand Learning with Limited Price Experimentation” in the journal “Operations Research”.
Cheung et al. partnered with Groupon to test a pricing algorithm that updated the initial price of each deal once with the aim to increase revenue. The price is only recalculated once to minimize regret and confusion over frequently changing prices on the customer-side. Groupon offered time-limited local deals of cooperating merchants in over 500 cities worldwide in an online shop. Most deals appeared only once in the online shop. The field experiment lasted for "several weeks" (Cheung et al., 2017, p. 1729) and included the product categories "beauty/healthcare, food/drink, leisure/activities, services, and shopping" (Cheung et al., 2017, p. 1729). 1,295 deals have received a recalculated price. The year of the field experiment is not specified. However, a note indicates that the experiment has been carried out around 2015. Therefore, the year of the experiment is set at 2015 for the analysis.
First, Cheung et al. developed a parametrical demand model using machine learning methods. The model was able to adapt itself based on transaction data for similar deals in the past. Similar deals shared features like length, product category, price range, and discount rate of historic deals. The model considered price policies such as Groupon’s constraint to only one price update per deal.
Second, Cheung et al. developed an algorithm to find the optimal price under consideration of the historic demand model. When new transaction data had been collected (e.g. after publishing a Groupon deal), the pricing algorithm took these new data into account and recalculated the optimal price.
Finally, Cheung et al. conducted the field experiment. Groupon only allowed price reductions so that sales volume could not decrease. The reason for this limitation is that the benefit of Groupon’s service for cooperating merchants relies on attracting a high number of new customers. The recommended price reduction was further limited to 5-30% off the initial price. The initial price had still been negotiated manually between Groupon and the merchants.
As a result of the recommended price reductions, daily sales5 increased for all product categories on average by 116%. Daily revenue increased on average by 21.7%. However, the price reductions had a negative impact on revenue for leisure/activities and services. Cheung et al. concluded that the demand model needed further adjustments to fit for all product categories.
3.3 Discussion
Between 2015 and January 2020, only four academic field experiments who evaluate the performance and suitability of advanced algorithms for pricing could be found. One possible explanation is confidentiality concerns of companies regarding their performance and pricing approach.
All four publications are considered to be of high quality regarding their academic rigor. Especially Fisher et al. (2018), Ferreira et al. (2016), and Cheung et al. (2017) disclosed relatively much information on the underlying models and the algorithms itself so that the approaches are understandable. Bradlow et al. (2017), however, gave only relatively limited insights into their pricing algorithm.
The four field experiments cover companies with different markets and business models within the B2C retail sector. Additionally, the length of the field experiments differs between five or several weeks and five months. Both factors reduce the comparability of the field experiments. Furthermore, the pricing algorithms pursue different goals, varying from initial product pricing over permanent and one-time price optimization to best-response pricing regarding changing competitor prices. This, in turn, shows different possibilities for application. Figure 2 in chapter 5 displays the tendential positions of the pricing algorithms in the ‘Algorithmic Pricing Advancement and Automation Model’ based on the available information.
While Fisher et al. (2018) and Bradlow et al. (2017) used statistical methods to identify the parameters of the underlying models, Ferreira et al. (2016) and Cheung et al. (2017) used machine learning approaches. The required effort to build the models was not reported.
Especially Ferreira et al. (2016) and Bradlow et al. (2017) advocated strongly for their approach. Hence, the authors appear to prefer certain techniques and methods. Despite these preferences, researchers and practitioners should be open-minded when looking for the optimal solution for a specific problem. Only Cheung et al. (2017) tested different approaches before deciding on a specific machine learning method (i.e. k-means clustering) to adjust their demand model.
The use of different data sources was rather limited. Mostly product data and transaction data were considered. Only Bradlow et al. (2017) involved data beyond these categories like seasonality and special events in the past. Still, there are many data sources that have not been considered in all four field experiments.
The degree of automation is considered low for the algorithm of Fisher et al. (2018) as the models were developed manually, the data were collected manually, and the prices were updated manually. In contrast, Ferreira et al. (2016) used machine learning to develop the model and integrated the algorithm fully automated into the architecture of the partnering company. The only intervention of employees was the acceptance or rejection of price recommendations. Additionally, the underlying model had been updated automatically. Regarding the algorithm of Bradlow et al. (2017), there is no information to which degree the algorithm is or can be automated. The degree of automation is considered low to medium for the algorithm of Cheung et al. (2017) as the model was developed manually and the initial prices were negotiated manually. However, there is no information about price execution. For all four field experiments, it is unclear to what extent the degree of automation could be increased when the algorithms were used for daily operations in contrast to a time-limited field experiment.
All field experiments showed an overall positive impact on sales, revenue, or profit margin. Revenue increased by 9.7% - 21.7%. The results of Ferreira et al. (2016) would even be higher if the lowest price level was not considered. The effect of an adjusted upper price boundary for the lowest price level as proposed by Ferreira et al. (2016) is not known. In the case of Bradlow et al. (2017), the margin improved by $0.407 per product per week per year. The extrapolation to $78,306.80 per year per store has not yet been validated.
Besides the financial impact of pricing algorithms, the interpretability of algorithms is a growing concern. Both statistical methods are well explainable as the models are developed manually. The interpretability of the machine learning-based models is limited. The parametrical model of Cheung et al. (2017) is developed manually; it uses, however, a multidimensional clustering method which can become difficult to comprehend. In the case of Ferreira et al. (2016), regression trees are in general comparably easy to interpret. From a practical point of view, however, the interpretability decreases with the depth of a regression tree. Ferreira et al. (2016) used the average prediction of 100 regression trees with an average depth of 582. Consequently, the practical interpretability of the machine learning methods is low.
After the field experiment, Rue La La implemented the pricing algorithm of Ferreira et al. (2016) in their daily operations and started a follow-up project to experiment on dynamic algorithmic pricing. While their algorithm is customized for initial product pricing for flash sales, Ferreira et al. (2016) suggested that regression trees are well suited to estimate the demand for new products in general as they are able to group new products with existing similar products. This would impact pricing, promotion, and production decisions for these new products.
Fisher et al. (2018) suggested that the developed models must be updated periodically to maintain their levels of prediction performance. Here again, automated approaches appear favorable to decrease manual efforts.
Under consideration of the positive results in all field experiments, more field experiments on algorithmic pricing are needed to validate the positive results for similar settings. More field experiments are also required to measure the performance of pricing algorithms in different product categories and market segments. As the given field experiments considered only a few data sources, field experiments using big data are needed to assess the additional benefits and challenges.
Future field experiments should include comparisons of different methods (e.g. theory-driven statistical approaches and data-driven machine learning approaches) to evaluate which algorithm is most suitable regarding certain goals, business environments, and available data sets.
Additionally, the given field experiments only considered the impact on product revenue and margin. Therefore, field experiments are needed to measure total costs and return on investments, too. These experiments must accompany the development, roll-out, and operation phases of pricing algorithms.
4 Expert Interviews on the Use of Algorithmic Pricing
4.1 Aim and Methodology
Expert interviews are conducted to gain insights about the use of algorithmic pricing in business practice. Based on the experience of the experts, the progress, state of the art, and future of algorithmic pricing will be examined. The results of these interviews will then be compared with the results of the literature review in chapter 5 to assess whether there is a research gap.
A qualitative interview-style mainly with open questions is chosen to gain deep insights from the interview partners. The interviews are semi-structured, meaning that pre-defined questions are asked whenever the questions fit in the course of the interview. This approach allows the expert to include the most important and valuable topics according to their perspective while still maintaining comparability between interviews (Young et al., 2018). The experts receive the questionnaires prior to the interview.
Furthermore, the categories for analysis are chosen in accordance with the categories for the analysis of the literature review to enable comparability to these results as well. The following categories for analysis are used for the expert interviews:
- Background (e.g. industry, company, type of business, position)
- Level of solution advancement (e.g. aim, techniques, and data sources)
- Degree of solution automation (e.g. automated tasks and human interaction)
- Assessment of algorithmic pricing activities (e.g. qualitative and quantitative, development over time)
- Outlook and further steps (e.g. personal and company’s perspective)
The analysis of the expert interviews is based on the structuring content analysis approach6 according to Mayring (2015). In a structuring content analysis, the qualitative insights of expert interviews are structured according to pre-defined categories. This approach appears suitable due to the semi-structured interview style, the given categories for analysis, and the non-uniform use of terms and keywords in the field of algorithmic pricing.
Experts with different backgrounds have been targeted to get a broad view on algorithmic pricing in business practice. The targeted experts are pricing consultants, solution providers, and professionals from the field of pricing and data science from companies that do or do not apply algorithmic pricing. Based on the categories for analysis, three questionnaires have been formulated to ensure a high fit for the different backgrounds of the experts.7
LinkedIn was used to search for professionals with matching job profiles. Search terms included among others “Pricing Manager”, “Data Scientist”, and the names of major companies from Europe and the US. The potential experts on the resulting longlist were then screened to ensure that only experts with appropriate experience and expertise are interviewed. The shortlisted experts then received an individualized message to ask for an expert interview.
Pricing consultancies and solution providers were identified through web research. Search terms included among others “IT consultancy”, “Pricing consultancy”, “Algorithmic pricing software”, “Price optimization software”, and “Dynamic pricing software”. The websites of the identified companies were screened to shortlist matching companies for an interview.
A total of 48 experts and companies were contacted by email or on LinkedIn in October and November 2019, resulting in 9 successfully conducted telephone interviews. Time constraints and confidentiality concerns were the most common reasons for refusing the interview.
4.2 Results
The results of the expert interviews are presented in the following.8
E1 (personal communication, November 29, 2019) works as Head of Pricing at a DAX company. His responsibilities include developing pricing strategies and tools. The company operates in several B2B markets. The company is currently in the implementation phase for a price management and analytics tool to derive price corridors from historic transaction data as guidance for future B2B quotations. These price corridors are recommended by micro-segmentation and clustering algorithms of third-party providers that identify similar deals from the past. The goals are to minimize value loss through consistent pricing and to improve efficiency and time to quote by automating the quoting process. To access the transaction data and to execute a price upon successful negotiation with the customer, the tool is directly connected to the ERP system.
According to E1, a major challenge before implementing such machine learning algorithms is the data pre-processing, especially when the company’s systems have grown and changed over time. Another problem occurs when historical data have only been stored in an aggregated form and not on a transaction level.
For E1’s company, a central pricing unit is relatively new. The price guidance tool helps to professionalize and partly automate the quotation process. Previously, pricing had been a time-consuming manual task. E1 expects that algorithmic pricing will become more important for B2C businesses and small B2B deals. However, B2B deals with a high strategic significance will still be negotiated manually but with some support of price guidance tools.
E2 (personal communication, November 28, 2019) works as International Pricing Manager and manages the internally used pricing tools. E2’s company is a Europewide retailer. The company uses different pricing tools both self-developed and customized for price calculations, simulations, reporting, and consolidation of competitor data. The goal is to optimize prices through algorithmic pricing and to enforce pricing strategies consistently.
Product managers can define pricing strategies and choose a calculation method for individual products. The influences of individual data sources and of pricing strategies for certain products are weighted in a scoring model. Considered data include internal data like transaction data and costs, external data like web traffic and product rankings in search engines and product reviews as well as competitor data like prices and availability. The pricing tools then calculate product prices according to this configuration and to general pricing rules. Further details on the underlying models and techniques have not been specified. The recommended prices are either automatically transferred into the ERP system or manually checked depending on the sales channel, region, and product group.
According to E2, the original motivation for the use of algorithmic pricing was to increase the efficiency of the manually driven pricing process. The interpretability of the automated price recommendations was of great importance for the software selection. Even now, there are still employees who do not trust the recommendations despite the high interpretability.
Due to a multitude of variables, there is no dedicated, quantitative performance indicator for algorithmic pricing at the company. However, the number of employees involved in the pricing process has been reduced significantly. Additionally, product managers benefited from algorithmic pricing to the extent that they could focus their attention on assortment decisions and supplier negotiations.
According to E2, fully automated pricing algorithms with self-improving demand and pricing models are desirable, but not yet conceivable in business practice for technological and cultural reasons.
E3 (personal communication, December 12, 2019) works as Domain Owner of Data Science for an internal IT service provider. The company’s main business is online and offline wholesale in Europe and Asia. E3 is responsible for data scientists and data engineers who improve corporate solutions through algorithms. The company uses different pricing tools, either self-developed or customized, for price personalization and price optimization for existing products. Dynamic pricing is not applied as the company’s customers expect price stability for most products. If customer expectations change, dynamic pricing might be reconsidered.
Machine learning algorithms are embedded in the pricing tools to develop a demand model and to identify related products for every product. According to E3, the problem of manually developed and rule-based models is that they are based on assumptions who can be error-prone. Additionally, the manual identification of related products is often based on obvious parameters while machine learning algorithms are able to unveil hidden patterns in the data. The yielded insights and patterns can be used beyond pricing.
The algorithm considers data like product hierarchy and relations, transaction data, holidays, and seasonality. The optimal prices are determined through optimization algorithms for different pricing strategies and goals like revenue or profit maximization.
For most products, price recommendations are automatically executed and updated in the price leading system. However, product managers can intervene, especially for products with a high focus and unusual price recommendations. Price recommendations are subject to restrictions regarding percentage price changes and absolute lower price boundaries. Electronic shelf labels have been used for years to reduce manual labor for price changes significantly.
According to E3, the company has been using pricing algorithms for at least 7 years. Since then, there has been a constant improvement of the algorithms, usability, workflows, and computational speed as well as an adaption to the omnichannel environment. Quantifiable performance indicators could not be specified but most improvements of the algorithms have had a positive financial impact.
For the future, E3 expects that pricing algorithms will be able to identify pricing strategies of competitors. Based on that, pricing algorithms can simulate scenarios for the optimal pricing strategy on product, category, and company level. Sufficient computational power and efficient data collection processes are prerequisites for this use case.
E4 (personal communication, December 16, 2019) also works at E3’s company and is the Product Owner for a specific pricing solution. He has been selected as an expert to complement the insights for their company from a different perspective than E3.
E4 is responsible for the third-party price optimization solution “Revionics”. It is mainly used for weekly shelf price optimization at the company and has been implemented in 7 countries so far. The company provides data like transaction data, current competitor prices, information on promotions, and product families and gaps. Additionally, the company defines pricing strategies, goals, and rules. Revionics applies a machine learning algorithm (Revionics, 2018) on the data to recommend optimal prices.
The collection of competitor prices, the import of data to Revionics, and the export of price recommendations from Revionics are performed semi-automatically. Depending on the importance of a product, price recommendations are either executed automatically or upon approval of a product manager. The automated execution of price recommendations is becoming increasingly popular as trust in the algorithm continues to grow. Revionics provides a high degree of transparency regarding the main influencing factors for a price recommendation. Revionics has been used for 6 years. Before that, active price management had only been performed for high-interest products.
Revionics predicts sales and profit lift resulting from price recommendations. Besides the positive impact on profit, the use of pricing algorithms has the advantage that all countries have to maintain a proper product and price architecture. This helps to convey a uniform price image to the customer.
In the future, more functionalities like price scales for different purchase quantities will be available. Additionally, the extension to algorithmic price personalization is pursued.
According to E4, there is no alternative to algorithmic pricing due to the ever-increasing complexity and dynamics. Depending on customer expectations and the product category, price adjustments could increase from weekly to daily or hourly changes.
E5 (personal communication, December 13, 2019) works as Head of Analytics. He is responsible for a team of data scientists who derive data-based insights to support various business decisions. The company operates retail stores in several European countries as well as a webshop with a delivery service near selected stores.
The company has a self-developed pricing tool that uses statistical models on category level to calculate product prices and recommended price ranges. The statistical models have been developed manually. The pricing tool considers data like competitor prices, defined limits for deviation from competitor prices, purchase prices, prices of related products, sales and profit forecasts, and price elasticities.
The pricing tool gives weekly recommendations that must be approved by a category manager. Especially products whose price is currently outside the recommended price range or for which the price recommendations have changed need to be reviewed. If there are important real-time events, prices might be updated more frequently. Most product prices only are changed a few times per year. The pricing tool is directly connected to the price leading system so that price recommendations can be directly transferred upon approval.
At E5’s company, the professionalization of pricing has started 10 years ago. First, simple tools to provide certain relevant data have been developed. Eventually, the available database became too large for manual evaluation. 7 years ago, the first simple pricing algorithms have been developed. Since then, the algorithms have been improved iteratively. E5 does not consider the pricing algorithms “top-rocket science”, yet they provide extremely helpful recommendations to accelerate and support pricing decisions, so that category managers do not have to rely solely on intuition anymore. Quantitative performance and financial impact of the pricing algorithms are not measured except for single price experiments. According to E5, transparency on influencing factors and interpretability of the results are very important in the food retail sector. The pricing tool of E5’s company provides full transparency on the influencing factors to meet these demands.
For the future, E5’s company considers to switch from category-specific to product-specific pricing models. Currently, product-specific pricing models are only used for promotions as the models are developed manually. Prerequisites for product-specific price models are sufficient data availability on product level and the automation of model development. “Real” dynamic pricing is only possible if all stores have electronic shelf labels and if customers accept frequently changing prices. Currently, roughly 40% of the stores are equipped with electronic shelf labels and German customers expect stable prices at supermarkets. Therefore, automated price personalization (e.g. through loyalty benefits or couponing) will be prioritized.
E6 (personal communication, December 6, 2019) works as Senior Consultant. E6’s consultancy is specialized in pricing and sales strategy mainly for B2B but also for B2C customers in Germany.
According to E6, pricing is, especially in B2B companies, often still a manual task without the use of pricing algorithms. In many companies, data and prices are calculated manually. When companies develop pricing models, these models are only seldom updated. Usually, considered data include costs, competitor prices, customer-specific characteristics, transaction data, size of the company, forecasts, estimated future potential of a product, market development scenarios, product and corporate strategies, and, especially for B2C, the real-time demand.
E6 evaluates the potential of self-learning pricing algorithms that automatically develop and update pricing models based on the available data as very high. However, pricing algorithms must be aligned with the entire marketing mix with clear price boundaries so that the company’s market positioning is not diluted.
With an increasing degree of automation and frequency of price changes, many companies struggle with the perceived loss of control. Especially for B2C companies, customer acceptance of frequent price changes is crucial. According to E6, frequently changing prices for all customers are perceived acceptable if the use of algorithms and influencing factors is communicated transparently and plausibly. Customer acceptance also depends on the perception of fairness regarding influencing factors. Many customers view obvious price personalization as critical because the prices are determined by personal factors.
Regarding pricing algorithms that recommend price corridors based only on historic transaction data in B2B, missing price governance in the past can negatively impact price recommendations. In such cases, target prices must be defined as a reference so that the price corridors move closer to the desired range. Algorithms that recommend price corridors should also be able to consider non-monetary terms of the contract as influencing factors.
E6 predicts that the significance of algorithmic dynamic pricing will increase in the future for B2C companies. However, many companies currently lack the required IT competencies to implement algorithmic pricing effectively.
E7 (personal communication, December 9, 2019) works as Vice President of Pricing and Marketing Analytics. She runs a team that maintains the analytics solutions for their customers. E7’s company is a US-based solution provider for forecasting, price and promotion optimization, personalization as well as custom solutions for retail pricing. Their customers are mainly located in the US and Europe. E7’s company uses statistical modeling and machine learning methods for big data solutions depending on the best fit for a given problem. The solutions are mainly based on transaction data, inventory levels, costs, and product hierarchies. They can also involve various third-party data into their models.
The solutions of E7’s company are mostly web-based applications instead of being directly integrated into the customers’ systems to satisfy customers’ security policies. Therefore, data and price recommendations are exchanged over secure file transfer servers. Customers usually build scripts to transfer the data and price recommendations to and from E7’s company. E7’s company automatically prioritizes its price recommendations regarding their expected impact and sends them to a decision-maker on the customer side. The prices have to be accepted manually by the customer for liability and accountability reasons.
E7 started 10 years ago at a startup that has later been acquired by E7’s company. Around that time, data analytics for pricing had a low priority in the retail sector. In Europe, manual pricing is still very common among retailers. E7 suggests that retailers start with manually defined, rule-based process automation for pricing decisions and only then implement optimization solutions. The reason is that many companies are mentally not prepared to give up all of their control over pricing at once. In some cases, process automation and speed are even more important in terms of profit maximization than analytics performance. The combination of all three factors, however, is most profitable.
Companies that consider implementing pricing tools often expect intuitive usability, an appealing interface, and dashboards. Analytics solutions must also be scalable to fit big data problems with billions of data points. Data engineering, IT architecture management, and workflow design, therefore, pose great challenges for big data projects.
E7 reported revenue improvements ranging from 1% for a company with $20,000,000 in annual sales to 0.2% for billion-dollar companies after implementing an E7’s company solution. Further benefits of algorithmic pricing include an increase in inventory turnover, less unsold inventory at the end of a season, and the reduction of required inventory space through more accurate forecasting.
According to E7, there is still a lot of potential regarding automation and optimization of pricing decisions, especially for retailers. Additionally, the importance of personalization will increase. However, European retailers face legal restrictions, data protection issues, and concerns by customers regarding price personalization to a greater extent than in the US and Asia.
E8 (personal communication, November 7, 2019) works as Account Development Manager and initiates customer relationships in the pre-sales phase. E8’s company is a US-based solution provider for various pricing problems mainly for B2B customers across industries worldwide. The company started with revenue optimization for airlines 30 years ago and introduced price optimization for B2B customers 13 years ago. The price optimization algorithms of E8’s company use machine learning to perform micro-segmentation of customers. The goal is to derive price corridors for quotations to new or existing customers. The price corridor reflects the customers’ usual willingness to pay.
The micro-segments are derived from historic transaction data, the location and size of customers, and customers’ revenues. Customers have the opportunity to add more data like competitor prices. The higher the number of transactions, the narrower and more precise is the price corridor.
The pricing solution is fitted to the specific situation of a customer by the implementation consultants of E8’s company. While standard data like transaction data are collected automatically, further data like market prices have to be provided manually or automatically by the customers of E8’s company. Upon the arrival of a new request, the price corridor is automatically calculated and send to the account managers of the customers of E8’s company. Account managers take the corridor as a reference for a quotation. The pricing algorithm of E8’s company is usually directly integrated into the customers’ customer relationship management (CRM) and ERP systems.
E8 reported average revenue increases of 2-4% and margin improvements of 1-3 percentage points per year through the use of the pricing algorithms of E8’s company. The advantage of price corridors is that account managers still have the perceived freedom to negotiate the final price while ensuring compliance with the price governance.
In the future, E8 expects pricing algorithms to become cheaper and quicker to implement so that they become more suitable for smaller companies. Additionally, data collection will be further automated.
E9 (personal communication, November 8, 2019) works as Chief Executive Officer (CEO). E9’s company is a solution provider for dynamic pricing and personalization in retail. The company serves clients worldwide in numerous countries. Various algorithms from the field of artificial intelligence like reinforcement machine learning and neural networks are applied on data like transaction data, weather, product features, product hierarchy, competitor prices, seasonality, and inventory level. Retailers can either set static pricing goals like profit, revenue, and sales maximization or design a ruleset according to which individual products are automatically assigned to specific pricing goals on a regular basis.
Recommendations for initial and regular pricing as well as for markdown pricing are calculated fully automatically within defined upper and lower boundaries. The level of automation depends on the product group. Price recommendations for high focus products and exceptions are often reviewed manually while price recommendations for products with a low focus are often executed automatically. The degree of automation also depends on the retailer's trust in the pricing tool. The pricing tools of E9’s company can be integrated into the customers’ ERP systems through interfaces.
In 2010, E9’s company began to develop and offer pricing solutions. According to E9, it is still common among retailers to use rule-based pricing approaches that depend on competitor prices. Additionally, markdown decisions are rather made on category-level instead of product-level.
E9 reported that margin improvements of up to 8 percentage points for regular pricing and up to 20 percentage points for markdown pricing can be achieved. This applies especially for product categories with relatively high margins, high freedom for pricing decisions and low comparability. Companies whose product categories do not share these characteristics benefit above all from process automation and the increased speed of reaction to changes in influencing factors.
Further benefits include, according to E9, that prices for products with low inventory levels can be increased automatically to delay or avoid out-of-stock situations. Thereby, retailers can reduce customer disappointment. In addition, the demand forecasting required by most pricing models can be used to optimize supply chain decisions. More accurate in-store inventory reduces handling and freight costs, storage space requirements, and markdowns at the end of a season.
In the future, employees will still be needed for pricing decisions. However, their tasks will change so that they rather define pricing boundaries and supervise the outcomes of pricing algorithms instead of setting individual prices.
Additionally, pricing algorithms will not only suggest promotion prices but also recommend which products to promote, personalized automatically on customer-level. In addition to improvements in the algorithms, however, process changes and rethinking on the part of applying companies are necessary to implement these solutions in business practice.
4.3 Discussion
The aim of the discussion is to compare and analyze the results of the expert interviews and to derive an answer to the second research question: “What progress and insights have companies made in using algorithmic pricing?”.
In total, 9 interviews have been conducted. In five of these interviews, data scientists and pricing experts from retailers, wholesalers, and manufacturers have been interviewed. These companies are active in food and non-food (B2C and B2B), consumer electronics (B2C), and adhesive technologies (B2B) industries. One interview has been conducted with a pricing consultant for B2B and B2C companies. Three interviews have been conducted with solution providers for algorithmic pricing. Two of the solution providers have a focus on B2C markets. Out of the represented companies, three operate worldwide, one operates in Europe and Asia, one operates in Europe and the US, two operate in Europe, and one operates in Germany.9 The different industries, target groups, and regions provide the experts with a variety of perspectives.
While all of the interviewed retailers, wholesalers, and manufacturers use pricing algorithms, the level of advancement and automation as well as the considered data vary. Figure 2 in chapter 5 displays the tendential positions of the pricing algorithms in the ‘Algorithmic Pricing Advancement and Automation Model’.
The interviewed companies use pricing algorithms for price corridor recommendations, price optimization of new and existing products, promotion optimization, markdown pricing, and price personalization. The level of advancement of the pricing algorithms, which are applied by the interviewed companies, varies from medium to high. While the pricing algorithm of E5’s company uses statistical methods and manually defined models (E5, personal communication, December 13, 2019), the pricing algorithm of E3 and E4’s company uses machine learning to define the underlying models (E3, personal communication, December 12, 2019; E4, personal communication, December 16, 2019). The solutions of E1’s company and E8’s company segment historic customer deals to derive price corridors using statistical and machine learning methods (E1, personal communication, November 29, 2019; E8, personal communication, November 7, 2019). The solution of E9’s company uses algorithms from the fields of machine learning and neural networks (E9, personal communication, November 8, 2019). In contrast, E6 (personal communication, December 6, 2019) reported that manual pricing is still very common in many companies. According to E7 (personal communication, December 9, 2019), this is especially true for European retailers.
The variety of considered data is mostly limited to transaction data, product information, and competitor prices. Beyond that, only the pricing algorithm of E2’s company considers web traffic and product rankings in search engines (E2, personal communication, November 28, 2019) and the algorithm of E9’s company considers weather data (E9, personal communication, November 8, 2019). Overall, however, a lot of data sources are still not considered.
Regarding the degree of automation, there is also a great variety. While the algorithm of E5’s company is based on a manually defined model and needs approval for its price recommendations (E5, personal communication, December 13, 2019), the algorithm of E3 and E4’s company is able to automatically define demand models and to execute price recommendations without approval (E3, personal communication, December 12, 2019). Still, all algorithms that offer automated price execution give employees the possibility to change or overwrite the product recommendations. E4 (personal communication, December 16, 2019) noted that the share of automated price executions rises with increasing trust in the algorithm. Additionally, all algorithms are provided with manually defined upper and lower boundaries and pricing goals. Only the algorithm of E9’s company allows for a rule-based allocation of products to pricing goals (E9, personal communication, November 8, 2019). While the self-developed algorithms of applying companies are directly integrated into the price-leading systems, third-party solutions often have file transfer servers or web interfaces for security reasons (E7, personal communication, December 9, 2019). While the companies of E2, E3, E4, and E9 already have algorithms with a relatively high degree of automation, it is still not at a desirable level according to E2 (personal communication, November 28, 2019).
Regarding the companies’ progress with algorithmic pricing, most of the experts reported that algorithmic pricing efforts began between 2009 and 2012. Since around 2009, the priority of data analytics and algorithmic pricing started to increase (E7, personal communication, December 9, 2019; E9, personal communication, November 8, 2019). In 2009, E5’s company started to consider a larger set of data for manual pricing in a more structured way. However, the data set quickly became too big for manual evaluation. Therefore, E5’s company implemented the first simple pricing algorithms in 2012 (E5, personal communication, December 13, 2019). E3 and E4’s company also started algorithmic pricing around 2012 (E3, personal communication, December 12, 2019; E4, personal communication, December 16, 2019). In contrast, the central pricing unit of E1’s company with the goal to professionalize pricing is relatively new so that their algorithmic pricing solution is still in the implementation phase (E1, personal communication, November 29, 2019). This finding suggests that pricing automation has a greater importance for B2C companies and B2B companies with a high number of relatively small deals compared to B2B companies with key account sales structures.
After the implementation of algorithmic pricing solutions, there have been constant, iterative improvements of the algorithms itself, the usability of the pricing tool, the embedded workflows, and the computational speed (E3, personal communication, December 12, 2019; E5, personal communication, December 13, 2019). E4 (personal communication, December 16, 2019) added that the addition of features and the rollout to country organizations proceed on a step-by-step basis. These results suggest that companies prefer to start with simple pricing algorithms and improve them constantly instead of trying to implement a full-scale solution for the whole organization at once. Additionally, E6 (personal communication, December 6, 2019) reported that many companies struggle initially when implementing an algorithmic pricing solution due to the perceived loss of control. Therefore, E7 (personal communication, December 9, 2019) suggested starting with simple rule-based pricing algorithms to automate the process and only then to include advanced optimization methods.
Regarding the quantitative assessment of pricing algorithms, only the three solution providers shared exact numbers: The revenue can be increased by 0.2-4% upon the implementation of algorithmic pricing (E8, personal communication, November 7, 2019; E7, personal communication, December 9, 2019), while the profit margin can be improved on average by 1-8 percentage points (E8, personal communication, November 7, 2019; E9, personal communication, November 8, 2019). Experts from applying companies reported that robust performance measurement of the isolated impact of algorithmic pricing is difficult to achieve and therefore not available (E2, personal communication, November 28, 2019; E5, personal communication, December 13, 2019). E3 (personal communication, December 12, 2019) and E4 (personal communication, December 16, 2019) reported that algorithmic pricing has had an overall positive financial impact. According to E9 (personal communication, November 8, 2019), the strength of the margin improvement depends on the product category and its market characteristics.
Besides the impact on revenue and profit margin, the experts reported numerous further advantages of algorithmic pricing. One implication of algorithmic pricing is the opportunity to automate processes. E7 (personal communication, December 9, 2019) reported that process automation and speed improvements often account for a higher share of profit maximization than analytics performance. Even simple algorithms can be extremely helpful for process automation (E5, personal communication, December 13, 2019) and lead, for example, to an improved time-to-quote in B2B key account environments (E1, personal communication, November 29, 2019).
While most companies still review the prices of high-interest products manually, they can also actively optimize the prices of low-interest products through the use of automated algorithmic pricing, which has not been the case before (E4, personal communication, December 16, 2019). The automation of pricing decisions allows reducing the required human resources so that employees can direct their attention to other challenges (E2, personal communication, November 28, 2019).
Another important benefit is that pricing strategies can be enforced more successfully, which leads to stronger price governance (E2, personal communication, November 28, 2019; E9, personal communication, November 8, 2019). Thereby, missed earnings can be reduced (E1, personal communication, November 29, 2019) and a more consistent price image can be presented to the customers.
Furthermore, the underlying models of most pricing algorithms yield insights that are valuable for business functions beyond pricing. For example, the demand forecasts can be used to optimize supply chain decisions like inventory management (E3, personal communication, December 12, 2019; E9, personal communication, November 8, 2019; E7, personal communication, December 9, 2019). Price adjustments depending on inventory levels, in turn, can avoid out-of-stock situations by using price elasticities to control demand (E9, personal communication, November 8, 2019).
Besides the aforementioned benefits, companies have made valuable insights on the use of algorithmic pricing. First, companies have to test different methods and model configurations to find the optimal solution for their specific pricing problem (E7, personal communication, December 9, 2019). This illustrates that a reliable way to measure the performance of pricing algorithms in business practice is needed.
Second, many companies face cultural and habitual issues on the employees’ side when implementing algorithmic pricing solutions. Transparency on influencing factors and interpretability of price recommendations are crucial for employees’ trust towards pricing algorithms (E2, personal communication, November 28, 2019; E4, personal communication, December 16, 2019; E5, personal communication, December 13, 2019). Regarding B2C environments, customers are also more likely to accept frequent price changes if the use of pricing algorithms and the influencing factors are transparently and plausibly communicated. Companies should ensure that the influencing factors are perceived fair by all customers to avoid negative reactions (E6, personal communication, December 6, 2019).
Third, sufficient data on product-level is needed to enable machine learning algorithms to generate pricing models for each individual product (E5, personal communication, December 13, 2019). The volume and the availability of the required data, however, can pose major challenges for data pre-processing and data engineering (E1, personal communication, November 29, 2019; E7, personal communication, December 9, 2019), especially if the IT landscape has changed over time (E5, personal communication, December 13, 2019). Furthermore, many companies currently lack the required IT competencies to solve the accompanying challenges and implement algorithmic pricing effectively (E6, personal communication, December 6, 2019).
Fourth, electronic shelf labels are needed in physical stores to reduce the manual effort of price changes significantly, especially if price changes occur more frequently through the use of algorithmic pricing (E3, personal communication, December 12, 2019). E5 (personal communication, December 13, 2019) even considered electronic shelf labels as a mandatory prerequisite for dynamic pricing scenarios in physical stores.
Overall, self-learning pricing algorithms that automatically develop and update pricing models have a very high potential when they are integrated with the entire marketing mix according to E6 (personal communication, December 6, 2019).
In the future, E3 (personal communication, December 12, 2019) expects that pricing algorithms will be able to identify the pricing strategies of competitors. Based on that, pricing algorithms will be able to simulate scenarios for the own optimal pricing strategy on product, category, and company level. Furthermore, pricing algorithms that recommend pricing corridors for B2B key account sales structures might become able to consider also non-monetary terms of the contract instead of mainly transaction data (E6, personal communication, December 6, 2019). Besides that, the importance and precision of the personalization of prices and promotions are expected to increase further (E7, personal communication, December 9, 2019; E9, personal communication, November 8, 2019).
5 Comparison and Synthesis of Literature Review and Expert Interviews
In this chapter, the results of the literature review and expert interviews will be compared to identify similarities, differences, and possible explanations.
Afterwards, the first research question will be answered: “What is the research gap between the current state of the literature and business practice regarding the use of advanced algorithms based on big data for algorithmic pricing?”. Then, the third research question will be answered: “How can algorithmic pricing be enhanced for future application?”.
Based on the assessments of the experts and the results of the literature review, the solutions are postitioned in the ‘Algorithmic Pricing Advancement and Automation Model’ in figure 2. The bubble sizes vary according to the availability and precision of the provided information.
Abbildung in dieser Leseprobe nicht enthalten
Figure 2: Assessment of the Pricing Algorithms from the Literature Review and Expert Interviews. (Source: Own illustration based on the results of the literature review and the expert interviews)
The solid, dotted, and dashed lines distinguish the applying companies, solutions providers, and researchers respectively. The positions of the applying companies support the results from expert interviews stated in subchapter 4.3 that many companies first increase the automation and then the complexity of their pricing algorithms.
Regarding the comparison of the results, it has been outlined that the field experiments as well as the expert interviews cover a variety of regions of operations and product categories in both online and offline markets. Although this variety decreases the direct comparability, it enables a more holistic review of algorithmic pricing. However, the field experiments covered only B2C environments.
The algorithms from the field experiments and the expert interviews represent a variety of algorithmic approaches from the fields of statistical modeling and machine learning, and multiple aims of the pricing algorithms (e.g. initial pricing, shelf pricing) so that the results are valid for multiple scenarios. One of the takeaways from the discussion of the field experiments, that different algorithmic approaches should be tested for a given problem, was supported by E7 (personal communication, December 9, 2019) in one of the expert interviews.
The range of data sources taken into account by all algorithms is comparable. As noted in both discussions, numerous data sources have not been considered. Although the field experiments covered the entire range of automation, especially the algorithms of Fisher et al. (2018) and Cheung et al. (2017) required many manual operation steps. It is unclear whether their algorithms can be further automated when they are not used in a time-limited field experiment but in running operations. The pricing algorithm of Ferreira et al. (2016), for example, was implemented by Rue La La after the field experiment and showed a high degree of automation.
Reports from both field experiments and expert interviews indicated a positive financial impact due to the pricing algorithms. The average increase in revenue in the field experiments was higher compared to the increase in revenue in the expert interviews. However, the financial assessments are not necessarily comparable as the methods for measurement may differ and different product categories may yield different results.
In contrast to the expert interviews, the literature review yielded not many non-financial benefits, insights, and visions for the future application of pricing algorithms. The reason for this is that the research goal of the field experiments was mainly a financial assessment of algorithmic pricing. However, most of the researchers and most of the experts shared their emphasis on the interpretability of the price recommendations.
Regarding the research gap between the literature and business practice, the positions of the applying companies, solution providers, and researchers in figure 2 visualize a good mix of these three groups in terms of both advancement and automation. While Fisher et al. (2018) deployed an algorithm with a rather low degree of automation, Ferreira et al. (2016) deployed an algorithm with a high degree of automation. Regarding the level of advancement, the algorithm of Bradlow et al. (2017) is at the lower end of the range, while the algorithm of Ferreira et al. (2016) is among the highest in this review. In sum, the pricing algorithms described in the expert interviews do not deviate substantially from the algorithms of the field experiments. This indicates a low research gap.
However, no field experiment has been conducted on algorithmic pricing for B2B key account sales structures. A field experiment for this business environment would close the research gap to business practice. Besides that, there is still a research need for field experiments on algorithmic pricing with the following goals as outlined in subchapter 3.3:
- validate the positive results of existing field experiments,
- investigate possible performance differences of pricing algorithms between product categories and market environments,
- use big data as input for pricing algorithms,
- compare the performance of different algorithmic methods for a given pricing problem, and
- examine the total costs and the return on investment for pricing algorithms from the initial development to the running operations phase.
Apart from the academic value of such field experiments, researchers could help companies to better account for influencing variables to reach a reliable performance measurement.
Ways to improve algorithmic pricing for future application were mostly identified by the experts and center around big data, pricing strategy, and integration with other business functions.
First, there are already machine learning algorithms that automatically derive demand models for individual products. What is missing is that algorithms are provided with access to all kinds of data sources. That would allow the algorithms to better evaluate which parameters are relevant for a specific product and how these parameters should be weighted in the price calculation. Thereby, the demand forecasts and price recommendations could become more accurate and the share of unexplained variance could be reduced. However, it is of great importance that the algorithms and their outcomes are interpretable to ensure acceptance by both employees and customers. Additionally, pricing algorithms for B2B key account sales structures could also consider the impact of non-monetary terms of past contracts to recommend a more accurate price corridor. A possible solution is to equip the pricing algorithms with textual analysis functionalities so that the algorithm could analyze the contracts and take these data into account.
Second, pricing algorithms can currently allocate rule-based pricing strategies to products. In the future, pricing algorithms could become able to analyze pricing strategies of competitors and to recommend an optimal pricing strategy for the own company on different aggregation levels. The algorithm would have to be equipped with:
- efficient, automated data collection processes to crawl all available competitor prices online and in databases ideally on a daily basis,
- sufficient data storage capacities to store all competitor product prices over time to monitor and identify possible patterns for price changes by competitors, and
- sufficient computational power to go through different scenarios regarding interdependencies between competitors and regarding best-response pricing strategies for the own company on category and product level.
In this scenario, it is important to consider the pricing strategy together with the entire marketing mix and corporate strategy to present a consistent market positioning to the customers.
Third, pricing algorithms are currently mainly used to optimize prices. Some experts reported opportunities to use “byproducts” of the price recommendations like demand forecasts for supply chain management, promotion planning, or personalization of product recommendations. Deep integration of the pricing algorithms into related business functions would enable further process automation and increase efficiencies beyond pricing decisions. A change in any of these functions could automatically trigger coordinated response recommendations from all other business functions. Such a deep, cross-functional integration requires the willingness of companies and their employees to automate these functions to a high degree.
6 Conclusion
Setting the right product prices at any given time holds substantial profit potential for companies and is a key factor to stay competitive. However, frequent price optimizations for the entire product assortment and all sales locations under consideration of all relevant data can be very time-consuming for companies. Therefore, companies increasingly use pricing algorithms to automize and optimize pricing decisions.
Pricing algorithms use techniques from the field of (big) data analytics to gain meaningful insights from the available data and recommend optimal prices. Pricing algorithms can vary regarding their level of advancement and degree of automation. For example, highly advanced learning algorithms can develop demand models based on the available data with minimal manual influence and execute the price recommendations fully automatically in the price-leading system of a company.
As no model could be found to compare the level of advancement and the degree of automation for pricing algorithms, the ‘Algorithmic Pricing Advancement and Automation Model’ has been proposed. The positions of multiple pricing algorithms in this model visualizes their scope and organizational integration and facilitates the direct comparison between them.
One research goal of this master thesis was to assess whether there is a research gap between the current state of the literature and business practice. Therefore, a literature review of four field experiments for algorithmic pricing and expert interviews with professionals from applying companies, a pricing consultant, and solution providers have been conducted. Despite the relatively small number of field experiments, a research gap has only been identified for pricing algorithms in B2B key account sales structures. In addition, research directions have been proposed to validate and extend the existing research and to encourage further advancements of algorithmic pricing, especially with regard to big data.
The second research goal was to assess what progress and insights companies have made in using algorithmic pricing. The results of the expert interviews have indicated that the interviewed companies started their algorithmic pricing efforts around 2010 and have improved their solutions steadily in small iterations since then. The experts have emphasized that the interpretability of the price recommendations and the trust of the employees in the pricing algorithms are important for the acceptance and the successful implementation of algorithmic pricing. Additionally, the mere automation of pricing decision processes has already led to increased efficiencies and higher revenues. All applying companies and solution providers reported positive financial and non-financial impacts. Part of the positive impacts came from the transfer of new insights and information to other business functions like promotion planning and supply chain management. Another advantage of algorithmic pricing is a stronger price governance through the more consistent enforcement of a company’s pricing strategy. Before implementing algorithmic pricing, companies should try different approaches and techniques for the given pricing problem. The volume, variety, veracity, and velocity of the required data can also pose a challenge to the data engineering that has to be considered. These insights might be used by companies to further improve their algorithmic pricing solutions or to initiate algorithmic pricing projects successfully.
The third research goal was to assess how algorithmic pricing can be improved for future application. One opportunity for improvement is to provide advanced pricing algorithms with access to truly big data, especially in terms of variety, rather than just to a limited selection of data sources. This is expected to enable advanced pricing algorithms to develop more accurate demand models and, hence, calculate better price recommendations. Another opportunity for improvement is to enable pricing algorithms to collect, store, and process competitor prices to derive the competitors’ pricing strategies. On this basis, the pricing algorithm could recommend an optimal response pricing strategy for the own company. A third opportunity for improvement is to expand the scope of the algorithms cross-functionally beyond pricing to integrate several business functions. A change in the demand forecast could automatically trigger a coordinated response recommendation for pricing, promotion planning, and supply chain management. The presented opportunities for improvement are expected to result in even more advanced algorithms and give a direction for further development efforts by researchers and companies.
The results of this master thesis are constrained by several limitations. First, the proposed ‘Algorithmic Pricing Advancement and Automation Model’ is missing a reliable scale for the level of advancement of the algorithm. Additionally, there is no clear allocation of combinations from the level of automation taxonomy by Save and Feuerberg (2012) to the degree of automation in the model. However, the model has proven useful for the comparison of the results from the literature review and expert interviews in this master thesis. For the future, the degree of interpretability could be added to the model as a third dimension as this has been found to be important in business practice.
The second limitation of this master thesis is that the meta-ethnographic approach used in the literature review is criticized for limited validity, despite the increasing use of meta-ethnography for interpretive reviews in marketing research. The reason for the criticism is that the results of the literature review depend partly on the interpretation of the context and the terms used in the examined publications (Rocque, Brisset, & Leanza, 2017).
The third limitation is related to the selection of the experts for the interviews. It may be that only those potential interviewees with a positive experience with algorithmic pricing responded. In this case, the assessment of pricing algorithms derived from the expert interviews would be positively biased.
Despite all positive reports regarding algorithmic pricing, companies must also consider the ethical challenges when prices are discriminated over time and based on location and personal customer data. Companies should proactively counter even unintended biases in their algorithms to prevent confusion and negative responses by customers (Seele, Dierksmeier, Hofstetter, & Schultz, 2019).
Appendix A – The Level of Automation Taxonomy by Save and Feuerberg
Abbildung in dieser Leseprobe nicht enthalten
(Source: Save & Feuerberg, 2012, pp. 48-50)
Abbildung in dieser Leseprobe nicht enthalten
Appendix B – Overview of Results of the Literature Review
Abbildung in dieser Leseprobe nicht enthalten
(Source: Own illustration based on the literature review)
Appendix C – Questionnaires for Expert Interviews
C.1 Applying companies
Q1: What is your name, company, industry, sales channels, and role?
Q2: Does your company use algorithms and big data for pricing?
If algorithms are used:
Q3a: What type of algorithm do you use?
Q4a: Do algorithms or data scientists determine the price model for each product?
Q5a: Which data do you use (e.g. internal, external, historical, real-time)? What are some examples of considered big data?
Q6a: How is the pricing algorithm integrated into business processes? Does it have a connection to IT systems, online-shops, and offline-stores?
Q7a: How does the interaction with human pricing or product specialists look like? Does the algorithm give recommendations, allow for a right of veto, or does it work fully automatically?
Q8a: Has the use of algorithmic pricing had a positive impact on profit and other performance indicators so far?
Q9a: How and when did your company start with the algorithmic pricing solution? How has it developed over time?
Q10a: How do you see the future of algorithmic pricing? Which related solutions might it lead to?
If algorithms are not in use:
Q3b: Why is algorithmic pricing not used? Is it planned in the future?
Q4b: How do you expect the future of pricing to be in your company and in general?
Q5b: What must change so that you could implement algorithmic pricing in the future?
C.2 Pricing Consultancies
Q1: What is your name, company, and role? Who are your clients?
Q2: Do some of your clients use algorithms and big data for pricing?
If algorithms are used:
Q3a: What type of algorithm do your clients use?
Q4a: Do algorithms or employees of your clients determine the price for each product?
Q5a: Which data do your clients use (e.g. internal, external, historical, real-time)?
Q6a: How is the pricing algorithm integrated into the business processes of your clients? Does it have a connection to IT systems, online-shops, and offline-stores?
Q7a: How does the interaction with human pricing or product specialists look like? Does the algorithm give recommendations, allow for a right of veto, or does it work fully automatically?
Q8a: Do you know how the performance of your clients changes after the implementation on average? If yes, please give an estimation.
Q9a: How and when did your clients start with their algorithmic pricing solution? How has it developed over time?
Q10a: How do you see the future of algorithmic pricing? Which related solutions might it lead to?
If algorithms are not in use:
Q3b: Why is algorithmic pricing not used? Do your clients plan it for the future?
Q4b: How do you expect the future of pricing to be for your clients and in general?
Q5b: What must change so that all companies could implement algorithmic pricing in the future?
C.3 Solution Providers
Q1: What is your name, company, and role? Who are your clients?
Q2: Could you please briefly describe the general idea of your pricing solution?
Q3: What type of algorithm do you use?
Q4: Do algorithms or data scientists determine the price model for each product?
Q5: Which data do you use (e.g. internal, external, historical, real-time)? What are some examples of considered big data?
Q6: How can your solution be integrated into business processes? Does it have a connection to IT systems, online-shops, and offline-stores?
Q7: How does the interaction with human pricing or product specialists look like? Does it give recommendations, allow for a right of veto, or does it work fully automatically?
Q8: Do you know how the performance of your clients changes after the implementation on average? If yes, please give an estimation.
Q9: How and when did your company start with the algorithmic pricing solution? How has it developed over time?
Q10: How do you see the future of algorithmic pricing? Which related solutions might it lead to?
Abbildung in dieser Leseprobe nicht enthalten
Appendix D – Overview of Results of the Expert Interviews
Abbildung in dieser Leseprobe nicht enthalten
References
Akter, S., & Wamba, S. F. (2016). Big data analytics in e-commerce: A systematic review and agenda for future research. Electronic Markets, 26(2), 173–194.
AmalNick, M. S., & Qorbanian, R. (2017). Dynamic pricing using wavelet neural network under uncertain demands. Decision Science Letters, 6, 251–260.
Archer, B. H. (1980). Forecasting demand: Quantitative and intuitive techniques. International Journal of Tourism Management, 1(1), 5–12.
Baker, W., Kiewell, D., & Winkler, G. (2014). Using big data to make better pricing decisions. Retrieved from https://www.mckinsey.com/business-functions/marketing-and-sales/our-insights/using-big-data-to-make-better-pricing-decisions
Bar-Gill, O. (2019). Algorithmic price discrimination: When demand is a function of both preferences and (mis)perceptions. The University of Chicago Law Review, 86(2), 217–254.
Bradlow, E. T., Gangwar, M., Kopalle, P., & Voleti, S. (2017). The role of big data and predictive analytics in retailing. Journal of Retailing, 93(1), 79–95.
Calvano, E., Calzolari, G., Denicolo, V., & Pastorello, S. (2019). Artificial intelligence, algorithmic pricing and collusion. SSRN Electronic Journal. Retrieved from https://dx.doi.org/10.2139/ssrn.3304991
Chen, L., Mislove, A., & Wilson, C. (2016). An empirical analysis of algorithmic pricing on Amazon marketplace. In J. Bourdeau, J. A. Hendler, R. N. Nkambou, I. Horrocks, & B. Y. Zhao (Eds.), Proceedings of the 25th International Conference on World Wide Web (pp. 1339–1349). New York, USA: ACM. Retrieved from https://doi.org/10.1145/2872427.2883089
Cheung, W. C., Simchi-Levi, D., & Wang, H. (2017). Technical note - dynamic pricing and demand learning with limited price experimentation. Operations Research, 65(6), 1722–1731.
Competition & Markets Authority (2018). Pricing algorithms: Economic working paper on the use of algorithms to facilitate collusion and personalised pricing. London, England. Retrieved from https://assets.publishing.service.gov. uk/government/uploads/system/uploads/attachment_data/file/746353/Algorithms_econ_report.pdf
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to algorithms (2nd ed.). Cambridge, England: MIT Press.
Dangeti, P. (2017). Statistics for machine learning: Build supervised, unsupervised, and reinforcement learning models using both Python and R. Birmingham, England: Packt.
Davenport, T., Guha, A., Grewal, D., & Bressgott, T. (2020). How artificial intelligence will change the future of marketing. Journal of the Academy of Marketing Science, 48(1), 24–42.
Den Boer, A. V. (2015). Dynamic pricing and learning: Historical origins, current research, and new directions. Surveys in Operations Research and Management Science, 20(1), 1–18.
Eilon, S., & Cosmetatos, G. P. (1979). Effect of price and allied changes on profit. Applied Economics, 11(2), 117–131.
Erevelles, S., Fukawa, N., & Swayne, L. (2016). Big data consumer analytics and the transformation of marketing. Journal of Business Research, 69(2), 897–904.
Ferreira, K. J., Lee, B. H. A., & Simchi-Levi, D. (2016). Analytics for an online retailer: Demand forecasting and price optimization. Manufacturing & Service Operations Management, 18(1), 69–88.
Fisher, M., Gallino, S., & Li, J. (2018). Competition-based dynamic pricing in online retailing: A methodology validated with field experiments. Management Science, 64(6), 2496–2514.
Gandomi, A., & Haider, M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35(2), 137–144.
Gartner (2018). Gartner top 10 strategic technology trends for 2019. Retrieved from https://www.gartner.com/smarterwithgartner/gartner-top-10-strategic-technology-trends-for-2019/
Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.‑Z. (2019). XAI - Explainable artificial intelligence. Science Robotics, 4(37), 1-2.
Klenow, P. J., & Kryvtsov, O. (2008). State-dependent or time-dependent pricing: Does it matter for recent U.S. inflation? The Quartarly Journal of Economics, 123(3), 863–904.
Knuth, D. E. (1997). Fundamental algorithms (3rd ed.). The art of computer programming: Vol. 1. Reading, USA: Addison-Wesley.
Landset, S., Khoshgoftaar, T. M., Richter, A. N., & Hasanin, T. (2015). A survey of open source tools for machine learning with big data in the hadoop ecosystem. Journal of Big Data, 2. Retrieved from https://journalofbigdata.springeropen. com/articles/10.1186/s40537-015-0032-1
Lee, I., & Shin, Y. J. (2020). Machine learning for enterprises: Applications, algorithm selection, and challenges. Business Horizons, 63(2), 157–170.
Mayring, P. (2015). Qualitative Inhaltsanalyse: Grundlagen und Techniken (12th ed.). Weinheim, Germany: Beltz.
Mikalef, P., Pappas, I. O., Krogstie, J., & Giannakos, M. (2018). Big data analytics capabilities: A systematic literature review and research agenda. Information Systems and E-Business Management, 16(3), 547–578.
Mohammed, R. (2012). Use pricing strategy to boost sales. Harvard Business Review. Retrieved from https://hbr.org/2012/07/use-pricing-strategy-to-boost
Nayak, A., & Patil, V. (2018), Wal-Mart Stores U.S. Patent 10,068,241 B2. USA.
Noblit, G. W., & Hare, D. R. (1988). Meta-ethnography: Synthesizing qualitative studies. Qualitative Research Methods: Vol. 11. Newbury Park, USA: Sage.
Parasuraman, R., Sheridan, T. B., & Wickens, C. D. (2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part a: Systems and Humans, 30(3), 286–297.
Provost, F., & Fawcett, T. (2013). Data science for business: What you need to know about data mining and data-analytic thinking. Sebastopol, USA: O’Reilly.
Rana, R., & Oliveira, F. S. (2014). Real-time dynamic pricing in a non-stationary environment using model-free reinforcement learning. Omega, 47, 116–126.
Revionics (2018). Pricing. Retrieved at 01.02.2020. Retrieved from https://www.revionics.com/solutions/pricing/
Rocque, R., Brisset, C., & Leanza, Y. (2017). Meta-ethnography. In M. Allen (Editor), The Sage encyclopedia of communication research methods (pp. 1013–1017). Thousand Oaks, USA: Sage.
Save, L., & Feuerberg, B. (2012). Designing human-automation interaction: A new level of automation taxonomy. In D. d. Waard, K. Brookhuis, F. Dehais, C. Weikert, S. Röttger, D. Manzey, . . . P. Terrier (Eds.), Proceedings HFES Europe Chapter Conference, Toulouse, France. Retrieved from https://www.hfes-europe.org/human-factors-view-integrative-perspective/
Seele, P., Dierksmeier, C., Hofstetter, R., & Schultz, M. D. (2019). Mapping the ethicality of algorithmic pricing: A review of dynamic and personalized pricing. Journal of Business Ethics. Retrieved from https://doi.org/10.1007/s10551-019-04371-w
Shankar, V. (2018). How artificial intelligence (AI) is reshaping retailing. Journal of Retailing, 94(4), vi–xi.
Simchi-Levi, D. (2017). The new frontier of price optimization. MIT Sloan Management Review, 59(1), 22–26.
Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2017). Critical analysis of big data challenges and analytical methods. Journal of Business Research, 70, 263–286.
Wal-Mart Stores (2015). Walmart acquires remaining shares to take full ownership of Yihaodian e-commerce business in china. Retrieved at 24.01.2020. Retrieved from https://corporate.walmart.com/newsroom/2015/07/22/walmart-acquires-remaining-shares-to-take-full-ownership-of-yihaodian-e-commerce-business-in-china
Young, J. C., Rose, D. C., Mumby, H. S., Benitez-Capistros, F., Derrick, C. J., Finch, T., . . . Mukherjee, N. (2018). A methodological guide to using and reporting on interviews in conservation science research. Methods in Ecology and Evolution, 9, 10–19. Retrieved from https://doi.org/10.1111/2041-210X.12828
[...]
1 For example, one pricing algorithm, which is discussed in the literature review, had already been improved for efficiency and still had a daily runtime of about 1 to 4.5 hours with a quad-core processor and 16 gigabyte of random access memory (see Ferreira, Lee, & Simchi-Levi, 2016). To solve this algorithm manually would be practically unrealistic.
2 See Appendix A for a description of all automation levels in the taxonomy by Save and Feuerberg (2012).
3 See subchapter 2.1 for examples of simple, medium-advanced, and highly advanced pricing algorithms.
4 See Appendix B for a tabulated overview of the results of the literature review.
5 Groupon denotes “sales” as “bookings”, meaning the price payable by the customer. By revenue, Groupon means the commission as a share of bookings (Cheung et al., 2017).
6 The original German name of this approach is „inhaltliche Strukturierung“ (Mayring, 2015).
7 See Appendix C for the interview questionnaires.
8 See Appendix D for a tabulated overview of the results of the expert interviews.
9 The stated regions represent the main regions of operations of the interviewed companies. Two interviews have been conducted with employees from the same company. Therefore, this company is only counted once.
- Arbeit zitieren
- Lukas Kern (Autor:in), 2020, Algorithmic Pricing based on Big Data. A Critical Reflection, München, GRIN Verlag, https://www.grin.com/document/704055
-
-
-
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen. -
-
Laden Sie Ihre eigenen Arbeiten hoch! Geld verdienen und iPhone X gewinnen.