The memory concept behind deep neural network models: An application in time series forecasting in the e-Commerce sector
DOI:
https://doi.org/10.31181/dmame622023695Keywords:
E-Commerce, time series, deep neural network, forecasting, prediction error, computational costAbstract
A good command of computational and statistical tools has proven advantageous when modelling and forecasting time series. According to recent literature, neural networks with long memory (e.g., Short-Term Long Memory) are a promising option in deep learning methods. However, only some works also consider the computational cost of these architectures compared to simpler architectures (e.g., Multilayer Perceptron). This work aims to provide insight into the memory performance of some Deep Neural Network architectures and their computational complexity. Another goal is to evaluate whether choosing more complex architectures with higher computational costs is justified. Error metrics are then used to assess the forecasting models' performance and computational cost. Two-time series related to e-commerce retail sales in the US were selected: (i) sales volume; (ii) e-commerce sales as a percentage of total sales. Although there are changes in data dynamics in both series, other existing characteristics lead to different conclusions. "Long memory" allows for significantly better forecasts in one-time series. In the other time series, this is not the case.
Downloads
References
Atsalakis, G. (2016). New Technology in Shopping: Forecasting Electronic Shopping With the Use of a Neuro-Fuzzy System. Journal of Food Products Marketing, 23(5), 522–532. https://doi.org/10.1080/10454446.2014.1000445
Brownlee, J. (2018). Deep learning for time series forecasting: predict the future with MLPs, CNNs and LSTMs in Python. Machine Learning Mastery.
Cavalcante, R. C., Brasileiro, R. C., Souza, V. L. F., Nobrega, J. P., & Oliveira, A. L. I. (2016). Computational Intelligence and Financial Markets: A Survey and Future Directions. Expert Systems with Applications, 55, 194–211. https://doi.org/10.1016/j.eswa.2016.02.006
Chatfield, C. (2016). The Analysis of Time Series: an introduction (6th ed.). Chapman and Hall/CRC.
Chollet, F. (2021). Deep Learning with Python, Second Edition. Manning Publications.
Corsini, R. R., Costa, A., Fichera, S., & Framinan, J. M. (2022). A new data-driven framework to select the optimal replenishment strategy in complex supply chains. IFAC-PapersOnLine, 55(10), 1423–1428.
Costa, A., Ramos, F. R., Mendes, D., & Mendes, V. (2019). Forecasting financial time series using deep learning techniques. In IO 2019 - XX Congresso da APDIO 2019. Instituto Politécnico de Tomar - Tomar.
Data Science Academy. (2019). Deep Learning Book. Retrieved from http://deeplearningbook.com.br/
Diniz, A. P. M., Ciarelli, P. M., Salles, E. O. T., & Coco, K. F. (2022). Long Short-Term Memory Neural Networks for Clogging Detection in the Submerged Entry Nozzle. Decision Making: Applications in Management and Engineering, 5(1), 154–168. https://doi.org/10.31181/dmame0313052022d
Ghosal, S., Dey, S., Chattopadhyay, P. P., Datta, S., & Bhattacharyya, P. (2021). Designing optimized ternary catalytic alloy electrode for efficiency improvement of semiconductor gas sensors using a machine learning approach. Decision Making: Applications in Management and Engineering, 4(2), 126–139. https://doi.org/10.31181/dmame210402126g
Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In JMLR W&CP: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2010) (pp. 249–256). Sardinia: JMLR Workshop and Conference Proceedings.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2015). LSTM: A Search Space Odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2222–2232. https://doi.org/10.1109/TNNLS.2016.2582924
Hang, N. T. (2019). Research on a number of applicable forecasting techniques in economic analysis, supporting enterprises to decide management. World Scientific News, 119, 52–67.
Higueras-Castillo, E., Liébana-Cabanillas, F. J., & Villarejo-Ramos, Á. F. (2023). Intention to use e-commerce vs physical shopping. Difference between consumers in the post-COVID era. Journal of Business Research, 157, 113622. https://doi.org/10.1016/j.jbusres.2022.113622
Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Jiang, Z., & Benbasat, I. (2014). Virtual Product Experience: Effects of Visual and Functional Control of Products on Perceived Diagnosticity and Flow in Electronic Shopping. Journal of Management Information Systems, 21(3), 111–147. https://doi.org/10.1080/07421222.2004.11045817
Jozefowicz, R., Zaremba, W., & Sutskever, I. (2015). An Empirical Exploration of Recurrent Network Architectures. In ICML - International Conference on Machine Learning.
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR.
https://doi.org/10.48550/arXiv.1412.6980
Koutník, J., Greff, K., Gomez, F., & Schmidhuber, J. (2014). A Clockwork RNN. 31st International Conference on Machine Learning, ICML 2014, 5, 3881–3889.
Lopes, D. R., & Ramos, F. R. (2020). Univariate Time Series Forecast. Retrieved from https://github.com/DidierRLopes/UnivariateTimeSeriesForecast
Lopes, D. R., Ramos, F. R., Costa, A., & Mendes, D. (2021). Forecasting models for time-series: a comparative study between classical methodologies and Deep Learning. In SPE 2021 – XXV Congresso da Sociedade Portuguesa de Estatística. Évora - Portugal.
Martínez-López, F. J., Feng, C., Li, Y., & Sansó Mata, M. (2022). Restoring the buyer–seller relationship through online return shipping: The role of return shipping method and return shipping fee. Electronic Commerce Research and Applications, 54, 101170. https://doi.org/10.1016/j.elerap.2022.101170
Modgil, S., Dwivedi, Y. K., Rana, N. P., Gupta, S., & Kamble, S. (2022). Has Covid-19 accelerated opportunities for digital entrepreneurship? An Indian perspective. Technological Forecasting and Social Change, 175, 121415. https://doi.org/10.1016/j.techfore.2021.121415
Pesaran, M. H., & Timmermann, A. (2004). How costly is it to ignore breaks when forecasting the direction of a time series? International Journal of Forecasting, 20(3), 411–425. https://doi.org/10.1016/S0169-2070(03)00068-2
Pineda, F. (1987). Generalization of Back propagation to Recurrent and Higher Order Neural Networks. Undefined.
Ramos, F. R. (2021). Data Science na Modelação e Previsão de Séries Económico-financeiras: das Metodologias Clássicas ao Deep Learning. (PhD Thesis, Instituto Universitário de Lisboa - ISCTE Business School, Lisboa, Portugal). https://doi.org/10.13140/RG.2.2.14510.02887
Ramos, F. R., Costa, A., Mendes, D., & Mendes, V. (2018). Forecasting financial time series: a comparative study. In JOCLAD 2018, XXIV Jornadas de Classificação e Análise de Dados. Escola Naval – Alfeite. https://doi.org/10.13140/RG.2.2.11548.41606
Ramos, F. R., Lopes, D. R., Costa, A., & Mendes, D. (2021). Explorando o poder da memória das redes neuronais LSTM na modelação e previsão do PSI 20. In SPE 2021 – XXV Congresso da Sociedade Portuguesa de Estatística. Évora - Portugal.
Ramos, F. R., Lopes, D. R., & Pratas, T. E. (2022). Deep Neural Networks: A Hybrid Approach Using Box&Jenkins Methodology. In Innovations in Mechatronics Engineering II. icieng 2022. Lecture Notes in Mechanical Engineering (pp. 51–62). Springer, Cham. https://doi.org/10.1007/978-3-031-09385-2_5
Ravichandiran, S. (2019). Hands-On Deep Learning Algorithms with Python: Master deep learning algorithms with extensive math by implementing them using TensorFlow. Packt Publishing Ltd.
Ren, X. X., Gong, Y., Rekik, Y., & Xu, X. (2022). Data-driven analysis on anticipatory shipping for pickup point inventory. IFAC-PapersOnLine, 55(10), 714–718. https://doi.org/10.1016/j.ifacol.2022.09.491
Rubio, L., & Alba, K. (2022). Forecasting Selected Colombian Shares Using a Hybrid ARIMA-SVR Model. Mathematics, Vol. 10, Page 2181, 10(13), 2181. https://doi.org/10.3390/math10132181
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(6088), 533–536. https://doi.org/10.1038/323533a0
Sezer, O. B., Gudelek, M. U., & Ozbayoglu, A. M. (2020). Financial time series forecasting with deep learning : A systematic literature review: 2005–2019. Applied Soft Computing, 90, 106–181. https://doi.org/10.1016/j.asoc.2020.106181
Tealab, A. (2020). Time series forecasting using artificial neural networks methodologies: A systematic review. Future Computing and Informatics Journal, 3(2). https://doi.org/10.1016/j.fcij.2018.10.003
Tkáč, M., & Verner, R. (2016). Artificial neural networks in business: Two decades of research. Applied Soft Computing, 38, 788–804. https://doi.org/10.1016/j.asoc.2015.09.040
Utku, A. (2023). Deep learning based an efficient hybrid prediction model for Covid-19 cross-country spread among E7 and G7 countries. Decision Making: Applications in Management and Engineering, 6(1), 502–534. https://doi.org/10.31181/dmame060129022023u
Utku, A., & Kaya, S. K. (2022). Multi-Layer Perceptron Based Transfer Passenger Flow Prediction in Istanbul Transportation System. Decision Making: Applications in Management and Engineering, 5(1), 208–224. https://doi.org/10.31181/dmame0315052022u
Wang, J., & Dai, C. H. (2004). A fuzzy constraint satisfaction approach for electronic shopping assistance. Expert Systems with Applications, 27(4), 593–607. https://doi.org/10.1016/j.eswa.2004.06.004
Willmott, C., & Matsuura, K. (2005). Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate Research, 30(1), 79–82. https://doi.org/10.3354/cr030079
Wilson, J. H., & Spralls III, S. A. (2018). What do business professionals say about forecasting in the marketing curriculum? International Journal of Business, Marketing, & Decision Science, 11(1), 1–20.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Decision Making: Applications in Management and Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
