# American Institute of Mathematical Sciences

• Previous Article
Optimum pricing strategy for complementary products with reservation price in a supply chain model
• JIMO Home
• This Issue
• Next Article
Stochastic machine breakdown and discrete delivery in an imperfect inventory-production system
July  2017, 13(3): 1537-1552. doi: 10.3934/jimo.2017006

## Optimal threshold control of a retrial queueing system with finite buffer

 School of Mathematics and Statistics, Central South University, Changsha 410083, Hunan, China

* Corresponding author: Jinbiao Wu

Received  September 2015 Published  December 2016

Fund Project: The second author is supported by the National Natural Science Foundation of China (11271373) and the third author is supported by the project of Mathematics and Interdisciplinary Science and Innovation-Driven of Central South University (10900-506010101) and the Yu Ying project of Central South University.

In this paper, we analyze the optimal control of a retrial queueing system with finite buffer K. At any decision epoch, if the buffer is full, the controller have to make two decisions: one is for the new arrivals, to decide whether they are allowed to join the orbit or not (admission control); the other one is for the repeated customers, to decide whether they are allowed to get back to the orbit or not (retrial control). The problems are constructed as a Markov decision process. We show that the optimal policy has a threshold-type structure and the thresholds are monotonic in operating parameters and various cost parameters. Furthermore, based on the structure of the optimal policy, we construct a performance evaluation model for computing efficiently the thresholds. The expression of the expected cost is given by solving the quasi-birth-and-death (QBD) process. Finally, we provide some numerical results to illustrate the impact of different parameters on the optimal policy and average cost.

Citation: Gang Chen, Zaiming Liu, Jinbiao Wu. Optimal threshold control of a retrial queueing system with finite buffer. Journal of Industrial and Management Optimization, 2017, 13 (3) : 1537-1552. doi: 10.3934/jimo.2017006
##### References:
 [1] H.-S. Ahn, I. Duenyas and M. E. Lewis, Optimal control of a two-stage tandem queuing system with flexible servers, Probability in the Engineering and Informational Sciences, 16 (2002), 453-469.  doi: 10.1017/S0269964802164047. [2] A. S. Alfa and K. S. Isotupa, An M/PH/k retrial queue with finite number of sources, Computers & Operations Research, 31 (2004), 1455-1464.  doi: 10.1016/S0305-0548(03)00100-X. [3] J. R. Artalejo, Accessible bibliography on retrial queues: Progress in 2000--2009, Mathematical and Computer Modelling, 51 (2010), 1071-1081.  doi: 10.1016/j.mcm.2009.12.011. [4] Y. Aviv and A. Federgruen, The value iteration method for countable state markov decision processes, Operations Research Letters, 24 (1999), 223-234.  doi: 10.1016/S0167-6377(99)00015-2. [5] S. Benjaafar, J.-P. Gayon and S. Tepe, Optimal control of a production--inventory system with customer impatience, Operations Research Letters, 38 (2010), 267-272.  doi: 10.1016/j.orl.2010.03.008. [6] L. Breuer, Threshold policies for controlled retrial queues with heterogeneous servers, Annals of Operations Research, 141 (2006), 139-162.  doi: 10.1007/s10479-006-5297-5. [7] R. Cavazos-Cadena and L. I. Sennott, Comparing recent assumptions for the existence of average optimal stationary policies, Operations Research Letters, 11 (1992), 33-37.  doi: 10.1016/0167-6377(92)90059-C. [8] E. B. Çil, F. Karaesmen and E. L. Örmeci, Dynamic pricing and scheduling in a multi-class single-server queueing system, Queueing Systems, 67 (2011), 305-331.  doi: 10.1007/s11134-011-9214-5. [9] E. B. Çil, E. L. Örmeci and F. Karaesmen, Effects of system parameters on the optimal policy structure in a class of queueing control problems, Queueing Systems, 61 (2009), 273-304.  doi: 10.1007/s11134-009-9109-x. [10] S. D. Flapper, J.-P. Gayon and L. L. Lim, On the optimal control of manufacturing and remanufacturing activities with a single shared server, European Journal of Operational Research, 234 (2014), 86-98.  doi: 10.1016/j.ejor.2013.10.049. [11] D. Gaver, P. Jacobs and G. Latouche, Finite birth-and-death models in randomly changing environments, Advances in Applied Probability, 16 (1984), 715-731.  doi: 10.1017/S0001867800022916. [12] B. Hajek, Optimal control of two interacting service stations, IEEE Transactions on Automatic Control, 29 (1984), 491-499.  doi: 10.1109/TAC.1984.1103577. [13] W. E. Helm and K.-H. Waldmann, Optimal control of arrivals to multiserver queues in a random environment, Journal of Applied Probability, 21 (1984), 602-615.  doi: 10.1017/S0021900200028795. [14] D. P. Heyman, Optimal operating policies for M/G/1 queuing systems, Operations Research, 16 (1968), 362-382.  doi: 10.1287/opre.16.2.362. [15] G. Koole, Monotonicity in Markov Reward and Decision Chains: Theory and Applications vol. 1, Now Publishers Inc, 2007. doi: 10.1561/0900000002. [16] B. K. Kumar and J. Raja, On multiserver feedback retrial queues with balking and control retrial rate, Annals of Operations Research, 141 (2006), 211-232.  doi: 10.1007/s10479-006-5300-1. [17] B. K. Kumar, R. Rukmani and V. Thangaraj, On multiserver feedback retrial queue with finite buffer, Applied Mathematical Modelling, 33 (2009), 2062-2083.  doi: 10.1016/j.apm.2008.05.011. [18] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, New York, 1994. [19] L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems vol. 504, John Wiley & Sons, New York, 1999. [20] J.-D. Son, Optimal admission and pricing control problem with deterministic service times and sideline profit, Queueing Systems, 60 (2008), 71-85.  doi: 10.1007/s11134-008-9087-4. [21] S. Stidham Jr and R. Weber, A survey of markov decision models for control of networks of queues, Queueing Systems, 13 (1993), 291-314.  doi: 10.1007/BF01158935. [22] H. C. Tijms, Stochastic Models: An Algorithmic Approach vol. 303, John Wiley & Sons Inc, 1994. [23] T. Van Do, An efficient computation algorithm for a multiserver feedback retrial queue with a large queueing capacity, Applied Mathematical Modelling, 34 (2010), 2272-2278.  doi: 10.1016/j.apm.2009.10.025. [24] J. Wu and Z. Lian, Analysis of the $M_{1}, M_{2}$/G/1 G-queueing system with retrial customers, Nonlinear Analysis: Real World Applications, 14 (2013), 365-382.  doi: 10.1016/j.nonrwa.2012.06.009. [25] J. Wu, J. Wang and Z. Liu, A discrete-time Geo/G/1 retrial queue with preferred and impatient customers, Applied Mathematical Modelling, 37 (2013), 2552-2561.  doi: 10.1016/j.apm.2012.06.011. [26] S. Yoon and M. E. Lewis, Optimal pricing and admission control in a queueing system with periodically varying parameters, Queueing Systems, 47 (2004), 177-199.  doi: 10.1023/B:QUES.0000035313.20223.3f. [27] X. Zhang, J. Wang and T. Van Do, Threshold properties of the M/M/1 queue under T-policy with applications, Applied Mathematics and Computation, 261 (2015), 284-301.  doi: 10.1016/j.amc.2015.03.109.

show all references

##### References:
 [1] H.-S. Ahn, I. Duenyas and M. E. Lewis, Optimal control of a two-stage tandem queuing system with flexible servers, Probability in the Engineering and Informational Sciences, 16 (2002), 453-469.  doi: 10.1017/S0269964802164047. [2] A. S. Alfa and K. S. Isotupa, An M/PH/k retrial queue with finite number of sources, Computers & Operations Research, 31 (2004), 1455-1464.  doi: 10.1016/S0305-0548(03)00100-X. [3] J. R. Artalejo, Accessible bibliography on retrial queues: Progress in 2000--2009, Mathematical and Computer Modelling, 51 (2010), 1071-1081.  doi: 10.1016/j.mcm.2009.12.011. [4] Y. Aviv and A. Federgruen, The value iteration method for countable state markov decision processes, Operations Research Letters, 24 (1999), 223-234.  doi: 10.1016/S0167-6377(99)00015-2. [5] S. Benjaafar, J.-P. Gayon and S. Tepe, Optimal control of a production--inventory system with customer impatience, Operations Research Letters, 38 (2010), 267-272.  doi: 10.1016/j.orl.2010.03.008. [6] L. Breuer, Threshold policies for controlled retrial queues with heterogeneous servers, Annals of Operations Research, 141 (2006), 139-162.  doi: 10.1007/s10479-006-5297-5. [7] R. Cavazos-Cadena and L. I. Sennott, Comparing recent assumptions for the existence of average optimal stationary policies, Operations Research Letters, 11 (1992), 33-37.  doi: 10.1016/0167-6377(92)90059-C. [8] E. B. Çil, F. Karaesmen and E. L. Örmeci, Dynamic pricing and scheduling in a multi-class single-server queueing system, Queueing Systems, 67 (2011), 305-331.  doi: 10.1007/s11134-011-9214-5. [9] E. B. Çil, E. L. Örmeci and F. Karaesmen, Effects of system parameters on the optimal policy structure in a class of queueing control problems, Queueing Systems, 61 (2009), 273-304.  doi: 10.1007/s11134-009-9109-x. [10] S. D. Flapper, J.-P. Gayon and L. L. Lim, On the optimal control of manufacturing and remanufacturing activities with a single shared server, European Journal of Operational Research, 234 (2014), 86-98.  doi: 10.1016/j.ejor.2013.10.049. [11] D. Gaver, P. Jacobs and G. Latouche, Finite birth-and-death models in randomly changing environments, Advances in Applied Probability, 16 (1984), 715-731.  doi: 10.1017/S0001867800022916. [12] B. Hajek, Optimal control of two interacting service stations, IEEE Transactions on Automatic Control, 29 (1984), 491-499.  doi: 10.1109/TAC.1984.1103577. [13] W. E. Helm and K.-H. Waldmann, Optimal control of arrivals to multiserver queues in a random environment, Journal of Applied Probability, 21 (1984), 602-615.  doi: 10.1017/S0021900200028795. [14] D. P. Heyman, Optimal operating policies for M/G/1 queuing systems, Operations Research, 16 (1968), 362-382.  doi: 10.1287/opre.16.2.362. [15] G. Koole, Monotonicity in Markov Reward and Decision Chains: Theory and Applications vol. 1, Now Publishers Inc, 2007. doi: 10.1561/0900000002. [16] B. K. Kumar and J. Raja, On multiserver feedback retrial queues with balking and control retrial rate, Annals of Operations Research, 141 (2006), 211-232.  doi: 10.1007/s10479-006-5300-1. [17] B. K. Kumar, R. Rukmani and V. Thangaraj, On multiserver feedback retrial queue with finite buffer, Applied Mathematical Modelling, 33 (2009), 2062-2083.  doi: 10.1016/j.apm.2008.05.011. [18] M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, New York, 1994. [19] L. I. Sennott, Stochastic Dynamic Programming and the Control of Queueing Systems vol. 504, John Wiley & Sons, New York, 1999. [20] J.-D. Son, Optimal admission and pricing control problem with deterministic service times and sideline profit, Queueing Systems, 60 (2008), 71-85.  doi: 10.1007/s11134-008-9087-4. [21] S. Stidham Jr and R. Weber, A survey of markov decision models for control of networks of queues, Queueing Systems, 13 (1993), 291-314.  doi: 10.1007/BF01158935. [22] H. C. Tijms, Stochastic Models: An Algorithmic Approach vol. 303, John Wiley & Sons Inc, 1994. [23] T. Van Do, An efficient computation algorithm for a multiserver feedback retrial queue with a large queueing capacity, Applied Mathematical Modelling, 34 (2010), 2272-2278.  doi: 10.1016/j.apm.2009.10.025. [24] J. Wu and Z. Lian, Analysis of the $M_{1}, M_{2}$/G/1 G-queueing system with retrial customers, Nonlinear Analysis: Real World Applications, 14 (2013), 365-382.  doi: 10.1016/j.nonrwa.2012.06.009. [25] J. Wu, J. Wang and Z. Liu, A discrete-time Geo/G/1 retrial queue with preferred and impatient customers, Applied Mathematical Modelling, 37 (2013), 2552-2561.  doi: 10.1016/j.apm.2012.06.011. [26] S. Yoon and M. E. Lewis, Optimal pricing and admission control in a queueing system with periodically varying parameters, Queueing Systems, 47 (2004), 177-199.  doi: 10.1023/B:QUES.0000035313.20223.3f. [27] X. Zhang, J. Wang and T. Van Do, Threshold properties of the M/M/1 queue under T-policy with applications, Applied Mathematics and Computation, 261 (2015), 284-301.  doi: 10.1016/j.amc.2015.03.109.
Optimal thresholds vs. $h$ for $\lambda=0.9, \mu=0.1, \xi=0.8, r=40, c=35$
Optimal thresholds vs. $r$ for $\lambda=1, \mu=1, \xi=0.6, h=0.6, c=30$
Optimal thresholds vs. $c$ for $\lambda=1, \mu=1, \xi=0.5, h=0.6, r=40$
Optimal thresholds and average cost vs. $\lambda$ for $\mu=1, \xi=0.6, h=0.8, r=30, c=28$
 arrival rate $\lambda$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.8 (4, 3, 8.1688) (17, 14, 0.7604) (22, 18, 0.2148) (22, 18, 0.0699) 0.85 (3, 3, 9.3409) (14, 11, 1.2510) (16, 12, 0.4664) (16, 12, 0.2033) 0.9 (3, 2, 10.5053) (11, 8, 1.9359) (12, 8, 0.9216) (10, 7, 0.5109) 0.95 (3, 2, 11.6892) (9, 6, 2.7854) (8, 5, 1.5937) (7, 4, 1.0490) 1 (3, 2, 12.9001) (7, 5, 3.7574) (6, 3, 2.4441) (4, 1, 1.8044) 1.05 (3, 2, 14.1340) (6, 4, 4.8209) (4, 2, 3.4291) (2, 1, 2.7468)
 arrival rate $\lambda$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.8 (4, 3, 8.1688) (17, 14, 0.7604) (22, 18, 0.2148) (22, 18, 0.0699) 0.85 (3, 3, 9.3409) (14, 11, 1.2510) (16, 12, 0.4664) (16, 12, 0.2033) 0.9 (3, 2, 10.5053) (11, 8, 1.9359) (12, 8, 0.9216) (10, 7, 0.5109) 0.95 (3, 2, 11.6892) (9, 6, 2.7854) (8, 5, 1.5937) (7, 4, 1.0490) 1 (3, 2, 12.9001) (7, 5, 3.7574) (6, 3, 2.4441) (4, 1, 1.8044) 1.05 (3, 2, 14.1340) (6, 4, 4.8209) (4, 2, 3.4291) (2, 1, 2.7468)
Optimal thresholds and average cost vs. $\mu$ for $\lambda=0.8, \xi=0.6, h=0.8, r=30, c=28$
 service rate $\mu$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.75 (3, 2, 10.7715) (5, 3, 4.1044) (3, 1, 2.9138) (2, 1, 2.3607) 0.8 (3, 2, 10.1999) (6, 4, 3.1936) (5, 2, 2.0532) (3, 1, 1.4776) 0.85 (3, 2, 9.6629) (8, 6, 2.3784) (7, 4, 1.3332) (6, 3, 0.8377) 0.9 (3, 2, 9.1583) (11, 8, 1.6889) (11, 8, 0.7815) (10, 6, 0.4095) 0.95 (3, 3, 8.6564) (14, 11, 1.1467) (16, 12, 0.4167) (15, 11, 0.1738) 1 (4, 3, 8.1688) (17, 14, 0.7604) (22, 18, 0.2148) (22, 18, 0.0699)
 service rate $\mu$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.75 (3, 2, 10.7715) (5, 3, 4.1044) (3, 1, 2.9138) (2, 1, 2.3607) 0.8 (3, 2, 10.1999) (6, 4, 3.1936) (5, 2, 2.0532) (3, 1, 1.4776) 0.85 (3, 2, 9.6629) (8, 6, 2.3784) (7, 4, 1.3332) (6, 3, 0.8377) 0.9 (3, 2, 9.1583) (11, 8, 1.6889) (11, 8, 0.7815) (10, 6, 0.4095) 0.95 (3, 3, 8.6564) (14, 11, 1.1467) (16, 12, 0.4167) (15, 11, 0.1738) 1 (4, 3, 8.1688) (17, 14, 0.7604) (22, 18, 0.2148) (22, 18, 0.0699)
Optimal thresholds and average cost vs. $\xi$ for $\lambda=1, \mu=1, h=0.3, r=35, c=33$
 retrial rate $\xi$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.1 (1, 1, 17.0052) (5, 3, 4.9228) (10, 6, 2.4449) (15, 9, 1.6616) 0.12 (2, 1, 16.8627) (6, 3, 4.7616) (12, 7, 2.3554) (16, 10, 1.6257) 0.14 (2, 1, 16.7071) (6, 4, 4.6076) (13, 8, 2.2816) (17, 10, 1.6030) 0.16 (2, 2, 16.5549) (7, 4, 4.4670) (14, 9, 2.2220) (17, 11, 1.5898) 0.18 (2, 2, 16.4067) (8, 5, 4.3381) (15, 10, 2.1744) (17, 11, 1.5828) 0.2 (2, 2, 16.2641) (8, 5, 4.2173) (15, 10, 2.1358) (18, 11, 1.5809)
 retrial rate $\xi$ Optimal thresholds and average cost $(m, n, g^{*})$ $K=1$ $K=5$ $K=10$ $K=15$ 0.1 (1, 1, 17.0052) (5, 3, 4.9228) (10, 6, 2.4449) (15, 9, 1.6616) 0.12 (2, 1, 16.8627) (6, 3, 4.7616) (12, 7, 2.3554) (16, 10, 1.6257) 0.14 (2, 1, 16.7071) (6, 4, 4.6076) (13, 8, 2.2816) (17, 10, 1.6030) 0.16 (2, 2, 16.5549) (7, 4, 4.4670) (14, 9, 2.2220) (17, 11, 1.5898) 0.18 (2, 2, 16.4067) (8, 5, 4.3381) (15, 10, 2.1744) (17, 11, 1.5828) 0.2 (2, 2, 16.2641) (8, 5, 4.2173) (15, 10, 2.1358) (18, 11, 1.5809)
 [1] Tuan Phung-Duc, Wouter Rogiest, Sabine Wittevrongel. Single server retrial queues with speed scaling: Analysis and performance evaluation. Journal of Industrial and Management Optimization, 2017, 13 (4) : 1927-1943. doi: 10.3934/jimo.2017025 [2] Vladimir Turetsky, Valery Y. Glizer. Optimal decision in a Statistical Process Control with cubic loss. Journal of Industrial and Management Optimization, 2022, 18 (4) : 2903-2926. doi: 10.3934/jimo.2021096 [3] Zhanyou Ma, Wuyi Yue, Xiaoli Su. Performance analysis of a Geom/Geom/1 queueing system with variable input probability. Journal of Industrial and Management Optimization, 2011, 7 (3) : 641-653. doi: 10.3934/jimo.2011.7.641 [4] Zhanqiang Huo, Wuyi Yue, Naishuo Tian, Shunfu Jin. Performance evaluation for the sleep mode in the IEEE 802.16e based on a queueing model with close-down time and multiple vacations. Journal of Industrial and Management Optimization, 2009, 5 (3) : 511-524. doi: 10.3934/jimo.2009.5.511 [5] Madhu Jain, Sudeep Singh Sanga. Admission control for finite capacity queueing model with general retrial times and state-dependent rates. Journal of Industrial and Management Optimization, 2020, 16 (6) : 2625-2649. doi: 10.3934/jimo.2019073 [6] Yi Peng, Jinbiao Wu. On the $BMAP_1, BMAP_2/PH/g, c$ retrial queueing system. Journal of Industrial and Management Optimization, 2021, 17 (6) : 3373-3391. doi: 10.3934/jimo.2020124 [7] Sin-Man Choi, Ximin Huang, Wai-Ki Ching. Minimizing equilibrium expected sojourn time via performance-based mixed threshold demand allocation in a multiple-server queueing environment. Journal of Industrial and Management Optimization, 2012, 8 (2) : 299-323. doi: 10.3934/jimo.2012.8.299 [8] Wendai Lv, Siping Ji. Atmospheric environmental quality assessment method based on analytic hierarchy process. Discrete and Continuous Dynamical Systems - S, 2019, 12 (4&5) : 941-955. doi: 10.3934/dcdss.2019063 [9] Yuan Zhao, Wuyi Yue. Performance evaluation and optimization of cognitive radio networks with adjustable access control for multiple secondary users. Journal of Industrial and Management Optimization, 2019, 15 (1) : 1-14. doi: 10.3934/jimo.2018029 [10] Omer Faruk Yilmaz, Mehmet Bulent Durmusoglu. A performance comparison and evaluation of metaheuristics for a batch scheduling problem in a multi-hybrid cell manufacturing system with skilled workforce assignment. Journal of Industrial and Management Optimization, 2018, 14 (3) : 1219-1249. doi: 10.3934/jimo.2018007 [11] Zhanyou Ma, Wenbo Wang, Linmin Hu. Performance evaluation and analysis of a discrete queue system with multiple working vacations and non-preemptive priority. Journal of Industrial and Management Optimization, 2020, 16 (3) : 1135-1148. doi: 10.3934/jimo.2018196 [12] Heyu Xu, Fangzhou Fu. A data-driven configurability evaluation method for spacecraft control systems. Mathematical Foundations of Computing, 2022  doi: 10.3934/mfc.2022022 [13] Dequan Yue, Wuyi Yue. Block-partitioning matrix solution of M/M/R/N queueing system with balking, reneging and server breakdowns. Journal of Industrial and Management Optimization, 2009, 5 (3) : 417-430. doi: 10.3934/jimo.2009.5.417 [14] Mathias Staudigl. A limit theorem for Markov decision processes. Journal of Dynamics and Games, 2014, 1 (4) : 639-659. doi: 10.3934/jdg.2014.1.639 [15] Zhanyou Ma, Pengcheng Wang, Wuyi Yue. Performance analysis and optimization of a pseudo-fault Geo/Geo/1 repairable queueing system with N-policy, setup time and multiple working vacations. Journal of Industrial and Management Optimization, 2017, 13 (3) : 1467-1481. doi: 10.3934/jimo.2017002 [16] Raina Raj, Vidyottama Jain. Optimization of traffic control in $MMAP\mathit{[2]}/PH\mathit{[2]}/S$ priority queueing model with $PH$ retrial times and the preemptive repeat policy. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022044 [17] Sung-Seok Ko. A nonhomogeneous quasi-birth-death process approach for an $(s, S)$ policy for a perishable inventory system with retrial demands. Journal of Industrial and Management Optimization, 2020, 16 (3) : 1415-1433. doi: 10.3934/jimo.2019009 [18] Bara Kim. Stability of a retrial queueing network with different classes of customers and restricted resource pooling. Journal of Industrial and Management Optimization, 2011, 7 (3) : 753-765. doi: 10.3934/jimo.2011.7.753 [19] Ning Chen, Yan Xia Zhao, Jia Yang Dai, Yu Qian Guo, Wei Hua Gui, Jun Jie Peng. Hybrid modeling and distributed optimization control method for the iron removal process. Journal of Industrial and Management Optimization, 2022  doi: 10.3934/jimo.2022003 [20] Hamed Fazlollahtabar, Mohammad Saidi-Mehrabad. Optimizing multi-objective decision making having qualitative evaluation. Journal of Industrial and Management Optimization, 2015, 11 (3) : 747-762. doi: 10.3934/jimo.2015.11.747

2021 Impact Factor: 1.411