Network, security

Beyond 5G無線通信システム

Beyond 5G Wireless Communication System

サブ6GHz及びミリ波帯を共用するBeyond 5Gネットワークの為の深層強化学習を活用したアクセスポイント選択法

Deep Reinforcement Learning-based User-to-Access Points Association in Sub-6 GHz/mmWave Beyond 5G Networks


ティハーリー ディン

Thi Ha Ly Dinh


金子 めぐみ

Megumi Kaneko



Information Systems Architecture Science Research Division Research Center|SOKENDAI|The Graduate University for Advanced Studies



本研究では,サブ6GHz及びミリ波帯を共用するBeyond 5Gネットワークの為のアクセスポイント(AP)選択法及ビームフォーミング法を考案する.提案法では各ユーザがDeep Q-Network (DQN)を活用し,ネットワーク全体の特性を最大化する最適なAP及びインターフェースを自律分散的に学習する. 計算機シミュレーションにより,従来法と比べて提案法は優れた特性を得られることを明らかにした.

We investigate the problem of joint user-to-access points(AP) association and beamforming in an integrated sub-6GHz and mmWave system, in order to fulfill the stringent Quality of Service (QoS) demands of Beyond 5G applications. We propose a method based on Deep Q-Networks (DQN), where each user learns the wireless network state and optimizes its AP association and interface requests, so as to is to maximize the long-term throughput of the system, while satisfying its QoS in a distributed manner. Computer simulation results validate the efficacy of the proposed method against conventional methods.



  1. Jia LIU より:

    Prof. Kaneko, thanks very much for your nice poster and presentation.
    I have two questions about this work:
    1) What is the essential difference between the resource allocation in 5G/beyond 5G and that in 4G? It seems that they are all formulated as a mathematical optimization problem.

    2) In this work, is there any issue regarding wireless interference? It seems to be a dense device environment.

    1. [NII]Thi Ha Ly Dinh and Megumi Kaneko より:

      Dear Dr. Jia Liu,

      Thank you very much for your interest in our poster and for your questions. Here are our replies.
      1. While 4G was focused on the provision of broadband mobile access, 5G/beyond 5G systems are expected to guarantee more diverse and stringent quality of service requirements (not only higher data rate but also better delay, reliability and connectivity performances). To do so, a variety of new technologies will be integrated, e.g., network densification, cloud/fog radio access networks, millimeter waves and massive multiple input multiple output (MIMO) to name a few.
      Although radio resource allocation problems may still be formulated as mathematical optimization problems, centralized resolution approaches will no longer be possible due to the increased complexity of the network and application requirements. This is why we propose here a deep reinforcement learning-based approach to solve this problem, where the conventional centralized optimization is shifted towards a distributed method by exploiting the edge computing and learning capabilities of future AI-powered devices.

      2. We consider indeed a dense environment where all APs and devices share the same spectrum in both bands (sub-6GHz and mmWave) and hence interfere with each other. Our proposed method enables each device to autonomously learn the best APs/bands to request for satisfying the specific requirements of their applications, while minimizing the overall interference over time, in a distributed manner.

      Best regards,
      Thi Ha Ly Dinh and Megumi Kaneko