PriCoM: Enabling Priority-based Concurrency for Multi-task CGRA

Modern SoCs schedule large workloads like AI inference to GPUs/NPUs, but fragmented tasks like data preprocessing remain inefficient on CPUs. Given the flexibility and performance advantages of CGRAs, leveraging them to accelerate fragmented tasks presents a potential solution. Unlocking this potential requires an effective CGRA tile allocation policy, which existing work implements by spatially partitioning tiles among multiple tasks to achieve parallelism. However, this approach suffers from limited performance because of frequent task scheduling of the CPU operating system and non-uniform memory access latency across tiles. CGRA Concurrency avoids those issues by allowing multiple tasks to share all tiles in different time slots. Nevertheless, prior CGRA concurrency does not consider task priority in the operating system. This limitation reduces the responsiveness of high-priority tasks and allows a single low-priority task to block all tasks during DDR memory access. To address these challenges, we propose PriCoM, a priority-based concurrency architecture for multi-task CGRA. PriCoM adopts a Multi-CGRA design with a centralized controller in each CGRA and a distributed controller in each tile. This enables high-priority task preemption within a CGRA and supports NoC forwarding to other CGRAs for parallelism, improving the responsiveness of high-priority tasks. In addition, the controllers can dynamically switch to other low-priority tasks when the current low-priority task encounters long-latency DDR access. This strategy effectively hides memory delays and increases the throughput of low-priority tasks. We evaluate PriCoM on 42 tasks of different priorities under various scheduling intervals. On average, CGRA Concurrency achieves a maximum 1.47× higher throughput than Parallel and a maximum 2.62× over the CPU’s local SIMD. PriCoM further reduces the response time of high-priority tasks by up to 0.62× and increases low-priority throughput by up to 1.31× relative to Concurrency, while adding only 0.0078mm² area and 4.62mW power overhead in 7nm technology.