Compute gets the attention. Data movement drives the cost. Widely cited ISSCC 2014 figures show that a DRAM access consumes hundreds of times more energy than an arithmetic operation on the same data. In a multi-processor SoC, this gap shapes performance, power, and area more than core count.
Teams who succeed in custom silicon design plan data paths with the same care as compute blocks. This article walks through the architecture decisions with the most impact: workload partitioning, interconnect, memory hierarchy, coherency, power and security, and early modeling.
Start With the Workload
Architecture follows workload. Profile the target applications and identify the heavy kernels. Measure compute intensity, which is operations per byte moved. High-intensity kernels, such as dense matrix math, suit NPUs and DSPs. Low-intensity kernels, such as control code, suit CPU clusters. The roofline model shows whether a kernel hits a compute limit or a bandwidth limit.
Take a 4K camera pipeline. A 3840 by 2160 frame in YUV 4:2:0 holds about 12.4 MB. At 30 frames per second, one pass moves roughly 373 MB/s. Every stage reads and writes, so one stage costs about 750 MB/s. Five stages through DRAM approach 3.7 GB/s before any other traffic. Streaming data between blocks on chip removes much of this load.
Select the Right Interconnect
Shared buses saturate as masters multiply. Crossbars scale poorly in wiring. Network-on-chip designs, or NoC, scale better with ring, mesh, or tree topologies. Use the traffic matrix, meaning who talks to whom and how much, to select topology and place blocks in the floorplan. Keep high-bandwidth pairs close.
Protocols matter. AMBA AXI serves most I/O and accelerator traffic. AMBA CHI serves coherent CPU clusters. Quality of service, or QoS, separates latency-sensitive masters, such as CPUs and display controllers, from throughput masters, such as GPUs and NPUs. Without QoS, a heavy NPU burst starves a display controller and causes underruns on screen. Use priority levels, bandwidth regulators, and virtual channels to protect critical traffic.
Design the Memory Hierarchy
An effective memory hierarchy keeps frequently used data close to processing units, reducing memory access delays and improving system performance. Key design considerations include:
- Use layered CPU caches: Arrange cache levels to keep frequently accessed instructions and data near CPU cores, reducing repeated trips to external memory.
- Introduce a shared system-level cache: A cache shared across multiple processing masters can reduce DRAM accesses and improve data availability.
- Allocate scratchpad SRAM for accelerators: Software-managed scratchpad memory supports predictable access patterns and gives accelerators greater control over data movement.
- Apply data tiling: Divide large tensors or video frames into smaller blocks that fit within on-chip memory. This allows data to be reused multiple times before fetching additional blocks.
- Use efficient DMA engines: Scatter-gather DMA transfers move data between memory regions without requiring continuous CPU involvement, freeing processing resources.
- Reduce memory traffic through compression: Lossless frame buffer compression can lower bandwidth requirements in display and video processing paths.
- Select suitable external memory: LPDDR5 operating at 6400 MT/s on a 32-bit channel offers approximately 25.6 GB/s of peak bandwidth. Real workloads should generally target 70 to 85% of this capacity because refresh cycles, bank conflicts, and read-write transitions reduce usable bandwidth.
- Evaluate high-bandwidth memory: HBM can serve data centre processors with demanding data requirements, although it introduces higher costs and packaging complexity.
Decide on Coherency
Hardware coherency keeps caches consistent automatically, simplifying software development and reducing the risk of bugs reaching the field. However, it increases area requirements, snoop traffic, and power consumption. Software-managed coherency relies on explicit cache maintenance. Although this approach keeps hardware leaner, developers must flush and invalidate caches at the correct moments to prevent subtle data corruption.
Circuit board design services also play a role in system integration by ensuring that board-level connectivity, signal routing, and power delivery support the SoC architecture. Most designs use a hybrid approach. CPU clusters remain fully coherent, while accelerators use I/O coherency for shared buffers. Streaming blocks operate as non-coherent units with explicit handoffs. Snoop filters help control coherency traffic as core counts increase.
Manage Power and Security
Data movement dominates dynamic power in many SoCs. Split the design into power domains. Apply clock gating and dynamic voltage and frequency scaling per domain. Place always-on logic in a small domain with its own supply. Security belongs in the architecture, not in a later patch. Use an IOMMU or SMMU to isolate DMA masters. Add interconnect firewalls to separate secure and non-secure traffic. Include secure boot, a hardware root of trust, and ECC on memories for safety targets such as ISO 26262.
Model Before You Build
Architecture errors cost the most when found late. Build virtual platforms in SystemC with transaction-level models. Drive them with traffic generators and real application traces. Sweep cache sizes, NoC widths, and DRAM configurations. Compare results against the bandwidth and latency budgets. Fix bottlenecks while changes cost days, not months. Keep software teams on the same models so drivers and firmware start before silicon arrives.
Plan Beyond the Die
Silicon never works alone. Package, memory interface, and board all affect results. Developing custom silicon alongside circuit board design services and firmware under one roof helps avoid interface surprises, since one team owns pin maps, power delivery, and bring-up plans. Choose package ball maps with board routing in mind. Budget the power delivery network for peak current steps. Define thermal limits for die, package, and enclosure together.
Conclusion
A strong multi-processor SoC balances compute and data movement from the first architecture review. Profile workloads, choose an interconnect with QoS, keep data on chip, select coherency by use case, and model early. Then carry the plan outward. Match package pinout, memory routing, and the PCB board to the silicon, so bandwidth promised on paper arrives in the lab.
Tessolve is a silicon and systems engineering partner with more than 22 years of experience. They work across 12 countries with over 4,000 employees. Their teams take projects from architecture to working silicon and systems, serve over 90 percent of the top 20 semiconductor companies, and plan to double in size over four years.
