Based on MPU, we can build a total hierarchy architecture for MPC system: from MPU, to MPU system on one motherboard, to MPC cluster.<\/p>\n\n\n\n
Herein, CPU can be CGPU which combines CPU and GPU. MPC is Memory Parallel Computing. MPU is Memory Parallel processing Unit.<\/p>\n\n\n\n
MPU sound much better and consistent with CPU\/GPU, so I change all names including PCMem be based on MPU, and MPU = MPU processor + MPU memory.<\/p>\n\n\n\n
One MPC cluster in MPC total architecture can be formed by 2 following layers:<\/p>\n\n\n\n
layer-1) MPU system: MPU to CPU on one MPU motherboard<\/p>\n\n\n\n
Structure: MPU memory unit of each MPU – <MPU internal bus> – MPU processor of the MPU – <MPU external bus> – MPU chip – <MPU PCB bus> – MPU PCB I\/O – <MPU motherboard bus> – MPU controllor – CPU or GPU or CGPU.<\/p>\n\n\n\n
A MPU bus forms a single broadcast domain – a MPU domain, and an MPU domain includes CPU, GPU, and MPUs which all share the same broadcasting on the bus.<\/p>\n\n\n\n
The MPU controller on a MPU motherboard bus can be separated from or integrated in CPU or GPU or CGPU.<\/p>\n\n\n\n
It can be designed as that DRAM connects to CPU in a MPU domain through a separated motherboard bus not the MPU motherboard bus, and it can also be designed as that DRAM connects to CPU in a MPC domain through the MPC motherboard bus.<\/p>\n\n\n\n
One MPU domain includes only one MPU motherboard bus and at least one CPU (CGPU).<\/p>\n\n\n\n
Multiple MPU domains (including buses and CPUs) can be integerated on one MPU motherboard with a shared DRAM through a separated DRAM bus.<\/p>\n\n\n\n
Multiple MPU domains on one MPU motherboard can form one broadcast domain by connecting their motherboard buses.<\/p>\n\n\n\n
A universal reduction tree on one MPC motherboard can span from inside each MPU to all MPC domains on the motherboard.<\/p>\n\n\n\n
layer-2) From motherboard to MPC cluster<\/p>\n\n\n\n
Multiple MPU domains on different motherboards can connect together by outside link\/server\/switch to form a MPC cluster, and this MPC cluster forms one broadcast domain combined by the multiple MPU domains.<\/p>\n\n\n\n
The MPC cluster could have a reduction tree for sum & max, which extends the universal reduction tree in each MPU motherboard, in each MPU domain, in and outside each MPU. (PS: by the way, reduction tree is separated and independent from bus, you guys know this basics, right? )<\/p>\n\n\n\n
The different MPU domains in one MPC cluster can do parallel read\/write to improve the read\/write speed of MPU and MPU memories.<\/p>\n\n\n\n
MPC cluster can be flexible and dynamic for all MPU domains connected together to be one MPC cluster or multiple parallel MPC clusters or even a hierarchy of multiple tiers of MPC clusters, which can adapt to requirements for different models, tasks, context sizes and broadcast domains. <\/p>\n\n\n\n
Through MPC cluster, more MPU sticks on separated motherboards can work together to contain larger memory space for longer context with higher read\/write speed of MPUs.<\/p>\n","protected":false},"excerpt":{"rendered":"
Based on MPU, we can build a total hierarchy architecture for MPC system: from MPU, to MPU system on one motherboard, to MPC cluster. Herein,…<\/p>\n