1. MPU broadcasting<\/p>\n\n\n\n
On a MPU motherboard bus, not only a MPU controller (previously called memory controller) can generate partial scope broadcasting, but a MPU can do too when MPU controller authorizes the broadcast to the MPU.<\/p>\n\n\n\n
Here is an example. In FFN, all layers can be done from input to last layer as one row by one row. Each MPU can do product of a row of input hidden states which is broadcasted and a column of weights that the MPU stored, parallelly together with other MPUs. Each MPU doing a part of the same multiplication between the broadcasted hidden states row and weights of a FFN layer ( a slice of weights per MPU) can broadcast its own product result sequentially to form a row of input hidden states for next layer, when the MPU controller give the command.<\/p>\n\n\n\n
2. MPU system, and MPU = MPU processor + MPU memory<\/p>\n\n\n\n
For all this architecture is built around MPU and MPU is consistent with CPU and GPU, so now rename it as MPU system which is previously called PCMem or MPCMem.<\/p>\n\n\n\n
A MPU system includes at least a MPU motherboard, a MPU PCB and MPU chips on the PCB.<\/p>\n\n\n\n
A MPU chip includes multiple MPUs, for example, a MPU chipe includes 0.2M MPUs and 2GB memory totally in which one MPU includes 10KB memory. <\/p>\n\n\n\n
A MPU includes a MPU processor and a MPU memory unit dedicated to the MPU, and the MPU processor and the MPU memory unit are connected by a dedicated link. <\/p>\n\n\n\n
A MPU processor may have multiple cores like present CPU\/GPU. <\/p>\n\n\n\n
3. MPU controller<\/p>\n\n\n\n
Previously called memory controller is renamed as MPU controller here.<\/p>\n\n\n\n
In fact, the task of MPU controller (previously called memory controller) is managing and controlling all MPUs directly on the same MPU motherboard bus not the memory directly.<\/p>\n","protected":false},"excerpt":{"rendered":"
1. MPU broadcasting On a MPU motherboard bus, not only a MPU controller (previously called memory controller) can generate partial scope broadcasting, but a MPU…<\/p>\n