physics pixel has 2 implications and PIM(IMC) will be revolutionary for AI

Below is about physics pixel’s 2 implications and PIM(IMC) is revolutionary for AI.

1. Physics pixel has 2 implications or 2 layers of meanings

1.1) physics pixel introduced a physics world model or data structure:

> Space as continuous objects: all space is filled with objects completely, like outdoor space is an object filled with air and outerspace is an object of vacuum,
> Interface between objects: an interface is between 2 adjacent objects, for example there is an interface between outdoor space and outerspace,
> Surfaces form interface: an object has surfaces, 2 surfaces of 2 adjacent objects form an interface between the two objects,
> Class membership: objects, surface and interface may belong to classes (a cat belong to cat class),
> Hierarchy tree decomposition: Space decomposes into a hierarchical parent/child structure for objects, classes, surfaces, and interfaces – object hierarchy (e.g., a surface is a child/part of a parent object), class hierarchy (a car class is a child of vehicle class), surface hierarchy (e.g., the front surface is a child/part of the whole surface), etc.
> Entity Parameters: all items above (object, surface, interface, class) may include its own id and physics parameters, like an object includes parameters of object id, class id, parent object id, weight/mass, velocity vector, rotation vector, etc, and like an surface includes parameters of surface id, parent object id, class id, friction coefficient, material id, hardness, etc.
> Coordinate systems: this world model can be represented in one or multiple different coordinate systems like rectangular coordinate, pixel ray perspective coordinate of a camera like sensor, or spherical coordinate,
> Mapping between different coordinate systems: different coordinate systems of this world model can be mapped into each other, so in video generating app the perspective ray coordinates and corresponding RGB can be generated from preset world of rectangular coordinates, and in robot or autodriving app the rectangular coordinates can be generated from perspective ray coordinates of cameras to let the AI of robot/autodriving understand the real world physics better. >

1.2) physics pixel introduced a visual physics based on ray perspective view of pixels

> Pixel Ray Projection: each pixel of a camera (or camera like sensor) form a ray view from the pixel,
> Ray-Interface Intersection form physics pixel: each intersection point between the ray view of a pixel and an interface of 2 objects form a physics pixel,
> Each Physics pixel’s related entities: each physics pixel corresponds to an interface, 2 objects(near/far objects) , 2 surfaces(near/far) of the 2 objects,
> Physics pixel’s related parameters: each physics pixel includes depth of the physics pixel from camera, RGBA, and parameters of the pixel, the near/far objects, the near/far surfaces , the interface.
> One pixel may correspond to multiple physics pixels: the multiple physics pixels are on different interfaces penetrated by the ray view of the pixel, and one of the multiple physics pixels can be seen or unseen by the camera for the different interfaces can be transparent or not,
> Reflection’s image physics pixel: there could be an image physics pixel, which represents an image of an real reflected physics pixel which is reflected by an real reflecting physics pixel, and the image physics pixel and the real reflecting physics pixel are on the same ray view of the same pixel, etc.

2. PIM/IMC (process in memory or in memory compute) is revolutionary for AI especially for edge AI

In present computer, data must be read unit by unit from memory units (like 512 bits per unit) through shared data bus (like bus bandwidth = 512 bits) shared by many memory units to send to computing units (CPU/GPU) to compute, in which present bottleneck is not the speed of computing but the limit of data bus speed of memory.

PIM/IMC (process in memory/in memory compute) integrate simple calculation logic circuits to each unit (or several units) of memory, so that if data in each memory unit need to do parallel computation which doesnt need sequential compute of different memory units, the data in all memory units can be processed or calculated parallely and simultaneously without being read through shared memory data bus, which eliminates the memory data bus bottleneck completely.

What a coincidence: Transformer’s prefix calculation {QKV=X@Wqkv}, KV calculation {M=softmax(Q@K.T)@V}, Q, K, V and softmax are exactly parallel computation requiring no sequential dependencies across memory units, in which, different X vectors are multiplied by same Wq,k,v, and K.T and V are multiplied by same Q, and softmax’s exp/sum/dividing can be parallel too.

So PIM/IMC can integrate prefix calculation {QKV=X@Wqkv}, KV calculation {M=softmax(Q@K.T)@V} and softmax into each memory unit, which will make the transformer’s latency is not related/correlated to the window size of prefix and context. For example, the speed of prefix and decode will be virtually identical for 1k, 1M, 1B and 1T of prefix and context size for PIM/IMC memory!

PIM/IMC is revolutionary for AI especially for edge AI like robot and autodriving which need to process huge bits of visual signal instantly.