Last updated on January 13, 2026
A physics pixel is on an interface, and an interface is between two adjacent surfaces. So we can use surface ID as basic parameter directly in physics pixel, with or without interface ID. A surfuce/surface ID is a part/child object of an object/object ID, so we can define surface ID directly with near/far objects together.
For some applications which dont need features of interface, there could be no interface ID but only surface ID directly in physics pixel, which has the same nature as interface ID but more direct and simple.
In training, surface ID can correlate to features on the surface in text table like road mark, text on page, etc, and interface ID can correlate to physics features of the interface. So the model can learn the exact meaning of surface ID and interface ID respectively after trained well and enough.
Dont forget to use different formats for interface ID and surface ID to help the model distinguish them better.
For each physics pixel in physics pixel table:
{
XYZ, RGBA, …
interface ID, // Optional //
chosen parameters of the point of the physics pixel on the interface, //Optional, may includes like refraction index //
near surface ID, // for object like outdoor space, this surface ID could be “0” for no meaning //
near object ID,
chosen parameters of the point of the physics pixel on the near surface,
far surface ID,
far object ID,
chosen parameters of the point of the physics pixel on the far surface,
…
}
~
~
~
PS(01/13/2026): below add a full omnitoken as example.
The OmniToken example below includes following data tables which all are orthogonal in different dimensions (here 1D=32bits).
——/global parameters for omnitoken or token header/——
1D: token ID,
1D: timestamp of borntime of token,
1D: data source, (input/external raw or output/internal generated, synthetic or not, physics complied or not, etc)
/each parameter in below tables have 2 highest bits of its all bits as Role-bits/
——/1024D for text/——
TABLE HEADER {
8 bits: table ID, (“01”)
16 bits: table type, (8 bits=“text” + 8bits=”0″)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: number of rows, (“1”)
16 bits: number of columns, (“1024”D)
}
1024D: text.
64D: vacant space. (to separate text field from other fields)
——/list for sensors in this token/——
TABLE HEADER {
16 bits: table type, (8 bits=“sensor list” + 8bits=”0″)
1D: predicted time, (invalid for this table, set as “0” )
1D: time to predict, (invalid for this table, set as “0”)
16bits: number of rows, (“3”, 3 types include camera, mm wave radar and microphone, each token includes one sensor of each type)
16bits: number of columns, (“6”D)
}
for (each of “3” sensors in this token) {
8bits: T=sensor type, (“1” means camera, “2” means mm wave radar, “3″ means microphone)
8bits: N=sensor ID, (SN of the sensor in its own type)
8bits: C=coordinate system type, (“C0”, means the unified rectangular coordinate system)
3D: position of the sensor, (position in above “C0” the unified rectangular coordinate system)
2D: direction of the sensor in spherical coordinates, (horizontal+vertical angles in above “C0” coordinate system)
16bits+16bits: resolution of seneor, (16bits is more than 65000 which is enough for seiral number of X or Y of camera, like 1920×1080 or 3840×2160; for mmwave radar, it’s XY serial number of virtual antennas)
1D: borntime or receiving time of the sensor signal,
}
16D: Vacant space.
——/table for physics pixel/——
TABLE HEADER {
8bits: table ID, (“03”)
16 bits: table type, (8 bits=“physics pixel” + 8bits= T “sensor type”)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (256=16×16)
16bits: columns, (?)
}
for (each of 256 physics pixels) {
3.5D: TNC+XYZ, (C= “01” means sensor’s own perspective coordinate system, XYZ is the position of physics pixel in perspective coordinate system of the camera, X=16bits is physics pixel‘s horizontal ordinal pixel number (from 1) in the 2D frame of the camera, Y=16bits is physics pixel‘s vertical ordinal pixel number (from 1) in the 2D frame of the camera, Z=48bits is distance from the physics pixel to the camera lens in micrometer, and “Z=0” means raw pixel received by camera and “Z=1” means points on camera lens)
2D: RGB, (RGB is raw 2D signal for pixel “X, Y” which is same for all physics pixel”X, Y, ?” including “X, Y, 0”)
2D: RGBA, (RGBA at “X,Y,0” is completely transparent “0, 0, 0, 0”, RGBA at camera lens interface is on “X, Y, 1” to be distinct from raw RGB, and for any invisible physics pixel RGBA=”-1,-1,-1,-1”)
3.75D: C+X’Y’Z’, (mapping 3D point of the physics pixel, C=”C0” unified rectangular coordinate system, X’, Y’ and Z’=48bits in unit micrometer)
1D: direction in spherical coordinates, (horizontal angle+vertical angle in “C0” system, which is the direction perpendicular to the interface which is between near object and far object and which the physics pixel is on)
0.5D: interface ID, (in labeled training this is labeled, in reference this is generated by the model)
1D: pressure,
1D: interface ID, (Optional )
?D: chosen parameters of the point of the physics pixel on the interface, (Optional, may includes overlap number of the interface, refraction index, etc)
1D: near surface ID, (for object like outdoor space, this surface ID could be “0” for no meaning )
1D: near object ID,
??D: chosen parameters of the point of the physics pixel on the near surface, (this field is optional, which may include temperature, velocity, rotation vector, hardness, density, material or no parameter at all)
1D: far surface ID,
1D: far object ID,
??D: chosen parameters of the point of the physics pixel on the far surface.
}
16D: Vacant space.
/table for objects in this token/
TABLE HEADER{
8bits: table ID, (“04”)
16 bits: table type, (8 bits=“object” + 8bits=”0″)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (4)
16bits: columns, (?)
}
for(i=1 to 4){
1D: object_i ID,
1D: object class ID,
1D: parent object ID,
1D: object material class ID, (optional)
1D: object mass,
3D: object velocity,
3D: object rotation vector,
1bit: object eatable or not,
}
16D: Vacant space.
/table for interfaces in this token – this table is Optional/
TABLE HEADER{
8bits: table ID, (“05”)
16 bits: table type, (8 bits=“interface” + 8bits=”0″)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (“4”)
16bits: columns, (?)
}
for(i=1 to 4){
??D: interface ID,
??D: object ID, (if features below are for boudary of both surfaces of the interface, object ID=”0″)
??D: interface material ID, (if not applicable then =”0″)
……
}
16D: Vacant space.
/table for mm wave radar/
TABLE HEADER{
8bits: table ID, (“06”)
16 bits: table type, (8 bits=“raw signal” + 8bits= T “mm wave radar”)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (“4”)
16bits: columns, (?)
}
/each table includes 4 ADC samples generated by this mm wave radar at one time/
For (i=1 to 4) {
??D: TNC+X’Y’, (T=”2” mm wave radar, N=SN of radar in its type, C=”C0” unified rectangular coordinate system, X’ and Y’ are both 24 bits)
32bits+32bits: I (In-phase)+Q (Quadrature).
}
16D: Vacant space.
/table for microphone/
TABLE HEADER{
8bits: table ID, (“07”)
16 bits: table type, (8 bits=“raw signal” + 8bits= T “microphone”)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (?)
16bits: columns, (?)
}
{
??D: TNC+Sound signal of 1/15 sec.
}
16D: Vacant space.
/table for control/
TABLE HEADER{
}
{
??D: user set parameters,
??D: internal sensors,
??D: an actuator – position sensor+operating time+operating value,
……
}
>< 16D: Vacant space.
/table for agent/
TABLE HEADER{
}
{}
512D: vacant
Be First to Comment