When first brought up physics pixel, I proposed to generate chosen physics parameters of physics pixels for only selected pixels of visual frame, like 1 pixel of each 3×3 or 4×4 pixels.
Now I realized that selected pixels method can be expanded to combined pixels, which is: a pixel of the visual frame is a raw pixel of the camera or a combined pixel formed from multiple raw pixels, in which the combined pixel includes visual information or one or more RGB values of the multiple raw pixels.
A raw pixel of the camera of a visual frame can correspond to at least one physics pixel, and a combined pixel including multiple raw pixels can also correspond to at least one physics pixel which can be called as combined phyisc pixel. Either of a physics pixel or a combined physics pixel can have its own parameters chosen for better efficiency and performance, and a raw pixel can correspond to a physics pixel and a combined physics pixel at same time or in a same visual frame as the example blow.
Below is an example. 1D=32bits.
——/table for physics pixel/——
TABLE HEADER {
8bits: table ID, (“03”)
16 bits: table type, (8 bits=“physics pixel” + 8bits= T “sensor type”)
1D: predicted time, (time that the predicted parameters were predicted for )
1D: time to predict, (time that the parameters to predict will be predicted for)
16bits: rows, (?)
16bits: columns, (?)
}
/chosen parameters for a combined physics pixel corresponding to all following 9 or 16 pixels of a visual frame/
{
??D: chosen parameters of a combined physics pixel corresponding to all following 9 or 16 pixels of a visual frame.
}
for (each of 9 or16 pixels of a visual frame) {
3.5D: TNC+XYZ, (C= “01” means sensor’s own perspective coordinate system, XYZ is the position of physics pixel in perspective coordinate system of the camera, X=16bits is physics pixel‘s horizontal ordinal pixel number (from 1) in the 2D frame of the camera, Y=16bits is physics pixel‘s vertical ordinal pixel number (from 1) in the 2D frame of the camera, Z=48bits is distance from the physics pixel to the camera lens in micrometer, and “Z=0” means raw pixel received by camera and “Z=1” means points on camera lens)
2D: RGB, (RGB is raw 2D signal for pixel “X, Y” which is same for all physics pixel”X, Y, ?” including “X, Y, 0”)
2D: RGBA, (RGBA at “X,Y,0” is completely transparent “0, 0, 0, 0”, RGBA at camera lens interface is on “X, Y, 1” to be distinct from raw RGB, and for any invisible physics pixel RGBA=”-1,-1,-1,-1”)
3.75D: C+X’Y’Z’, (mapping 3D point of the physics pixel, C=”C0” unified rectangular coordinate system, X’, Y’ and Z’=48bits in unit micrometer)
1D: direction in spherical coordinates, (horizontal angle+vertical angle in “C0” system, which is the direction perpendicular to the interface which is between near object and far object and which the physics pixel is on)
0.5D: interface ID, (in labeled training this is labeled, in reference this is generated by the model)
1D: pressure,
1D: interface ID, (Optional )
?D: chosen parameters of the point of the physics pixel on the interface, (Optional, may includes overlap number of the interface, refraction index, etc)
1D: near surface ID, (for object like outdoor space, this surface ID could be “0” for no meaning )
1D: near object ID,
??D: chosen parameters of the point of the physics pixel on the near surface, (this field is optional, which may include temperature, velocity, rotation vector, hardness, density, material or no parameter at all)
1D: far surface ID,
1D: far object ID,
??D: chosen parameters of the point of the physics pixel on the far surface.
}
Be First to Comment