A self-recursive training on seed frames labeled and on pure original flow without label

Last updated on October 29, 2025

A self-recursive training on seed frame labeled and on pure original flow without label is:

First, pretrain the model (the prime model with physics pixel) with seed frames (or frame flow) of visual signal or other sensor signal or combination of different sensor signals.

The seed frames include typical scenes which include as many kinds of objects as possible, and the seed frames include original signal of visual or other sensor, and the seed frames are labelled by selected physics parameters of physics pixel like object name(near/far), 3D points cloud, mapping relationship for fusion of different kind senor signals, etc.

This pretraining may also include text training which include hierarch class of objects and other typical text.

Second, train the model on huge unlabeled original signal frame flow of visual or other signals or combined signals:

1) based on pretraining, let the model estimate all selected physics parameters of physics pixels for the orignal signal in the start frames of a flow, and the selected physics parameters include the name of object(near/far), 3D points cloud, mapping relation in fusion of different kind sensor signal, etc;

2) let the model predict all data of next frames based on all data of the start frames or present frames in which the all data include original signal and selected physics parameters;

3) use the differece between the real original signal of next frames and predicted signal of frames to adjust weights of the model (like the gradient-based optimization to change weight one by one to get the partial derivative to adjust the weight to approximate real frames);

4) go back to step 2), if the predicted signal converge to real signla to next step, if not converge in a set number of iteration then recalculate the physics parameters of the present frames and go back to step 2);

5) set next frames as present frames and go back to step 2).

During the sencond step, periodically reinfore with the pretraining stuff or other typical training data labeled.

This training may be used for the prime model with physics pixel which is based on transformer or diffusion or hybrid of above two.

If the training data of original signal flow is huge enough, once backpropagation may outperform the recursive in second step for better marginal effect, but the recursive share the same nature as diffusion on finer subtle pattern abstraction from same data, so you may need to balance or trade off or experiment on the number of the recursive.