Your “noise” is always someone else’s precious and irreplaceable data – preserve it carefully .
replying to https://x.com/docmilanfar/status/1866671417443619080
Your “noise” is always someone else’s precious and irreplaceable data – preserve it carefully.
If you “remove noise” you must preserve the noise in a separate data stream. So when, later, as it will happen, you find your cleaning algorithm is way too simplistic, and essentially blocks all future super-resolution efforts — you can go back to the raw data and try again.
Most efforts at removing “noise” on the whole Internet lead to cheap quick reduction in bandwidth, at the cost of unverifiable and irreversible loss of information. It is simple – “always have a backup”. In case after case, some tired group under pressure makes a fix to get something out the door, and few minutes time saved for the designer cascades in billion fold tiny mistakes and inefficiencies.
I made a noise generator with a unique spectrum of sources and patterns. My son could not hear its beauty and depth, simply because his noise cancelling earphones were removing anything not in a narrow band of “approved speech sounds”. One person or group can cause such filtering to be inserted into systems so they are completely blind to rich universes of new signals. You could say the system was “designed to be blind to anything not in a narrow spectrum of the builders experiences “.
Consider a baby monitor that cannot recognize or anticipate gasping sounds, so the child dies because the designers have too narrow a concept of signals that might be important and understandable to any human parent. An update in the algorithm later is wasted because the device is not designed with lossless and open methods.