RAWd

An improved RAW image format suggestion
Michiel Kamermans, July 2008

Traditional image formats are static, one-time snapshots of a scene, specifying
 color luminescence in some color space (RGB, HSV, etc). However, storing 
 photographic images in this fashion misses out on some of the lighting
  information that can be derived given the technology used.

If instead of storing “raw” pixels, a CCD/CMOS chip is sampled several times 
during an exposure, then it is possible to record the luminescence information
per discrete unit as a delta value, rather than as an absolute pixel value. 
Using delta values rather than absolute pixel intensities means that the image
will look as intended if “rendered” using the same shutter time as the image 
was recorded with, but offers more functionality in post production, where 
the image (or even just parts of the image) can be developed using alternative 
exposure time, allowing greater control in the development phase of digital 
photographic work flow.

In addition, this image format would be smaller or equal in file size when 
compared to traditional raw formats, as the only thing that needs to be 
recorded is the delta value. This value can be represented as a positive 
integer ratio value, rather than the decimal fraction 1/ratio per time unit.
As an example: a ‘pixel’ with a luminescence that increases in steps of 7
luminescence units per (fixed) time unit, can simply be encoded as this number,
rather than being encoded as some fraction value over the full exposure time.
As such, the image’s bit space can be set dynamically, requiring three channels 
of anywhere up to whatever the number of bits used by traditional RAW images is,
before going over the equivalent RAW file size. In addition, a variable bit 
encoding scheme can be used (x+1 bit “integer with has-more bit” scheme for
instance), to even further decrease file size.

It may be necessary to pick a time unit that ensures a luminescence change of
1 can be minimally recorded, which highlights one challenge for implementing 
this image format: if the fastest shutter speed is 1/4000s, for instance, then 
this corresponds to 250 nanoseconds of exposure. If we sample the chip N times,
then under these condition, any pixel that does not increase luminescence by 1
over at least 250/N nanoseconds will have a - likely wrong - luminescence 
delta of 0, rather than some value between 0 and 1. Thus, it becomes necessary 
to pick a useful time unit and record this along with the image data. This is
easily doable on a per-camera basis, where the recording properties of the
CCD/CMOS chip and minimum shutter speed are known, but may also be possible 
on a per-image basis, where it would lead to processing overhead that might
slow down the FPS of a camera employing this RAW mode.

Of course, responsibility to indicate when using RAWd format would be unsuitable,
and switch over to normal RAW mode instead, would lie with the camera itself.
As this would only be the case at very high shutter speeds, and the fact that
with most everyone moving to CMOS (which has very quick readout time indeed,
as is evident by CMOS use in 1000+fps cameras), it is expected this to affect
an almost insignificant number of photographic events.


Advantages

Smaller file size for the same image (uncompressed) 
Virtually impossible to over-expose an image,
Pixel-for-pixel equivalent to traditional format when rendered with same exposure.
Very easy to post-manipulate image lighting without affecting other operations

Disadvantages

Multiple samplings on CCD/CMOS chip may be restricted at high shutter speeds
Requires determining a suitable time-unit either per-camera or per-image