Color processing simulation

Actual image - data that comes to the sensor through the lens	The image data that reaches the sensor pixels through the Bayer color filters	Same data with the visible color removed - anyway each pixel is monochrome as it can detect only a single color
Registration of the images by the image sensor through the Bayer color mosaic filters

	Y0	Y1	Cb(4)	Cr(5)
	Y2	Y3
sensor Bayer data	Six 8x8 pixel blocks ready for compression - 4 intensity (Y) blocks and 2 differential color components (Cb and Cr)
Color processing to convert Bayer pixel data into color YCbCr 4:2:0 ready for JPEG or Ogg Theora compression

00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff	00 02 04 06 08 0a 0c 0e 01 03 05 07 09 0b 0d 0f 20 22 24 26 28 2a 2c 2e 21 23 25 27 29 2b 2d 2f 40 42 44 46 48 4a 4c 4e 41 43 45 47 49 4b 4d 4f 60 62 64 66 68 6a 6c 6e 61 63 65 67 69 6b 6d 6f 80 82 84 86 88 8a 8c 8e 81 83 85 87 89 8b 8d 8f a0 a2 a4 a6 a8 aa ac ae a1 a3 a5 a7 a9 ab ad af c0 c2 c4 c6 c8 ca cc ce c1 c3 c5 c7 c9 cb cd cf e0 e2 e4 e6 e8 ea ec ee e1 e3 e5 e7 e9 eb ed ef 10 12 14 16 18 1a 1c 1e 11 13 15 17 19 1b 1d 1f 30 32 34 36 38 3a 3c 3e 31 33 35 37 39 3b 3d 3f 50 52 54 56 58 5a 5c 5e 51 53 55 57 59 5b 5d 5f 70 72 74 76 78 7a 7c 7e 71 73 75 77 79 7b 7d 7f 90 92 94 96 98 9a 9c 9e 91 93 95 97 99 9b 9d 9f b0 b2 b4 b6 b8 ba bc be b1 b3 b5 b7 b9 bb bd bf d0 d2 d4 d6 d8 da dc de d1 d3 d5 d7 d9 db dd df f0 f2 f4 f6 f8 fa fc fe f1 f3 f5 f7 f9 fb fd ff	Y0	Y1	Cb(4)
		Y2	Y3	Cr(5)
sensor Bayer data	Re-arrangement of the pixels that groups each color componet together	Six 8x8 pixel blocks ready for compression - 4 intensity (Y) blocks and 2 dummy differential color components (Cb and Cr) needed for compatibility with the standard JPEG decoders
Preparation of the sensor raw Bayer data for efficient JPEG compression (JP4 mode)

00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff	00 02 04 06 08 0a 0c 0e 01 03 05 07 09 0b 0d 0f 20 22 24 26 28 2a 2c 2e 21 23 25 27 29 2b 2d 2f 40 42 44 46 48 4a 4c 4e 41 43 45 47 49 4b 4d 4f 60 62 64 66 68 6a 6c 6e 61 63 65 67 69 6b 6d 6f 80 82 84 86 88 8a 8c 8e 81 83 85 87 89 8b 8d 8f a0 a2 a4 a6 a8 aa ac ae a1 a3 a5 a7 a9 ab ad af c0 c2 c4 c6 c8 ca cc ce c1 c3 c5 c7 c9 cb cd cf e0 e2 e4 e6 e8 ea ec ee e1 e3 e5 e7 e9 eb ed ef 10 12 14 16 18 1a 1c 1e 11 13 15 17 19 1b 1d 1f 30 32 34 36 38 3a 3c 3e 31 33 35 37 39 3b 3d 3f 50 52 54 56 58 5a 5c 5e 51 53 55 57 59 5b 5d 5f 70 72 74 76 78 7a 7c 7e 71 73 75 77 79 7b 7d 7f 90 92 94 96 98 9a 9c 9e 91 93 95 97 99 9b 9d 9f b0 b2 b4 b6 b8 ba bc be b1 b3 b5 b7 b9 bb bd bf d0 d2 d4 d6 d8 da dc de d1 d3 d5 d7 d9 db dd df f0 f2 f4 f6 f8 fa fc fe f1 f3 f5 f7 f9 fb fd ff	Y0	Y1
		Y2	Y3
sensor Bayer data	Re-arrangement of the pixels that groups each color componet together	Four 8x8 pixel blocks ready for compression. Dummy blocks are removed as the diffrential encoding requires modification of the libjpeg anyway
Modified JP4 mode (JP4D): primary green color (G) is encoded as absolute data (Y1), three other color components use difference from G: Y0=R-G, Y1=G, Y2=G2-G, Y3=B-G

00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff	00 02 04 06 08 0a 0c 0e 01 03 05 07 09 0b 0d 0f 20 22 24 26 28 2a 2c 2e 21 23 25 27 29 2b 2d 2f 40 42 44 46 48 4a 4c 4e 41 43 45 47 49 4b 4d 4f 60 62 64 66 68 6a 6c 6e 61 63 65 67 69 6b 6d 6f 80 82 84 86 88 8a 8c 8e 81 83 85 87 89 8b 8d 8f a0 a2 a4 a6 a8 aa ac ae a1 a3 a5 a7 a9 ab ad af c0 c2 c4 c6 c8 ca cc ce c1 c3 c5 c7 c9 cb cd cf e0 e2 e4 e6 e8 ea ec ee e1 e3 e5 e7 e9 eb ed ef 10 12 14 16 18 1a 1c 1e 11 13 15 17 19 1b 1d 1f 30 32 34 36 38 3a 3c 3e 31 33 35 37 39 3b 3d 3f 50 52 54 56 58 5a 5c 5e 51 53 55 57 59 5b 5d 5f 70 72 74 76 78 7a 7c 7e 71 73 75 77 79 7b 7d 7f 90 92 94 96 98 9a 9c 9e 91 93 95 97 99 9b 9d 9f b0 b2 b4 b6 b8 ba bc be b1 b3 b5 b7 b9 bb bd bf d0 d2 d4 d6 d8 da dc de d1 d3 d5 d7 d9 db dd df f0 f2 f4 f6 f8 fa fc fe f1 f3 f5 f7 f9 fb fd ff	Y0	Y1
		Y2	Y3
sensor Bayer data	Re-arrangement of the pixels that groups each color componet together	Four 8x8 pixel blocks ready for compression
JP4 mode with high dynamic range (HDR) data - second of the 2 green components has high analog gain setting - different from that of the three other colors

00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff	00 02 04 06 08 0a 0c 0e 01 03 05 07 09 0b 0d 0f 20 22 24 26 28 2a 2c 2e 21 23 25 27 29 2b 2d 2f 40 42 44 46 48 4a 4c 4e 41 43 45 47 49 4b 4d 4f 60 62 64 66 68 6a 6c 6e 61 63 65 67 69 6b 6d 6f 80 82 84 86 88 8a 8c 8e 81 83 85 87 89 8b 8d 8f a0 a2 a4 a6 a8 aa ac ae a1 a3 a5 a7 a9 ab ad af c0 c2 c4 c6 c8 ca cc ce c1 c3 c5 c7 c9 cb cd cf e0 e2 e4 e6 e8 ea ec ee e1 e3 e5 e7 e9 eb ed ef 10 12 14 16 18 1a 1c 1e 11 13 15 17 19 1b 1d 1f 30 32 34 36 38 3a 3c 3e 31 33 35 37 39 3b 3d 3f 50 52 54 56 58 5a 5c 5e 51 53 55 57 59 5b 5d 5f 70 72 74 76 78 7a 7c 7e 71 73 75 77 79 7b 7d 7f 90 92 94 96 98 9a 9c 9e 91 93 95 97 99 9b 9d 9f b0 b2 b4 b6 b8 ba bc be b1 b3 b5 b7 b9 bb bd bf d0 d2 d4 d6 d8 da dc de d1 d3 d5 d7 d9 db dd df f0 f2 f4 f6 f8 fa fc fe f1 f3 f5 f7 f9 fb fd ff	Y0	Y1
		Y2	Y3
sensor Bayer data	Re-arrangement of the pixels that groups each color componet together	Four 8x8 pixel blocks ready for compression.
Differential mode adjusted for the HDR applications (JP4DH): both primary green color (G) and the second (high gain) one (G2) are encoded as absolute data (Y1,Y2), two other color components use difference from G: Y0=R-G, Y1=G, Y2=G2, Y3=B-G