Corrupt pictures/JPGs when using Camera2 API and sharing from a gallery (EXIF subsec field separators wrong?)
Expected Behavior
Using LOS 18.1 on a OnePlus 3
Taking a picture using OpenCamera or FreeDCam (configured to use camera2-API), works as expected.
Sharing picture using the share dialog and sending it to a nextcloud instance via the app, or using "sendreduced free", or using K9-Mail should result in sending correct picture data.
Current Behavior
Showing images in stock gallery or "Simple Gallery" works flawlessly.
However, when using the mentioned sharing procedure, the image gets corrupted.
This is what ImageMagick has to say:
display: Corrupt JPEG data: 134 extraneous bytes before marker 0xc0 `bad.jpg' @ warning/jpeg.c/JPEGWarningHandler/401.
display: Quantization table 0x00 was not defined `bad.jpg' @ error/jpeg.c/JPEGErrorHandler/346.
This a diff of the hexdumps of both files. The actual image data is irrelevant, the difference is always the same at the same address.
diff -u good.txt bad.txt
--- good.txt 2021-09-24 23:16:45.041429329 +0200
+++ bad.txt 2021-09-24 23:16:37.346356342 +0200
@@ -45,7 +45,7 @@
000002c0 32 39 32 38 00 00 02 00 01 00 02 00 00 00 04 52 |2928...........R|
000002d0 39 38 00 00 02 00 07 00 00 00 04 30 31 30 30 00 |98.........0100.|
000002e0 00 00 00 00 00 01 00 05 00 05 00 00 00 01 00 00 |................|
-000002f0 02 ea 00 00 00 00 00 00 00 00 00 00 00 64 ff db |.............d..|
+000002f0 02 ea 00 00 00 00 00 00 00 00 00 00 00 00 00 db |................|
00000300 00 84 00 03 02 02 03 02 02 03 03 03 03 04 03 03 |................|
00000310 04 05 08 05 05 04 04 05 0a 07 07 06 08 0c 0a 0c |................|
00000320 0c 0b 0a 0b 0b 0d 0e 12 10 0d 0e 11 0e 0b 0b 10 |................|
If I choose the picture directly with the app (Nextcloud for example) and upload the file, it stays intact.
I does not matter which gallery I use, be it "Simple Gallery" or the stock gallery, pictures get corrupted.
If I use the stock file manager, sharing works as it should!
(com.simplemobiletools.filemanager.pro does not work - it seems to use the same code as "Simple Gallery" for showing images in the file manager)
I could not find many other reports of this except this one for a OnePlus 7 LOS build, which is the exact same problem:
https://forum.xda-developers.com/t/rom-official-hotdog-11-lineageos-18-1.4177353/post-84343087
Also, the guys at signal seem to get flooded with what seem to be very similar issues. This is just one example:
https://github.com/signalapp/Signal-Android/issues/11244
Attached files:
The only big difference metadata-wise in files created with the two different APIs seems to be there is a thumbnail in the APIv1 generated file vs none in the APIv2 file. I do not know whether this is relevant.
I removed the thumbnail from the APIv1 file. The two files(' headers) then looked pretty similar (different timestamps, different actual image data). Still, sharing the now stripped APIv1 file still works. There must be something else that triggers the error.
Steps to Reproduce
- Take "good.jpg" picture from above, and share it via the share option in the stock gallery to one of the mentioned apps (or most likely, all apps).
- Check file - it will be corrupted.
Possible Solutions
I played around a little more in the data that differs between a thumbnail-stripped APIv1 image and the APIv2 one.
If I change these two bytes (0x2b5 and 0x2bd) right before the thumbnail block, these are the sub second fields, the sharing suddenly works!
diff -u api2{,_modded}.txt
--- api2.txt 2021-09-25 01:11:54.788389192 +0200
+++ api2_modded.txt 2021-09-25 01:12:03.457458950 +0200
@@ -41,7 +41,7 @@
00000280 39 3a 32 34 20 32 33 3a 33 33 3a 33 36 00 00 00 |9:24 23:33:36...|
00000290 12 23 00 00 03 e8 00 00 00 c8 00 00 00 64 00 00 |.#...........d..|
000002a0 00 00 00 00 00 64 00 00 10 a4 00 00 03 e8 35 38 |.....d........58|
-000002b0 34 32 38 34 00 30 35 38 34 32 38 34 00 3a 35 38 |4284.0584284.:58|
+000002b0 34 32 38 34 00 00 35 38 34 32 38 34 00 00 35 38 |4284..584284..58|
000002c0 34 32 38 34 00 00 02 00 01 00 02 00 00 00 04 52 |4284...........R|
000002d0 39 38 00 00 02 00 07 00 00 00 04 30 31 30 30 00 |98.........0100.|
000002e0 00 00 00 00 00 01 00 05 00 05 00 00 00 01 00 00 |................|
Modified image: api2_modded.jpg
I don't know whether I corrected an error that the APIv2 generates already or whether I manipulated the data in such a way that the sharing action bug does not get triggered. Judging from what
exiftool -subsectime=584284 bad.jpg
does (more or less the same changes outlined above, plus a few other bytes), I guess this can be considered the correct way and the camera2 API primarily is to blame for writing invalid separators between these numbers. OTOH, the sharing action should not manipulate the picture at all (IMHO) and if, then not in a way that corrupts the file, so this I guess is a bug in both functions/apps/pieces of software.
I am just a sysadmin - I hope some dev might now know what the actual error might be! If I can help, please say so, but I guess that's about how much I can help.
/device oneplus3 /version lineage-18.1 /date 2021-07-12 /kernel 3.18.124-lineageos-g04ff888001f2 /baseband c1.9-00102-M8996FAAAANAZM-1.197095.1.198697.1 /mods F-Droid
I have read the directions.