From eae105bb72770518a980d34c5950734b420f482f Mon Sep 17 00:00:00 2001 From: NiLuJe Date: Tue, 22 Oct 2019 01:25:00 +0200 Subject: [PATCH] Minor dev documentatioon tweaks (#5518) * Made Porting not so hilariously outdated. * Minor tweaks to the macOS build instructions and the various quirks involved in getting it to behave (and accompanying actual build fixes). --- base | 2 +- doc/Building.md | 10 +++--- doc/Porting.md | 93 ++++++++++++++++++++++++++++--------------------- 3 files changed, 60 insertions(+), 45 deletions(-) diff --git a/base b/base index d9a685c68..a1fc4e43b 160000 --- a/base +++ b/base @@ -1 +1 @@ -Subproject commit d9a685c6817d073a9fc0cd231e567a998250a9c6 +Subproject commit a1fc4e43b7cce7a76b13224e145f9bada343d8ea diff --git a/doc/Building.md b/doc/Building.md index 487784323..322566e8e 100644 --- a/doc/Building.md +++ b/doc/Building.md @@ -33,7 +33,7 @@ Install the `libstdc++-static`, `SDL` and `SDL-devel` packages using DNF: sudo dnf install libstdc++-static SDL SDL-devel ``` -### MacOS +### macOS Install the prerequisites using [Homebrew](https://brew.sh/): @@ -43,12 +43,14 @@ sdl2 lua@5.1 luarocks gettext pkg-config wget md5sha1sum echo 'export PATH="/usr/local/opt/gettext/bin:$PATH"' >> "$HOME"/.bash_profile ``` -If you run into a gettext error while building glib, try `brew link --force gettext` to override the built-in Mac OS BSD gettext with GNU GetText. +If you run into a gettext error while building glib, try `brew link --force gettext` to override the built-in macOS BSD gettext with GNU gettext. -*Note:* in Mojave (10.14) you need to set a minimum deployment version higher than 10.04. Otherwise you'll get the error `ld: library not found for -lgcc_s.10.4`. +*Note:* With current XCode versions, you *will* need to set a minimum deployment version higher than `10.04`. Otherwise, you'll hit various linking errors related to missing unwinding libraries/symbols. +On Mojave, `10.09` has been known to behave with XCode 10, And `10.14` with XCode 11. When in doubt, go with your current macOS version. ``` export MACOSX_DEPLOYMENT_TARGET=10.09 ``` +*Note:* On Catalina (10.15), you will currently *NOT* want to deploy for `10.15`, as [XCode is currently broken in that configuration](https://forums.developer.apple.com/thread/121887)! (i.e., deploy for `10.14` instead). ## Getting the source @@ -107,7 +109,7 @@ Once you have the emulator ready to rock you can [build for other platforms too] ## Testing You may need to check out the [circleci config file][circleci-conf] to setup up -a proper testing environment. +a proper testing environment. Briefly, you need to install `luarocks` and then install `busted` and `ansicolors` with `luarocks`. The "eng" language data file for tesseract-ocr is also need to test OCR functionality. Finally, make sure that `luajit` in your system is at least of version 2.0.2. diff --git a/doc/Porting.md b/doc/Porting.md index a5e2882a1..26846da16 100644 --- a/doc/Porting.md +++ b/doc/Porting.md @@ -2,72 +2,83 @@ This page aims to provide guidance on how to port KOReader to other platforms. -There are mainly two modules that you need to take care of: input and output. -After you finish these two, KOReader should have no problem running on your -platform. Feel free to open issues in our issue tracker if you need further help on this topic :) +There are mainly two modules that you need to take care of: input and output. +After you finish these two, KOReader should have no problem running on your platform. +Feel free to open issues in our issue tracker if you need further help on this topic :) ## Output Module -KOReader uses framebuffer to control EInk devices, so the output module here is -[base/ffi/framebuffer_einkfb.lua](https://github.com/koreader/koreader-base/blob/master/ffi/framebuffer_einkfb.lua). +### Current mxcfb eInk devices + +KOReader uses the Linux framebuffer to control eInk devices, so the output module for mxcfb (i.e., those based on Freescale/NXP hardware) devices is [`base/ffi/framebuffer_mxcfb.lua`](https://github.com/koreader/koreader-base/blob/master/ffi/framebuffer_mxcfb.lua). + +Most common bitdepths are supported, although no devices should actually be using anything other than 8bpp, 16bpp and 32bpp. +For 8bpp, we assume the grayscale palette is NOT inverted. +At 32bpp, we generally assume the pixel format is BGRA, and we honor Alpha, despite it being effectively ignored by the display (see Kobos). +At 16bpp, we assume the pixel format is RGB565. + +For obvious performance reasons, we prefer 8bpp, and we will attempt to enforce that on devices which are not natively running at that depth (i.e., [on Kobos](https://github.com/koreader/koreader/blob/d1cd5e7ad4283611c57007b2c2d3dd5f7dab7057/platform/kobo/koreader.sh#L138-L186)). +As explained below, the same considerations should be kept in mind regarding the effective 16c palette of eInk screens. +When we're in control of the data, we attempt to always use "perfect" in-palette colors (c.f., the [COLOR constants](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/blitbuffer.lua#L1881-L1889) in the BlitBuffer module). +Otherwise, when there'd be signficiant gain in doing so (i.e., when displaying mainly image content), we attempt to make use of [dithering](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/blitbuffer.lua#L227-L271), ideally [offloaded to the hardware](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/framebuffer_mxcfb.lua#L412-L423) when supported. + +The actual framebuffer content is then refreshed (i.e., displayed) via device-specific ioctls, making the best effort in using device-specific capabilities, whether that be [optimized waveform modes](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/framebuffer_mxcfb.lua#L643-L655), hardware dithering or [hardware inversion](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/framebuffer_mxcfb.lua#L253-L256). + +### Legacy einkfb eInk devices + +KOReader uses the Linux framebuffer to control eInk devices, so the output module for legacy einkfb devices is [`base/ffi/framebuffer_einkfb.lua`](https://github.com/koreader/koreader-base/blob/master/ffi/framebuffer_einkfb.lua). Following are the framebuffers that `framebuffer_einkfb.lua` currently supports: - * 4BPP inverted framebuffer - * 16 scale 8BPP inverted framebuffer - * 16 scale 8BPP framebuffer + * 4bpp framebuffer (palette is always inverted) + * 16c 8bpp framebuffer (inverted grayscale palette) -For 4BPP framebuffer, it means every pixel is represented with 4 bits, so we -have 2 pixels in 1 byte. So the color depth is 16. The inverted part means all -the bits are flipped in the framebuffer. For example, two pixels `[0x00, 0xf0]` -will be stored as `0xff0f` in framebuffer. +For 4bpp framebuffers, it means every pixel is represented with 4 bits, so we have 2 pixels in 1 byte. +That also effectively limits the palette to 16 colors. +The inverted part means that every pixel's color value is flipped (`^ 0xFF`). +For example, two pixels `0x00` and `0xF0` will be flipped to `0xFF` and `0x0F`, before being packed to accomodate the framebuffer's pixel format (here, [into a single byte](https://github.com/NiLuJe/FBInk/blob/4f0230b17c480cdc75dd5497fddf33937781c812/fbink.c#L106-L133)). -For 16 scale 8BPP framebuffer, it means each pixel is instead stored in 1 byte, -but the color depth is still 16 (4bits). Since 1 byte has 8 bits, so to fill -up the remaining space, the most significant 4 bits is a copy of the least -significant one. For example, pixel with grey scale 15 will be represented as -`0xffff`. If it's a inverted 16 scale 8BPP framebuffer, then all the bits are -flipped in the same way as 4BPP inverted framebuffer does. +For 8bpp framebuffers, it means each pixel is instead stored in 1 byte, making addressing much simpler. +The effective color palette of the display is still limited to 16 shades of gray: it will do a decimating quantization pass on its own on refresh. +So, while a black pixel will indeed be `0x00`, any color value < `0x11` (the next effective shade of gray in the [palette](https://github.com/NiLuJe/FBInk/blob/4f0230b17c480cdc75dd5497fddf33937781c812/fbink_internal.h#L327-L334)) will be displayed as pure black, too. +If the palette is expected to be inverted, then all the bits are flipped in the same way as done on a 4bpp framebuffer. +In practice, [this is always the case](https://github.com/koreader/koreader-base/blob/a1fc4e43b7cce7a76b13224e145f9bada343d8ea/ffi/framebuffer_linux.lua#L242-L245). -If your device's framebuffer does not fit into any of the categories above, -then you need to add a new transformation function in `framebuffer_einkfb.lua`. +The actual framebuffer content is then refreshed (i.e., displayed) via device-specific ioctls. -The `framebuffer_einkfb.lua` module works in following ways for non 4BPP framebuffers: +## Blitter Module - * a shadow buffer is created and structured as 4BPP inverted framebuffer. - * all updates on screen bitmap are temporally written into the shadow buffer. - * each time we want to reflect the updated bitmap on screen, we translate the shadow buffer into a format that the real framebuffer understands and write into the mapped memory region. (varies on devices) - * call ioctl system call to refresh EInk screen. (varies on devices) +All the intermediary buffers are handled in a pixel format that matches the output module in use as closely as possible. +The magic happens in [`base/ffi/blitbuffer.lua`](https://github.com/koreader/koreader-base/blob/master/ffi/blitbuffer.lua), with some help from the [LinuxFB](https://github.com/koreader/koreader-base/blob/master/ffi/framebuffer_linux.lua) frontend to the output modules. -KOReader will handle the 4BPP shadow buffer for you, all you need to do is to -teach `framebuffer_einkfb.lua` how to control the EInk screen and translate the 4BPP inverted -bitmap into the format that your framebuffer understands. +Note that on most devices, a [C version](https://github.com/koreader/koreader-base/blob/master/blitbuffer.c) is used instead for more consistent performance. +Which version is more easily readable to a newcomer is up for debate, so, don't hesitate to cross-reference ;). +Feature-parity should be complete, with the exception of 4bpp support in the C version. +If you need a bit of guidance, you can also take a look at [FBInk](https://github.com/NiLuJe/FBInk), and/or ping [@NiLuJe](https://github.com/NiLuJe) on gitter. ## Input Module -We have a `input.c` module in [koreader-base][kb-framework] that reads input -events from Linux's input system and pass to Lua frontend. Basically, you don't -need to change on that module because it should support most of the events. +We have an [`input.c`](https://github.com/koreader/koreader-base/blob/master/input/input.c) module in [koreader-base][kb-framework] that reads input events from Linux's input system and passes it on to the Lua frontend. +Basically, you don't need to change that module because it should support most of the events. For this part, the file you have to hack on is [`koreader/frontend/ui/input.lua`](https://github.com/koreader/koreader/blob/master/frontend/ui/input.lua). -Firstly, you need to tell which input device to open on KOReader start. All the -input devices are opened in `Input:init()` function. +Firstly, you need to tell which input device to open on KOReader start. All the input devices are opened in `Input:init()` function. -Next, you might need to define `Input:eventAdjustHook()` function in -`Input:init()` method. We use this hook function to translates events into a -format that KOReader understands. You can look at the KindleTouch initialization code for real example. +Next, you might need to define `Input:eventAdjustHook()` function in `Input:init()` method. +We use this hook function to translate events into a format that KOReader understands. +You can look at the KindleTouch initialization code for a real-world example. -For Kobo devices (Mini, Touch, Glo and Aura HD) the function `Input:eventAdjustHook()` was skipped and the functions `Input:init()` and `Input:handleTypeBTouchEv` were changed to allow the single touch protocol. For Kobo Aura with multitouch support an extra function `Input:handlePhoenixTouchEv` was added. +For some Kobo devices (Mini, Touch, Glo and Aura HD) the function `Input:eventAdjustHook()` was skipped and the functions `Input:init()` and `Input:handleTypeBTouchEv()` were changed to accomodate for the single touch protocol. +For the Kobo Aura (and others with the same kernel quirks) with multitouch support, an extra function `Input:handlePhoenixTouchEv()` was added. Linux supports two kinds of Multi-touch protocols: * -Currently, KOReader supports gesture detection of protocol B, so if your device sends out -protocol A, you need to make a variant of function `Input:handleTouchEv()` (like `Input:handleTypeBTouchEv` and `Input:handlePhoenixTouchEv`) and simulate protocol B. -Also you are welcome to send a PR that adds protocol A support to KOReader. +Currently, KOReader supports gesture detection of protocol B, so if your device sends out protocol A, you need to make a variant of function `Input:handleTouchEv()` (like `Input:handleTypeBTouchEv()` and `Input:handlePhoenixTouchEv()`) and simulate protocol B. +You are also welcome to send a PR that adds protocol A support to KOReader. More information on Linux's input system: @@ -75,3 +86,5 @@ More information on Linux's input system: * [kb-framework]:https://github.com/koreader/koreader-base + +