Commit Graph

430 Commits (ce624be8b83e880b1a81106a143e1440eeda4aab)

Author SHA1 Message Date
NiLuJe ce624be8b8 Cache: Fix a whole lot of things.
* Minor updates to the min & max cache sizes (16 & 64MB). Mostly to satisfy my power-of-two OCD.
  * Purge broken on-disk cache files
  * Optimize free RAM computations
  * Start dropping LRU items when running low on memory before pre-rendring (hinting) pages in non-reflowable documents.
  * Make serialize dump the most recently *displayed* page, as the actual MRU item is the most recently *hinted* page, not the current one.
  * Use more accurate item size estimations across the whole codebase.

TileCacheItem:

  * Drop lua-serialize in favor of Persist.

KoptInterface:

  * Drop lua-serialize in favor of Persist.
  * Make KOPTContext caching actually work by ensuring its hash is stable.
3 years ago
poire-z 9ef435c97a
bump crengine: more granular font weights (#7616)
Includes:
- MathML: a few minor fixes
- (Upstream) lvtext: fix possible index out of range
- Fonts: RegisterExternalFont() should take a documentId
- Fonts: fix: letter-spacing should not be applied on diacritic
- (Upstream) Fonts: more granular synthetic weights
- Fonts: synthesized weights: tweak some comments
- Fonts: keep hinting with synthetic weight
- Fonts: fix synthesized weight inconsitencies
- Fonts: fix getFontFileNameAndFaceIndex()
- Fonts: adds LVFontMan::RegularizeRegisteredFontsWeights()
- Fonts: handle synth_weight tweaks in glyph/glyphinfo slots
- (Upstream) Fonts: fix some compiler warnings
- Fix hyphenation on Armenian and Georgian text

Update the bottom menu widget "Font Weight" to allow more
granular weights than the previous "regular | bold".

Also bump thirdparty/luasec to v1.0.1.
3 years ago
Martín Fernández 53234fcdc1
add hasSystemFonts device property (#7535)
Add system + user paths to the ReMarkable (has normal linux paths)
3 years ago
poire-z b9ffc3d05b
bump crengine: add support for MathML (#7465)
Includes (among others):
- LVImg: Tweak JPEG decoding some more
- toStringV2(): fix (again) when target node is a boxing node
- LVFontCache::find(): give more weight to first fonts in list
- Page splitting: more accurate rendering progress
- getRenderedWidths(): fix nowrap around image/inlineBoxes
- Tables rendering: tweak column widths algorithm
- CSS: parse/handle "currentcolor", default for border-color
- CSS: add units 'ch' (just like 'ex')
- SVG images: proper alpha blending
- MathML: add parsing and rendering support files
- MathML: plug MathML code into crengine core
- MathML: <epub:switch/case/default>: accept MathML
- (Upstream) Make crengine.font.fallback.faces plural
- (Upstream) Option to not limit font size to a set
- Text: dont adjust space after consecutive initial marks/dashes
- Update German hyphenation patterns
3 years ago
NiLuJe f3341d9dc0
PdfDocument: Unbreak highlights (#7457)
Regression since #7411
Fix #7456
3 years ago
Toromtomtom 3706196bfe
Update PDF annotations when changing bookmark text (#7411) 3 years ago
NiLuJe bf6c0cdd6c
LuaSettings: Add a method to initialize a setting properly (#7371)
* LuaSettings/DocSettings: Updated readSetting API to allow proper initialization to default.
Use it to initialize tables, e.g., fixing corner-cases in readerFooter that could prevent settings from being saved.
(Fixes an issue reported on Gitter).
* LuaSettings/DocSettings: Add simpler API than the the flip* ones to toggle boolean settings.
* Update LuaSettings/DocSettigns usage throughout the codebase to use the dedicated boolean methods wher appropriate, and clean up some of the more mind-bending uses.
* FileChooser: Implement an extended default exclusion list (fix #2360)
* ScreenSaver: Refactor to avoid the pile of kludges this was threatening to become. Code should be easier to follow and use, and fallbacks now behave as expected (fix #4418).
3 years ago
Frans de Jonge ac668ecb64
Add a few more mimetypes for OPDS (#7258)
Doesn't include application/zip as CBZ, but it will be downloaded (as ZIP).

Doesn't include CBR since that's not supported.

Closes #7218, closes #5997.
3 years ago
zwim 3118d0dba0 Refresh AltStatusBar once a minute, if there are changes 3 years ago
poire-z 05126b94b6 Dual pages: shown as 2 columns on a single page
Rework Dual pages code so that the view is considered
a single page number, so it looks more like 2-columns
on a single page.
This solves a few issues like:
- Page number and count are consistent between top
  and bottom status bars
- SkimTo -1/+1 doing nothing every other tap
- Statistics being wrong (like "Pages read" never
  going over half of the book page count)
3 years ago
poire-z 7779e2d8e7 CRe: use getDocumentRenderingHash() to detect rendering changes
Instead of just relying on document full height
and number of pages.
3 years ago
poire-z 8ff50a9e24 CreDocument: disable crengine image scaling options
Since their handling in crengine has been re-enabled.
3 years ago
Frans de Jonge 1ef6d0b257
[feat] Support mimetypes in DocumentRegistry:hasProvider() (#7155)
And make .djvu the canonical extension for DjVu.

Fixes #5478.
3 years ago
poire-z 396d1fbf46
bump crengine: parsing, lists, 2-pages mode fixes & tweaks (#7138)
Includes:
- EPUB: fix truncated HEAD>STYLE stylesheet
- XML parsing: slightly better parsing of <script>
- Update German hyphenation patterns
- (chore) Silence some clang warnings
- (Upstream) CSS content: fix regression with open-quote/close-quote
- (Upstream) HTML lists: support the 'reversed' attribute
- (Upstream) Tweak list items disc/circle/square symbols
- 2-pages mode: option to skip geometry checks

CRE bottom menu: allow toggling Dual Pages in portrait mode.
3 years ago
poire-z dd74194e0a cre.getWordFromPosition(): fix a few issues
Drop the use of crengine's getWordFromPosition() which
is a bit unreliable: it may returns wrong coordinates,
or words from far away in the book (ie. when holding
in the margins).
Rely only on the robust getTextFromPositions() that
we already use for multi words selection.
Having good coordinates allows refreshing a smaller region
(the higlighted word, or the 2 lines if hyphenated).
3 years ago
NiLuJe d80d6dc562 Handle the BlitBuffer struct changes
* stride is now a size_t
  On some platforms, that's 64 bits, which means it's no longer
  automatically converted to a Lua number to avoid precision loss.
  Do that ourselves, because lua-serialize doesn't know how to handle an
  uint64_t cdata ;).
3 years ago
jperon 8b7d60299f
JPG/PNG: MuPDF as default provider (#6931)
As said in #6929
3 years ago
jperon 8eeb010dc9
Paged documents: rework zoom options (#6885)
- Move zoom options from top menu to bottom config
- Add option to manually define zoom (relative to
  page width) and overlap (in percent)
- Add options to zoom to columns or rows, possibly
  with overlap. Add panning direction options when
  page forward in these modes
3 years ago
Jellby 5e3c554dd7 Hide non-linear fragments
Add option to hide (skip) non-linear fragments, only working
in 1-page mode. Tweaks mostly to footer, toc and skim code
to make it clear(er) which pages belong to linear or non-linear
fragments.
4 years ago
Jellby f892d4559f Fix typos 4 years ago
yparitcher edec69ac8b
[CRe] Tweak nightmode and CRe call cache interaction (#6859)
Simplify 4caf8f28 (#6854), allowing us not to track
nightmode in two places.
4 years ago
NiLuJe 4caf8f281d
[CRe] Ensure toggling nightmode invalidates the drawCurrentView cache (#6854)
* Use a CRe set* method when toggling nightmode

This ensures it gets flagged as add_reset by the call cache, and that
CRe will actually re-render, as it's necessary if nightmode_images is
enabled (the default).

Fix #6845

* Prevent ReaderHighlight:onTap from running ReaderHighlight:clear when
it's unnecessary.

Avoiding a clearSelection call in CRe that could invalidate the cache
and cause unnecessary redraws.

* Don't store empty highlight arrays when all HLs on a page have
been deleted
4 years ago
Martín Fernández 40b4ccffa8
KoptInterface:getWordFromBoxes: guard against nil boxes (#6827)
Fixes #6825
4 years ago
poire-z 8b886e5922
bump crengine: sync with upstream, new hyphenation languages (#6746)
Includes:
- (Upstream) various unimpacting changes to keep in sync
- (Upstream) Fix issues with legacy text rendering
- (Upstream) FB3/DocX/ODT: get lang and description metadata
- (Upstream) remove some global settings, make them per-doc
- TextLang, hyphenation: add Armenian, Friulian, Piedmontese,
  Romansh, Zulu and Brazilian Portuguese.

Also make cre the prefered engine for .xhtml (over MuPDF)
and for .xml (which might be HTML).
4 years ago
ezdiy 7a40b496fb
blitbuffer: fixups for base (#6714)
fixups for base bumped to https://github.com/koreader/koreader-base/pull/1201
otherwise tilecache breaks
4 years ago
Galunid b297fb7cf6
Panel zoom - improve output image resolution (#6709) 4 years ago
Galunid 15455b594d
[feat] Comics: zoom to panel (#6511)
This pull requests aims to provide convenient way to zoom in comics. The idea is when user holds/double taps (not decided yet) on a manga/comic panel, it gets cut out from the rest of the image and zoomed. More details in koreader/koreader-base#1148. Depends on koreader/koreader-base#1159
4 years ago
poire-z 4d53ec76fe
bump crengine: ODT support, more typography languages (#6675)
Includes:
- Update German hyphenation patterns
- (Upstream) Adds ODT (ODF) format support
- TextLang, hyphenation: add Basque, Croatian, Esperanto,
  Estonian, Georgian, Latvian, Lithuanian, Macedonian,
  Occitan, Welsh; update Bulgarian, Irish, Portuguese,
  Slovak, Dutch, Norwegian, Spanish; update hyphen min
  for Czech, English, Greek; fix Romanian and Ukrainian
  pattern file names
- HyphMan: adds HyphMethod::getLeft/RightHyphenMin()
- epub.css: update HR default style
- fb2.css: keep <date> in main text left-aligned
- getRenderedWidths(): inline-block and table fixes
- CSS: avoid style hash mismatch when serializing content:''
- Tables: re-order row groups when necessary
- XML parsing: don't drop trailing text
- HTML parser: tweak implicit head/body insertion code
- Fix text search failure when blank at start or end

readertypography.lua: some cleanup by removing the first
table listing hyph dicts, and including their filenames
in the second table so we can build the first table
dynamically. Also fetch hyph left/right min limits from
crengine to not have to duplicate them here and keep them
in sync.
css_tweaks.lua: for in-page footnotes, switched from
"-cr-only-if: epub-document" to ": -fb2-document"
("not" FB2 documents), so these tweaks also work with
DocX and ODT footnotes.

Also included in base bump:
- KOPTContext: add comic panel detection function/additional
  leptonica declarations
- Forcibly disable docs when building giflib
4 years ago
NiLuJe 99045b4311
Minor util & ffi/util cleanups (#6657) 4 years ago
ezdiy 426f7b5ddc
Add API to set OS level file associations (#6615) 4 years ago
Martín Fdez 454ed0ebeb enable usage of system fonts 4 years ago
poire-z 8d99523388
bump crengine: new HTML parser, libRu and FB2 tweaks (#6560)
Includes:
- Revert "FB2: don't draw cover in scroll mode"
- (Upstream) FB2: fix coverpage drawing in scroll mode
- FB2 footnotes: only merge run-in when next is erm_final
- fb2.css: use OTF tabular-nums for footnote numbers
- Text: ignore ascii and unicode control chars
- Fix HR positionning when floats involved
- writeNodeEx(): minor tweaks
- OnTagClose(): add self_closing_tag parameter
- HTML format detection: accept HTML5 doctype
- HTML parser: rework Lib.ru specific handling
- HTML parser: new more conforming implementation
- HTML parser: ensure foster parenting inside tables

Also includes:
- SDL: don't bypass X11 WM compositor
4 years ago
Frans de Jonge da507f8607
[chore] Update MD5 calls with new sha2 library (#6411)
Depends on <https://github.com/koreader/koreader-base/pull/1149>.
4 years ago
Martín Fernández f9293a67b0
[fix] Remove isAllowedScript leftovers (#6366)
Fixes #6365
4 years ago
Martín Fernández d87b09d11c
handle newlines in exported pdf highlights (#6247) 4 years ago
NiLuJe 64825eccd2
Update bundled fonts (#6242)
* Update bundled fonts

* Minor FreeFont update
* Noto Sans CJK downgrade to fix a number of issues w/ the git version
* Initial import of Noto Naskh Arabic

* Replace Noto Sans Arabic UI by Noto Naskh Arabic in the CRe fallback
list
4 years ago
poire-z 5536ce996a CRE: use same marks in top and bottom progress bars
Feed to crengine the same ticks (build from the TOC
entries) that we use in the bottom status bar.
(crengine otherwise builds a tick for each DocFragment,
which most often is really different than what's seen
in the bottom bar.)
4 years ago
poire-z 177551918f
bump_crengine: various fixes, tweaks and cleanups (#6188)
Includes:
- Don't reset invisible BR to display: inline
- ldomTextCollector: handle inline-block/table as inline
- Avoid re-rendering on font hinting mode change
- List markers: fix positionning when floats involved
- GIF decoding: avoid uneeded abort on LZW table overflow
- Adds getUnknownEntities(), can help with debugging
- Don't check stylesheet hashes when loading from cache
- Hardcoded elements and attributes list: cleanup
- Full rendering: 2 small optimisations
- Strut confinning: deal with images earlier
- lvtextfm: some cleanup and reordering
cre.cpp:
- fix getPageFromXPointer() and getPosFromXPointer()
  to not fail on invisible nodes and work on the next
  visible node (so links whose target is display:none
  can work).

Add support for 2 settings one can set manually
in reader.settings.reader.lua:
["cre_background_color"] = "0xccffcc",
["cre_background_image"] = "/path/to/bg_paper.jpg",
4 years ago
poire-z e074b603e0 Adds Word Expansion: use of letter spacing for justification
Bump crengine:
- Text: fix occasional BiDi bad word splitting
- Font: fix HB fallback measurement/drawing mismatches
- Font: do not add letter spacing on diacritics
- Text: tunable use of letter spacing for justification
- Text: dont adjust space after initial quotation mark/dash (rework)
- Text: fix possible bad widths after collapsed spaces

xtext.cpp: pick crengine fix with HB fallback
measurement/drawing mismatches

Adds "Word Expansion" to bottom Font size menu, and moved
"Word Spacing" there too, to balance the panels' heights
to 4 items.
4 years ago
poire-z 1a91893576
Allow toggling use of additional fallback fonts (#6095) 4 years ago
poire-z 7d83a0c967
bump crengine: multiple fallback fonts (#6090)
Includes:
- Simplify libunibreak includes
- Text: fix read/write outside array bounds
- lvtextfm: dont adjust space after initial quotation mark/dash (rework)
- Fonts: allow providing and using multiple fallback fonts

Users can set their prefered fallback font, which will be completed
with a few of our shipped fonts for maximum coverage.
4 years ago
poire-z 6dc8bbcc49
Adds ReaderTypography (replaces ReaderHyphenation) (#6072)
Replace Hyphenation menu with Typography menu.
This works mostly like before:
- typography/hyphenation is chosen according to the book
  metadata language,
- one can set a default or a fallback language,
- hyphenation is now just a subset of typography, and
  can be disabled while still setting a language and
  enabling the other features,
- the typography language enables newly added features
  to crengine: per-language line breaking rules and
  per-language Harfbuzz glyph selection.
4 years ago
Martín Fdez 8e831eb756 desktop: disable battery on touch menu and reader footer
isDesktop and isEmulator are now different entities
4 years ago
Frans de Jonge 2945a2fb79
Add EPUB3 extension to MuPDF document (#6013)
Loads fine, see 026140f809 (commitcomment-38122799)
4 years ago
poire-z 026140f809 Adds ReaderPageMap, to optionally show source pages numbers
bump crengine: support for EPUB3 nav toc and page maps
Includes:
- Fix lvRect:isRectInside(rc) with 0-width or 0-height rect
- TOC: parse EPUB3 nav toc, fallback to spine when no toc
- Parse and cache various hardcopy page list maps
- epub.css: hide EPUB3 <span epub:type="pagebreak"> content
cre.cpp: add a few PageMap helper functions.

Adds ReaderPageMap which will add a new menu (under TOC and
Bookmarks) that will allow:
- to list source page numbers (like a TOC)
- to show visible page labels in the right margin
- to use these source page numbers in the footer, the TOC,
  the GoTo and SkimTo widgets, and to use the source page
  number in the standard bookmark and highlight initial text.
4 years ago
Frans de Jonge c2aac0f71e
[fix] DocumentRegistry: only add provider once (#5947)
Fixes <https://github.com/koreader/koreader/issues/5946>.
4 years ago
poire-z fa6fed8569 cre: setFontFace(): increase bias given to the main font
Solve the issue when a font without bold and italic variants
is used as the default font (e.g. FreeSerif), and the style
tweak "Ignore publisher font-family" is used (which uses
a trick to cancel any font-family by requesting a font
named "NoSuchFont"):
When text is italic or bold, any of the registered fonts
which have a real italic or bold variant would win over our
default font, as the best substitute to NoSuchFont-Italic.
This gives our default font a bit more bias so it can
win in its scoring against the other fonts, and be rendered
as fake italic or fake bold - which will ensure consistent
font and line height.
(A bit hacky, but no alternative solution found.)
4 years ago
Martín Fernández 5d103a41f8
Avoid writting highlights into read-only PDFs (#5889) 4 years ago
poire-z 5a4f5b4d59 bump crengine, migrate books to normalized xpointers
Enable new rendering feature COMPLETE_INCOMPLETE_TABLES
on all enhanced rendering mode, but have it disabled for
earlier cre_dom_version.
Also increase default_cre_storage_size_factor from 20 to 40.
4 years ago
NiLuJe 5499d85cbc
Allow running shell scripts from the FileManager/Favorites (#5804)
* Allow running Shell/Python scripts from the FM

* Show an InfoMessage before/after running the script

Since we're blocking the UI ;).

* Allow running scripts from the favorites menu, too.
4 years ago