Commit Graph

29 Commits (master)

Author SHA1 Message Date
Romain 1aef90ce19 fix: make stats displayed during indexing more explicit
Before this, the "total" value represented files on disk
which was confusing in partial and incremental modes
5 months ago
Romain 9238a180fd refactor: replace event-stream with native Node streams 5 months ago
Romain 8639c9d635 refactor: remove obsolete exiftool workaround for bad encoding
Fixed in exiftool version 10.85 (March 2018)
5 months ago
Romain 6535caab7d chore: replace fs-extra by fs when possible 5 months ago
Romain 3590b15cfb chore: use node prefix for all builtin modules 6 months ago
Romain 8a43cebc72 feat: support multiple scan modes for partial gallery generation 1 year ago
Romain 34b957fb7e fix: supports nested folders for --include (fixes #332) 1 year ago
Romain 8cb7a3596b chore: upgrade to latest version of StandardJS 1 year ago
Romain 2537436c2f
Add mpg and mpeg file extension support
Fixes #280
2 years ago
Romain f3571718ce fix: index filenames with utf-8 characters on Windows
Fixes #184
3 years ago
Romain 0218f90f38
Add README for ListrWorkQueue component 3 years ago
Romain 7e5ccbfbaa chore(deps): update major dependencies 4 years ago
Romain 77ba7f09ab [feat] Support HEIC images, including iOS single / burst / live photos 5 years ago
Romain c4719328af fix(database): add support for AVI and WMV videos
fixes #147
5 years ago
Romain 9bda0a4835 feat(database): new include/exclude arguments to filter the input folder 5 years ago
Romain f30eddb6dd feat(options): new flags to selectively include photos / videos / raw photos
fixes #128
5 years ago
Romain 9425604dd7 chore(dependencies): upgrade Standard from v11 to v12 which introduces a style change 6 years ago
Romain 4030025bc9 feat(index): support m4v files, video format used when exporting from Apple Photos 6 years ago
Romain bf60ae4677 refactor(components): bring ListrWorkQueue into this repo to simplify the build process
- This component was not published to the rpm registry anyway
- We depended on the repo’s master branch which can break things for everyone
- Its repo was not getting much attention which meant
  * no tests, no coverage report
  * no linting
  * no package linting (e.g. had 2 dependencies that weren’t actually used)

It will be simpler to manage this way
6 years ago
Felix Eckhofer 81a85004c2 Fix behavior for non-traversable directories (#108)
fix(index): Continue on errors during find

This ensures the gallery is created even when some files are
inaccessible (for example due to invalid names or file permissions).

Requires running the test in Docker as non-root to allow file access test
6 years ago
Felix Eckhofer 7dc914f924 fix(index): skip non-existing files
This can happen when files have invalid names or they are being deleted
while thumbsup is running.

Fixes #106
6 years ago
Romain f7d7b827e5 feat(cli): --concurrency also controls the number of batch exiftool processes 6 years ago
Romain 81ee517062 feat(index): ignore files in a root #recycle folder (Synology recycle bin) 7 years ago
Romain d0bbf94b1c fix(index): don’t crash when trying to parse exiftool output for a badly encoded file 7 years ago
Romain a2042f3128 Fix logging option for non-TTY + highlight URL in welcome / error messages 7 years ago
Romain 47a924883e New --log option to print a full text log with different verbosity levels 7 years ago
Romain cbf8a1f3b4 Don’t index media from Synology thumbnail folders (@eaDir) nor folders starting with “.” 7 years ago
Romain ac520d40ad Fix case insensitive glob + add unit tests for globbing 7 years ago
Romain 24b2f9bd7c Major optimisations: SQLite index + faster disk glob + new exiftool streaming
1. Move from a JSON index to a SQLite database.
  - This allows the indexing to be interrupted & resumed
  - Updating the index consumes less RAM than loading / saving an entire JSON object
  - Loading the index consumes less RAM since it can be streamed, only exacting the properties we need every time (instead of loading all EXIF data in memory, only to discard most of it later)
  - These make a big difference when processing 10,000+ photos

2. Switch from <glob> to a manual <readdir>
  - Glob would take several hundred or GB of RAM when asked to find several thousand files
  - Manual approach with <micromatch> library does the same thing in a fraction of the time / memory usage

3. Exiftool optimisations
  - Run 1 exiftool process per CPU, still in batch mode (divide all files to be read into 1 bucket per CPU)
  - Stream the exiftool output instead of buffering it in memory
7 years ago