Commit Graph

242 Commits (master)

Author SHA1 Message Date
Romain 03c3b38e82 fix(model): ignore the EXIF date if it’s not valid e.g. 0000:00:00
Closes #94
6 years ago
Felix Yan 0ff1699c5c Correct a typo in step-index.js 6 years ago
Romain cd3d699b34 feat(cli): new --dry-run option to skip resizing the files / creating the website
The index is still updated, and the media folder is still scanned to log how many tasks are required.
This is useful to know what thumbsup is about to do, without actually running the expensive tasks.
7 years ago
Romain e5f6df3d28 refactor(cli): deprecated options logic now handled in <options> module 7 years ago
Romain 286dc8d15f feat(albums): --albums-from can be a list of patterns with special keywords
- %path expands to the path of the photo/video
- %keywords expands to the IPTC keywords of the photo
- {format} expands to the photo's EXIF date, e.g. {YYYY MM}
7 years ago
Romain fce8d8d8af refactor(model): album mappers can also return '/' to mean the root album 7 years ago
Romain a5486687e7 feat(log): print more debug logs about how the number of files and tasks 7 years ago
Romain 81ee517062 feat(index): ignore files in a root #recycle folder (Synology recycle bin) 7 years ago
Romain d0bbf94b1c fix(index): don’t crash when trying to parse exiftool output for a badly encoded file 7 years ago
Romain a2042f3128 Fix logging option for non-TTY + highlight URL in welcome / error messages 7 years ago
Romain 47a924883e New --log option to print a full text log with different verbosity levels 7 years ago
Romain 06ecd2edad Add usage reporting with Google Analytics + greeting/error messages
This will help understand usage patterns to know what to focus on, e.g.
- are many people using thumbsup on Windows?
- are there many galleries with > 10,000 photos?
7 years ago
Romain 3152361e65 Fix #83: handle IPTC keywords returned as string or array 7 years ago
Romain f66208e052 New options for the {{date}} helper: custom date format + how long ago 7 years ago
Romain 515ce3e8c1 Fix bug in #compare and #times Handlebars templates introduced in 3e0982e
When declared with () => instead of function (), the context is missing from child blocks.
7 years ago
Romain d7d01adc82 Fix #78: ignore invalid photos / videos
The current code doesn't create an output structure for them, so we don't create thumbnails.
This is good since the thumbnail generation would likely fail.

However we still try to render thumbnails in the themes.
The themes could be smart enough to skip invalid files, but it's easier to ignore them from the start.
7 years ago
Romain d19a230a6c Cleanup unused code 7 years ago
Romain 3e098e2328 Extract Handlebars helpers to prepare for theme structure + add unit tests 7 years ago
Romain cbf8a1f3b4 Don’t index media from Synology thumbnail folders (@eaDir) nor folders starting with “.” 7 years ago
Romain ac520d40ad Fix case insensitive glob + add unit tests for globbing 7 years ago
Romain 24b2f9bd7c Major optimisations: SQLite index + faster disk glob + new exiftool streaming
1. Move from a JSON index to a SQLite database.
  - This allows the indexing to be interrupted & resumed
  - Updating the index consumes less RAM than loading / saving an entire JSON object
  - Loading the index consumes less RAM since it can be streamed, only exacting the properties we need every time (instead of loading all EXIF data in memory, only to discard most of it later)
  - These make a big difference when processing 10,000+ photos

2. Switch from <glob> to a manual <readdir>
  - Glob would take several hundred or GB of RAM when asked to find several thousand files
  - Manual approach with <micromatch> library does the same thing in a fraction of the time / memory usage

3. Exiftool optimisations
  - Run 1 exiftool process per CPU, still in batch mode (divide all files to be read into 1 bucket per CPU)
  - Stream the exiftool output instead of buffering it in memory
7 years ago
Romain 30f203af4b Render progress using Listr + split the main process into "steps" which are easier to test 7 years ago
Romain b92724472e Fix #71: read both Quicktime CreationDate and CreateDate 7 years ago
Romain 0b27d37881 Read metadata from Picasa.ini if present 7 years ago
Romain 3e64d2ab38 Create a Metadata model attached to input files
- for easier unit testing
- to enable input filtering (e.g. only include photos with this keyword)
7 years ago
Romain d24c5110cc Fix #70: files in the root of the source folder get added to the root gallery (index.html)
Before, they were added to an album called “.” which didn’t look good,
and the corresponding “.html” file (no base name) caused issues on web servers
7 years ago
Romain ac67a08da9 Use new “downsize” package that was split out (to make testing edge cases easier) 7 years ago
Romain dc40fbac09 Fix inferring data from the filename: was getting confused if the folder contained digits 7 years ago
Romain 80f7a88c35 Fix #61: new options to avoid copying original photos to the output folder 7 years ago
Romain 6fde22e9d1 Fix #62: crash prevented the error message from showing when migrating from V1 to V2 7 years ago
Romain 1df4ee48f3 Fix #63: run ffmpeg using <spawn> and don't buffer stdout/stderr 7 years ago
Romain 3531f1e533 Try to infer dates from the filename if there is no "date" metadata 7 years ago
Romain a7ad8dc9fc Fix reading the original creation date of QuickTime and H264 movies 7 years ago
Romain 9a24acc6df Fix copying of --original-photos 7 years ago
Romain 91e55daf9b Upgrade VideoJS to 5.17 7 years ago
Romain 9c78c1cd79 Rename album sorting function for clarity (they don't take a raw File, but a Media object) 7 years ago
Romain fb77791c80 Fix #33: new --cleanup option to remove unused generated thumbnails/media 7 years ago
Romain 39ee2adf05 Read XMP ratings 7 years ago
Romain 449ee0b180 Read the caption from all standard EXIF/IPTC/XMP tags (issue #24) 7 years ago
Romain 7e73e6a10b Refactor album mapping code to be much simpler based on (media) => string 7 years ago
Romain 07ec7702cd Fix #f7451db when counting available CPUs (missing array length) 7 years ago
Romain f7451db36e Run as many photo/video tasks as CPUs 7 years ago
Romain 59b9bdbbd2 Use <standard> for code style, which removed lots of unused variables/requires 7 years ago
Romain e71e5d4207 Replace custom "make" DSL with the output task list (#60) 7 years ago
Romain e719bf15b1 Single source of truth for input/outputs + view model for display (#58) 7 years ago
Romain 794c76c232 Copy missing VideoJS fonts (still using version 4.12.15, current is 5.16.0) 7 years ago
Romain 0c125e90df Thumbnails are back to matching the source extension for now, for compat with V1
Before changing it back, need to list the rules for extension changes. For example:

- should GIF thumbnail should be JPG, to avoid animations on the album page?
- what about transparent GIFs, will they look weird in JPG?
- maybe GIFs should stay as GIFs, but kept to a single frame only for thumbnails
- same thing for pngs, which might be better kept as PNG for transparency
- all other non-browser-friendy formats should become JPG

These rules will be a lot easier to implement when the new input data structure is in place
7 years ago
Romain d3781b8fcf Fix parsing of MIME Type into Media type (image/video)
This fixes download links, and being able to right-click on a video and save-as
7 years ago
Romain 3a01c678eb Support for IPTC captions 7 years ago
Romain 4e8277b279 Improve progress bar to include ETA , e.g. "3min left" 7 years ago
Romain cc280a2180 Set the DEBUG env var to print verbose logs (including GraphicsMagick commands etc) 7 years ago
Romain 8dccb88f25 Use [exiftool-json-db] to maintain the JSON database of media files 7 years ago
Romain e36464a76a New --albums-output-folder option to write all albums to a subfolder (cleaner output)
The main changes are
- this introduces relative paths, since some HTML files have to go ".." to get to public/media
- it also introduces the difference between album.path (OS dependent) and album.url (forward slashes)
7 years ago
Romain e930af7f49 Add GIF support, fixes #22.
One major change here is that thumbnails will always be generated as ".jpg".
This is potentially a breaking change, in the sense that all "png" or "jpeg" thumbnails
would be to re-calculated and re-uploaded.
7 years ago
Romain 6772e9c9d9 Fix final options for nested albums, not being passed recursively 7 years ago
Romain 7575da7474 New <footer> config setting to add a line of text (or HTML) at the bottom 7 years ago
Romain dc7a7b5848 Must pass options when finalising album hierarchy. Fixes album sorting. 7 years ago
Romain 277fe126f7 Video model also includes 'large' URL so it's a superset of photos.
Without this, template code like {{previews.0.urls.large}} only works for photos.
7 years ago
Romain f235675f86 Don't concatenate the root album name to be 100% compatible with existing version for flat albums 8 years ago
Romain 0208babae1 Pick 10 previews per album, to make the mosaic theme work 8 years ago
Romain b905d62ac3 Fix unit tests (album.finalise argument should be optional) 8 years ago
Romain eb95cc929c Fix #30 Fix #31 use <mtime> instead of <ctime> which is more relevant
Also means we can use <touch> to force a file to rebuild, since touch changes <mtime>
8 years ago
Romain 315e08e0e5 Fix #30 Fix #31 allow asc/desc order when sorting albums and media 8 years ago
Romain 58c37bd429 Fix model paths 8 years ago
Romain d91fdbf525 Cleanup output + group photo/video processing 8 years ago
Romain fd9bf181fd Extract model from website generation 8 years ago
Romain b023b72cc0 Extract album creation as separate step + consolidate progress bar logic 8 years ago
Romain 7c0fb8a3bf Enable --albums-from = [ folders | date ] and --albums-date-format 8 years ago
Romain b64843a8a1 Re-implement "classic" theme = same as thumbsup v1 8 years ago
Romain 707e25ca34 Increase preview count for the Mosaic theme 8 years ago
Romain 6d162bd110 Fix sorting of albums & media 8 years ago
Romain Prieto 72fd69bb79 by-date generator supports nested albums + custom album title format 8 years ago
Romain 942f673670 Merge pull request #46 from rprieto/nested-albums
Nested albums
8 years ago
Romain Prieto d8f8a0c1fe Add handy Handlebars helper to repeat a block N times
Usage: {{#times 4}}hello{{/times}
Can also use any variables in context, e.g. {{#times things.length}}
8 years ago
Romain Prieto f69d2e3d48 Add support for nested folders 8 years ago
Romain Prieto 9c0071c00d Work in progress: support multiple themes 8 years ago
Romain Prieto c2f290bbd6 thumbsup v2: rewrite the website around 'albums' 8 years ago
Romain Prieto 035b960c4a Fix rotation bug for images and videos (more details in PR) 8 years ago
Navneet Karnani c8c5947c9b Add support for custom start file 8 years ago
Romain Prieto 98d98c40bc Fix relative path to LightGallery (npm dedupe issue) 8 years ago
Romain Prieto b4378356f1 Update all packages, now compatible with Node 5 8 years ago
Romain Prieto 837f5eee5c Align progress bar for the static website 8 years ago
Romain Prieto f7c6e70b29 Read EXIF captions, and display at the bottom of the slideshow. 9 years ago
Romain Prieto 7d6afac2d0 Upgrade to LightGallery 1.2.2, and use VideoJS for playing videos 9 years ago
Romain Prieto 9473462871 Move to lightgallery instead of blueimp (had video player issues) 9 years ago
Romain Prieto 543e9148b0 Fix bug reading EXIF metadata (error spawn EMFILE)
When too many files needed to be processed, it would try to spin up the EXIF tool (ffprobe) on all files in parallel.
Setting the parallel limit to 100 fixes it.
9 years ago
Romain Prieto 718210b378 Fix #23: URLs should only have forward slashes + catch EXIF errors 9 years ago
Romain Prieto 1ca3cfe7df Support for "m2ts" format (and smaller video example) 10 years ago
Romain Prieto e74733f52d Homepage with grid of 4 images per gallery 10 years ago
Romain Prieto 82c560d96f Big refactor / cleanup to support multiple pages on the website 10 years ago
Romain Prieto 5de02a543f Always display all build steps, with "up to date" if nothing to do 10 years ago
Romain Prieto beb9bf403d Refactor view model URLs 10 years ago
Romain Prieto 6f53b594c5 Bug fix #18: download link points to "large size" if original is not available 10 years ago
Romain Prieto 745779fa9f Fix EXIF date parsing 10 years ago
Romain Prieto b031d043f6 Fix #20, rotate video previews if the video itself has rotation EXIF data
note: no need to rotate the video itself, the player reads it properly
10 years ago
Romain Prieto 1bd5ac8298 Always print an empty progress bar first, in case the first file take a while to process 10 years ago
Romain Prieto cb835d0ed2 Part of #20, auto-rotates photos based on the EXIF data 10 years ago
Romain Prieto ef294ad513 Fix #18: don't upload original media by default, add new command line options 10 years ago
Romain Prieto c72f46e463 Fix #19: support for MTS files, with full-frame mp4 export 10 years ago
Romain Prieto 92c6d6b9d3 Fix #12, add support for --google-analytics config (uses latest GA tracking code) 10 years ago
Romain Prieto 2037bf4455 Remove deleted media from the generated HTML pages 10 years ago
rprieto 58f1cb5c5e Fix error in new options 10 years ago
rprieto f1f09adc14 Sort folders by date or by name 10 years ago
rprieto 3c8ecce018 Use <yargs> to parse command line args, and support JSON config 10 years ago
rprieto 436d126050 Remove <regen> dependency (faster builds)
Since we read all the file metadata for EXIF dates, and we need it as well for the view model,
we should use it to generate the thumbnails and save many calls to glob() and fs.stat()
10 years ago
rprieto a13b94ea3e Fix view model for videos 10 years ago
rprieto 7f7b36ab7b Improve metadata collection (single file stat, ...) 10 years ago
rprieto 8fc19fab97 Metadata fake progress report 10 years ago
rprieto 63ee950684 Use the pre-built metadata to create the galleries view model
Fixes #4 : sorts the files by either EXIF date (photos) or file date (videos)
10 years ago
rprieto 11575c65d4 Read and cache EXIF data into <metadata.json> 10 years ago
rprieto ba8c4a59f3 Fix website generation (bad refactor) 10 years ago
rprieto 1687999976 Fix #1 : report progress of operations in the console 10 years ago
rprieto d7d9a9e3a9 Fix #3. Display web-friendly videos (smaller size, and keeps original) 10 years ago
rprieto 88ad9d6d05 Fix over-eager copy of original media 10 years ago
rprieto 8a468026da Fix "split" title when it's more than 2 words 10 years ago
rprieto 8531b26b2a Use modified time instead of creation time when copying folders 10 years ago
rprieto 8b763ef509 Upgrade to regen 0.0.3 10 years ago
rprieto 3c2c3aae96 Extract reusable "regen" module to process source / dest files 10 years ago
rprieto eaa8bd2831 support for custom title and CSS 10 years ago
rprieto 9a0234531c Large photos in 90% quality (smaller sizes, usually still OK) 10 years ago
rprieto 7970913ec7 Fix thumbnail sizes in HTML 10 years ago
rprieto 1608b3cfac Generated website contains original media too
- now generate thumbs + large (original is too big for web download)
- simpler build system
10 years ago
rprieto 3a388181ee Fix thumbnail size in HTML to avoid jumping content while loading 10 years ago
rprieto 551d5d5752 Better thumbnail quality 10 years ago
rprieto 288ee7f87c Only create thumbs / copy files if they have changed 10 years ago
rprieto 1268f5557e Remove wrench: using glob() everywhere 10 years ago
rprieto 3238707ba5 remove unused packages 10 years ago
rprieto aa055e2baf Cleanup thumbnail generation 10 years ago
rprieto e938aa811a Sort photos by date (newest at the end) 10 years ago
rprieto 12c9b3746a Cleanup 10 years ago
rprieto e91d2bcee5 Read file dates for sorting 10 years ago
rprieto 4a86ddc385 Remove unused template 10 years ago
rprieto 7b20cf1722 Remove Gulp dependency (see below)
Gulp is very good at buffered / streamed content.
However GraphicsMagick + FFmpeg work well with file paths,
so it's easier to just glob(**) and call the command line tools.
10 years ago
rprieto 9a75d7e672 Extract thumbs into separate module (moving away from gulp) 10 years ago
rprieto 2ddaa83728 BlueImp also supports MOV files 10 years ago
rprieto ddcf0c48aa Working "bin" script, and fix path for use from other modules 10 years ago
rprieto b9c315165a Spike to generate video thumbnails + poster with the right aspect ratio 10 years ago
rprieto dd93e87f42 Video thumbnails - no poster yet 10 years ago
rprieto dfd059cc1f support custom thumbnail sizes 10 years ago
rprieto 4859d3a6b2 index page = first gallery 10 years ago
rprieto 235a9c60e4 Activate BlueImp gallery viewer 10 years ago
rprieto 62c0d09868 Initial commit 10 years ago