Gijs Kruitbosch
7cf95bd427
Fix same-line loops and if statements
6 years ago
Gijs Kruitbosch
d9f7bb2965
Fix quotes
6 years ago
Gijs Kruitbosch
7d03bec52d
Fix issues with finding nytimes content caused by in-article ads
6 years ago
David A Roberts
ea4165721f
Remove single-cell tables
6 years ago
David A Roberts
72bd1a8532
Don't nest paragraphs
6 years ago
David A Roberts
afcc4b8e49
Fix titles not being trimmed sometimes
6 years ago
Gijs Kruitbosch
d4b842c82a
Match headings on trimmed strings to avoid whitespace causing mismatches
6 years ago
Gijs Kruitbosch
8c02a0d34c
Fix #283 and remove hidden nodes
6 years ago
David A Roberts
656a6673d9
Don't put non-phrasing content into paragraphs
6 years ago
David A Roberts
5ae90930cd
Don't convert DIVs to Ps when more than 25% links
6 years ago
David A Roberts
9f2c5cb42e
Put phrasing content into paragraphs
...
This removes the need for `p.readability-styled` elements.
6 years ago
Gijs Kruitbosch
f4ab856992
Check for a document being passed
...
This provides a descriptive error message if no document is passed, and
ignores the first argument if the second argument looks like
a reasonable DOM document instance.
6 years ago
David A Roberts
7a24801958
Don't include root html node in candidates
...
Fixes #435
6 years ago
David A Roberts
d60184966c
Remove unused URI parameter from constructor
6 years ago
David A Roberts
5ee03bc960
Stop Readability depending on Node.* constants
6 years ago
Andres Rey
6fd816496c
Clean <aside> tags on _prepArticle
6 years ago
David A Roberts
8414158fa9
Fix _replaceBrs
...
Previously, `nextElem` was not actually proceeding to the next element, and therefore aborting the paragraph at the first `<br>` (rather than the first `<br><br>` as the comment indicates).
6 years ago
Joan Espasa Arxer
3ff9a166fb
Changed wordThreshold to charThreshold to better reflect the semantics.
6 years ago
Brad Philips
8525c6af36
Fix relative URIs given <base> tags ( #422 )
6 years ago
Andres Rey
834672ef86
Return longest text after failing to detect text longer than the configured value ( #423 )
...
Save extracted text across attempts and return the longest one when all attempts fail, and add a test case from hukumusume
6 years ago
Tom Z?hner
264b8e8968
Remove link elements when preparing article for display
6 years ago
Andres Rey
01ffd0c617
Remove "modal" from strings to remove
7 years ago
Gijs Kruitbosch
807bf05aa3
Fix className usage so it deals correctly with SVG nodes ( fixes #412 ).
7 years ago
Gijs Kruitbosch
092a8aeaff
Revert removing ids from elements
7 years ago
Andres Rey
9ce4d87232
Fall back to the original title if after trimming the text we have too many words before the colon.
7 years ago
Andres Rey
5a5c8ba1a2
Add node to elementsToScore when _hasSinglePInsideElement is true
7 years ago
Cameron McCormack
6729538c77
Clean IDs and classes from output.
7 years ago
Björgvin Ragnarsson
c3ff1a2d2c
remove dead code
7 years ago
Iqbal Ahmed
b3fde168cb
Allow the word threshold parameter to be configurable
7 years ago
Taylor Hunt
b7c32feb25
Remove presentational HTML attributes ( #385 )
...
* Remove presentational HTML attributes
Fixes #383
This patch loops through a list of known-presentational attributes in HTML, attempting to remove each from each cleaned element. (Checking for the attribute's existence first seems to just add needless overhead.)
The extra check for the HTML namespace is to avoid removing attributes that inline SVG needs.
* Only remove `width`/`height` for certain elements
Embedded media elements are allowed to have them, but not others.
* Address PR feedback
* Fix loop index formatting
* Only remove `width`/`height` from certain elements
* Combine logic into a single check/remove
* Attempt fixing my recursion
* One weird trick to get your loops to run
* Add inline SVG bailout
Try not to touch any styles for `<svg>`, because it's inherently presentational.
* Update tests to match newly-removed attributes
* Oh those wacky SVGs
The `position:absolute` is a trick to import clipping paths into the document without putting a big 300×150 empty space in it. (`display:none` and such disable the clipPath in some browsers.)
* Whoops, missed some `width`s
* Normalize SVG tagName
JSDOMParser differs from the official DOM here
7 years ago
Gijs Kruitbosch
c387d12bc4
Fix eslint issue by unpacking ternary into a proper if statement
7 years ago
Gijs Kruitbosch
a63e63c91e
Clarify title-splitting code (part of #357 )
...
This adds a number of comments and a wordCount helper function to try to make the code
a bit more readable (which seemed appropriate considering the number of regular expressions).
7 years ago
Farbdose
aed7f10594
Remove general part of title when viewing websites with hierarchical titles
7 years ago
Evan Tseng
c7c26382b9
Bug 1217007 - Remove nodes without any content, r=Gijs
7 years ago
Evan Tseng
1369f923db
Merge pull request #348 from gijsk/1322674-table-divs
...
Fix bug 1322674 by only conditionally removing content from tables when the tables are layout tables, r=evanxd
7 years ago
Gijs Kruitbosch
9baea36169
Fix bug 1322674 by not removing content in data tables
7 years ago
Evan Tseng
19f2124c82
Bug 1259763 - Remove h2 when there is only one h2 and its text content substantially equals article title, r=Gijs
7 years ago
Evan Tseng
15e1f03261
Bug 1300697 - Reader View missed first few paragraphs on New York Times website, r=Gijs
7 years ago
andrei-ch
5b1e69bdf2
Bug fix: still not grabbing full content from nytimes.com articles
...
Solution: strip one level of empty <DIV> elements so they don’t
obstruct merging adjacent content downstream.
7 years ago
andrei-ch
486927ebd9
Bug fix: many pages only grab partial content (dirty.ru, nytimes.com)
...
1) Avoid conversion of whitespace text nodes into paragraphs. They
create a lot of noise and actually prevent sibling joining logic from
working in many pages.
2) Handle case when adjacent content is actually located in parent's
sibling node instead of top candidate’s sibling.
7 years ago
Gijs Kruitbosch
2b28d6022f
Merge branch 'Treora-commonjs'
7 years ago
Gijs Kruitbosch
f1797fcf7e
Merge branch 'commonjs' of https://github.com/Treora/readability into Treora-commonjs
7 years ago
Gerben
43c54e8a71
Fix tiny mistake (typeof dump !== 'undefined')
7 years ago
Evan Tseng
e1c7170d85
Merge pull request #309 from evanxd/bug-1167568
...
Bug 1167568 - Find a better topCandidate if there are other nodes scores are high enough, r=Gijs
7 years ago
Evan Tseng
131d923d38
Bug 1167568 - Find a better topCandidate if there are other nodes scores are high enough, r=Gijs
7 years ago
Gerben
d63576be0f
CommonJS support: set module.exports = Readability
7 years ago
Evan Tseng
cec7d90a14
Merge pull request #336 from gijsk/remove-ews-hack
...
Remove hack for EWS content management system, r=evanxd
7 years ago
Gijs
0d3043dacb
Remove hack for EWS content management system
...
Per issue #322
7 years ago
Sebastian Hengst
5e9c7a3910
Bug 1322327 - Only remove images which are not descendants of a figure if there is more than one image, update affected tests
8 years ago
andrei-ch
c5ff44d8fe
Clean <input>,<textarea>,<select>,<button> elements
8 years ago
Evan Tseng
63230a307a
Bug 1142312 - Add two more types of unlikely candidates: cover-wrap and yom-remote, r=Gijs
8 years ago
andrei-ch
4a0d08c56a
font-to-span conversion skips half the font elements on 'real' DOMs
8 years ago
Evan Tseng
e84c0c3f07
Bug 1285543 - Only use "og:title" or "twitter:title" if _getArticleTitle does not return a valid title, r=Gijs
8 years ago
Evan Tseng
33dc8fa023
Bug 1255978 - Remove legends candidate, r=Gijs
8 years ago
Evan Tseng
af0aa5c59f
Bug 1173548 - Find out text direction from ancestors of final candidate, r=Gijs
8 years ago
Evan Tseng
4fa0d1b207
Bug 1177619 - Score div nodes which have br nodes. r=Gijs
8 years ago
Taylor Hunt
71aa562387
Add microformats2 class names to heuristics ( #303 )
...
Microformats updated their old `hentry` to [a newer
`h-entry`](http://microformats.org/wiki/h-entry ).
With the [number of IndieWeb sites breaking into the
ten-thousands](http://tantek.com/2016/190/b1/state-of-indieweb-summit ),
this seems like a fair idea.
8 years ago
Gijs
1a12befa41
Fix code style, tighten up eslint rules ( #301 )
8 years ago
Ivan Persidsky
fd11f92adb
Use a dedicated method and backward iteration for removing nodes ( #300 )
...
This improves compat with "real" DOMs that provide a live NodeList as the return value of getElementsByTagName.
8 years ago
Gijs Kruitbosch
140d4c4aca
Only compute textContent once.
8 years ago
usergit
327bfcb93f
exposed textContent to be returned
...
this returns the text content only, this is useful as it allows the content to be easily accessible
8 years ago
Gijs
69b81f5d70
Fix #287 : convert getElementsByTagName result to an array ( #288 )
8 years ago
Gijs Kruitbosch
46b08a5ea5
Address issue #277 by marking 'modal' unlikely+negative
8 years ago
Peter deHaan
b380917b4b
Convert nested function declaration to function expression
8 years ago
Gijs Kruitbosch
e830ac9dd8
Fix eslint issues identified in m-c
8 years ago
Gijs Kruitbosch
dffa760c04
Fix issue #267 by ignoring hash URIs when making URIs absolute
8 years ago
Gijs Kruitbosch
a9597efc17
Fix bug 1230050 by checking for the 'hid' class specifically, r?MattN
9 years ago
Gijs
a801846a45
Merge pull request #204 from mozilla/tweak-great-grandparent-scoring
...
Updated great grandparent node scoring.
9 years ago
Nicolas Perriault
ae0833522c
Improved embedded video elements detection.
9 years ago
Nicolas Perriault
46304bb5fe
Updated great grandparent node scoring.
9 years ago
Nicolas Perriault
88ef3893b5
Fixes #180 - Score intermediary headings.
9 years ago
Nicolas Perriault
dc1b2c9fa0
Refs #195 - Exclude nodes likely to be related content.
9 years ago
Nicolas Perriault
cc18cb5787
Ref #195 - Add support for dailymotion videos.
9 years ago
Nicolas Perriault
9dbc009376
Fixes #113 - Recursive node ancestor scoring.
9 years ago
Nicolas Perriault
44879722b6
Fixes #183 - Preserve list items.
9 years ago
Gijs
79aa2fca87
Merge pull request #189 from mozilla/dont-remove-headings
...
Fixes #150 - Keep article intermediary headings.
9 years ago
Margaret Leibovic
af6da2a87d
Merge pull request #190 from mozilla/improved-author-meta-extraction
...
Improved author metadata detection.
9 years ago
Nicolas Perriault
7aee44adb2
Improved author metadata detection.
9 years ago
Gijs Kruitbosch
5f184053cd
Make isProbablyReaderable include <pre>, and deal with long <br>-separated paragraphs and/or shorter-than-5-paragraph text and such.
9 years ago
Nicolas Perriault
2451a07a7d
Fixes #150 - Keep article intermediary headings.
9 years ago
Margaret Leibovic
319a50b4f0
Fixes #184 - Don't strip class names from article content
9 years ago
Gijs
49e40768aa
Merge pull request #185 from mozilla/score-section-tags-by-default
...
Fixes #139 #143 : Added more weight to section tags.
9 years ago
Nicolas Perriault
f6ffa6acde
Fixes #139 #143 : Added more weight to section tags.
9 years ago
Nicolas Perriault
58cd789cd3
Improved title extraction 'algorithm'.
9 years ago
Gijs
b37ff08bc7
Merge pull request #169 from mozilla/clean-footer-tags
...
Fixes #163 - Avoid including footer tag contents.
9 years ago
Nicolas Perriault
12c6a11f67
Fixes #163 - Avoid including footer tag contents.
9 years ago
Nicolas Perriault
6eeabf90c1
Fixes #164 - Add support for title alt semantic metadata.
9 years ago
Gijs Kruitbosch
0ff82de0f4
Implement createTextNode, do more relaxed escaping there, update testcase.
9 years ago
Margaret Leibovic
37a8cd4171
Bug 1147584 - Don't remove unlikely <a> tags, and replace <a> tags with their text content if they won't be useful links
9 years ago
Gijs
a6014f5854
Merge pull request #132 from gijsk/heise-ad-prioritization
...
Don't look at banners and skyscrapers, remove <noscript> elements
9 years ago
Gijs Kruitbosch
a6346a0ad4
Don't look at banners and skyscrapers, remove <noscript> elements
9 years ago
Nicolas Perriault
4424b0bad7
Refs #128 - Add support for options to Readability constructor. r=@gijsk
9 years ago
Nicolas Perriault
4d41f5e4ed
Refs #117 - Drop social/share buttons.
9 years ago
Gijs Kruitbosch
7c60dba3b6
Fix Readability.js to work with jsdom's DOM implementation (in particular: no firstElementChild implementation...)
9 years ago
Margaret Leibovic
eb3a8e8dc4
Bug 1150695 - Move isProbablyReaderable function to Readability.js
9 years ago
Nicolas Perriault
f8d37e4276
Don't remove elements containing figures or having them as a parent.
9 years ago
Nicolas Perriault
b6730703a1
Fixes #81 - Keep article images.
9 years ago
Gijs
194a5376c8
Merge pull request #63 from mozilla/preserve-embedded-tweets
...
Preserve inline tweets as they're part of article contents.
9 years ago
Gijs Kruitbosch
b4332328f3
Fix an issue where we don't track scores for the parents appropriately.
9 years ago
Gijs
14b33b69db
Merge pull request #65 from mozilla/support-embed-videos
...
Fixes #56 - Updated support for embedded Youtube & Vimeo videos.
9 years ago