Archives
/
readability
2
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Bug 1143725 - fix the Herald Sun website

Browse Source
pull/50/head
Gijs Kruitbosch 9 years ago
parent a77dcab362
commit 955951659d
4 changed files with 1317 additions and 0 deletions
Show Stats Download Patch File Download Diff File

23
Readability.js
Unescape Escape View File

@ -634,6 +634,29 @@ Readability.prototype = {
page.appendChild(topCandidate);
this._initializeNode(topCandidate);
} else if (topCandidate) {
// Because of our bonus system, parents of candidates might have scores
// themselves. They get half of the node. There won't be nodes with higher
// scores than our topCandidate, but if we see the score going *up* in the first
// few steps up the tree, that's a decent sign that there might be more content
// lurking in other places that we want to unify in. The sibling stuff
// below does some of that - but only if we've looked high enough up the DOM
// tree.
var parentOfTopCandidate = topCandidate.parentNode;
// The scores shouldn't get too low.
var scoreThreshold = topCandidate.readability.contentScore / 3;
var lastScore = parentOfTopCandidate.readability.contentScore;
while (parentOfTopCandidate && parentOfTopCandidate.readability) {
var parentScore = parentOfTopCandidate.readability.contentScore;
if (parentScore < scoreThreshold)
break;
if (parentScore > lastScore) {
// Alright! We found a better parent to use.
topCandidate = parentOfTopCandidate;
break;
}
parentOfTopCandidate = parentOfTopCandidate.parentNode;
}
}
// Now that we have the top candidate, look through its siblings for content

5
test/test-pages/herald-sun-1/expected-metadata.json
Unescape Escape View File

@ -0,0 +1,5 @@
{
"title": "Angry media won’t buckle over new surveillance laws\n\t\t\t\t\t\t| Herald Sun",
"byline": "JOE HILDEBRAND",
"excerpt": "A HIGH-powered federal government team has been doing the rounds of media organisations in the past few days in an attempt to allay concerns about the impact of new surveillance legislation on press freedom. It failed."
}

97
test/test-pages/herald-sun-1/expected.html
Unescape Escape View File

@ -0,0 +1,97 @@
<div id="readability-page-1" class="page">
<div>
<div class="article-media article-media-main">
<div class="image">
<div class="image-frame">
<img data-src="http://api.news.com.au/content/1.0/heraldsun/images/1227261885862?format=jpg&amp;group=iphone&amp;size=medium"
alt="A new Bill would require telecommunications service providers to store so-called ‘metadat"
/>
</div>
<p class="caption"> <span id="imgCaption" class="caption-text">A new Bill would require telecommunications service providers to store so-called ‘metadata’ for two years.</span>
<span
class="image-source"><em>Source:</em>
Supplied</span>
</p>
</div>
</div>
<p><strong>
A HIGH-powered federal government team has been doing the rounds of media organisations in the past few days in an attempt to allay concerns about the impact of new surveillance legislation on press freedom. It failed.
</strong>
</p>
<p>The roadshow featured the Prime Minister’s national security adviser,
Andrew Shearer, Justin Bassi, who advises Attorney-General George Brandis
on crime and security matters, and Australian Federal Police Commissioner
Andrew Colvin. Staffers from the office of Communications Minister Malcolm
Turnbull also took part.</p>
<p>They held meetings with executives from News Corporation and Fairfax,
representatives of the TV networks, the ABC top brass and a group from
the media union and the Walkley journalism foundation. I was involved as
a member of the Walkley board.</p>
<p>The initiative, from Tony Abbott’s office, is evidence that the Government
has been alarmed by the strength of criticism from media of the Data Retention
Bill it wants passed before Parliament rises in a fortnight. Bosses, journalists,
even the Press Council, are up in arms, not only over this measure, but
also over aspects of two earlier pieces of national security legislation
that interfere with the ability of the media to hold government to account.</p>
<div
id="read-more">
<div id="read-more-content">
<p>The Bill would require telecommunications service providers to store so-called
“metadata” — the who, where, when and how of a communication, but not its
content — for two years so security and law enforcement agencies can access
it without warrant. Few would argue against the use of such material to
catch criminals or terrorists. But, as Parliament’s Joint Committee on
Intelligence and Security has pointed out, it would also be used “for the
purpose of determining the identity of a journalist’s sources”.</p>
<p>And that should ring warning bells for anyone genuinely concerned with
the health of our democracy. Without the ability to protect the identity
of sources, journalists would be greatly handicapped in exposing corruption,
dishonesty, waste, incompetence and misbehaviour by public officials.</p>
<p>The Press Council is concerned the laws would crush investigative journalism.</p>
<p>“These legitimate concerns cannot be addressed effectively short of exempting
journalists and media organisations,” says president David Weisbrot.</p>
<p>The media union is adamant journalists’ metadata must be exempted from
the law. That’s what media bosses want, too, though they have a fallback
position based on new safeguards being implemented in Britain.</p>
<p>That would prevent access to the metadata of journalists or media organisations
without a judicial warrant. There would be a code including — according
to the explanatory notes of the British Bill — “provision to protect the
public interest in the confidentiality of journalistic sources”.</p>
<p>In their meetings this week, the government team boasted of concessions
in the new Data Retention Bill. The number of agencies able to access metadata
will be reduced by excluding such organisations as the RSPCA and local
councils. And whenever an authorisation is issued for access to information
about a journalist’s sources, the Ombudsman (or, where ASIO is involved,
the Inspector-General of Intelligence and Security) will receive a copy.</p>
<p>That does nothing to solve the problem. The Government has effectively
admitted as much by agreeing that the parliamentary committee should conduct
a separate review of how to deal with the issue of journalists’ sources.</p>
<p>But another inquiry would be a waste of time — the committee has already
received and considered dozens of submissions on the subject. The bottom
line is that the Government does not deny that the legislation is flawed,
but is demanding it be passed anyway with the possibility left open of
a repair job down the track. That is a ridiculous approach.</p>
<p>Claims that immediate action is imperative do not stand up. These are
measures that won’t come into full effect for two years. Anyway, amending
the Bill to either exempt journalists or adopt the UK model could be done
quickly, without any risk to national security.</p>
<p>AS Opposition Leader Bill Shorten said in a letter to Abbott last month:
“Press freedom concerns about mandatory data retention would ideally be
addressed in this Bill to avoid the need for future additional amendments
or procedures to be put in place in the future.”</p>
<p>The Data Retention Bill will be debated in the House of Representatives
this week. Then, on Friday, CEOs from leading media organisations will
front the parliamentary committee to air their concerns before the legislation
goes to the Senate.</p>
<p>Those CEOs should make it clear they are just as angry about this as they
were about Stephen Conroy’s attempt to impinge on press freedom through
media regulation under the previous Labor government.</p>
<p>Memories of the grief Conroy brought down on his head would undoubtedly
make Abbott sit up and take notice.</p>
<p><b>LAURIE OAKES IS THE NINE NETWORK POLITICAL EDITOR </b>
</p>
</div>
</div>
</div>
</div>

1192
test/test-pages/herald-sun-1/source.html
View File

File diff suppressed because it is too large Load Diff
Write Preview
Loading…
Cancel
Save