Tightening the Rebuild

The rules this site is rebuilt under give a session one instruction when it finds something it is not authorised to fix: write the problem down and carry on with the work it is allowed to do. On 28 September every one of those open items was ruled on in a single pass, and each ruling became a fix: a change to the rules, a change to the checks, or a new lane of work with a defined order. This entry is about the fixes. There are six of them, plus a few things deliberately set aside.

1. The badge now means verified facts

Every song page carries a status badge, and the yellow review badge is a claim: it says the page has been rebuilt against the primary sources and passes the machine checks that define done. A full run of those checks across all 128 badged pages found 54 passing and 74 failing, which on its face meant the badge was false on more than half the pages carrying it.

The failures were not what they looked like. Every one of the 74 included the naming check: the house convention that musicians are referred to by first name in running prose (John, Paul, George, Ringo), with surnames kept for the credit line. Twenty-seven pages, including Dear Prudence and A Day in the Life, failed on that and nothing else. Their facts, citations and depth all passed.

The fix splits the check in two:

• Fact checks decide the badge: structure, sources and citations, depth, the releases format, quotations and the ban on editorial history.
• Style checks are reported alongside, as warnings: the naming convention, plus three new consistency checks. Once the backlog behind each one is cleared, it is promoted from a warning to a failure.

The badge now promises what a reader relies on: the facts are verified and the depth is there.

House style gets its own lane:

• The 27 style-only pages go first, five to eight per session.
• Then come the remaining occurrences, about 3,100 of them across 209 pages. Surnames that can only mean one person are replaced mechanically, with quotations protected.
• George is handled by hand, because a session has to tell George Harrison from George Martin, and a script cannot.

The three consistency checks come out of an earlier failure we wrote up as The Page That Disagreed With Itself. Nothing in the toolchain read a page against itself. Now three things will be:

• the song length in the infobox against any running time quoted in the prose;
• the number of takes against the highest take in the recording timeline;
• the recording dates against the first and last dates in the timeline.

2. One lane for equipment

Three separate defects had been found in the Equipment table on song pages:

• A stock equipment list of microphones, amplifiers, guitars and processors, inherited from the first version of the site, sat on 211 pages whether or not any of it was documented for that song.
• A borrowed photo caption. One caption in Kehew and Ryan’s Recording The Beatles describes the setup at a single session, the If I Fell session of 27 February 1964. It had been read as a general description of that year’s setup and copied onto 28 pages, eleven of them 1965 recordings that a February 1964 photograph cannot document.
• A machine that had not arrived yet. A four-track tape machine that reached Abbey Road in 1965 was named on 38 pages recorded before that. In 1964 the book puts the four-track work on a Telefunken machine. In 1962 and 1963 the band recorded on twin-track, so a simple find-and-replace would swap one anachronism for another.

These are one defect: equipment rows written from general knowledge rather than from the page of a book. The fix is a single lane, in an order where each step makes the next smaller:

1. Strip the stock list from the 87 draft pages, which make no claim to be finished.
2. Fix the borrowed caption, starting with the pages where it is unambiguously wrong.
3. Correct the machine on the 1964 pages, where the substitution is a single verified change.
4. Leave the twin-track years to each page’s own rebuild, because they need fresh source work.

Two standing rules come with the lane. Every session now reads a page’s Equipment and Quick Facts tables before rebuilding it; the prose had been reviewed carefully while the tables beside it were not. And a quarterly check now compares every machine and microphone named on the site against the year of each recording. Both the caption and the machine were found by accident, while a session was sourcing a different page. A problem that turns up twice by accident should be looked for on purpose.

3. Stricter sourcing for releases

Every song page ends with a Releases section: a listener’s guide to the versions of the song you can actually buy and hear, and how their mixes differ. Two fixes tighten it.

2009 remasters now come from one source, at one level of detail. The revised edition of Brennan’s guide to the recording variations lists the 2009 CDs song by song. An entry takes from it the year, the label, the catalogue number and which mix the disc carries, each line tied to its mix by position on the printed page. There is no release day and no engineer credits, because those are the details that had previously crept in without a source. If the book has no 2009 line for a song, the page carries no 2009 entry.

Each book now has a defined job. Brennan decides which mix a record carries; that is what his guide is for. Lewisohn’s session sheets decide when a mix was made. The case that forced the line was Mr. Moonlight. Brennan dates its stereo mix to 27 October 1964; Lewisohn’s sheet puts the stereo mixes on 4 November and the mono on 27 October. The page already said 4 November and was right. Without the rule, a session could have “corrected” it.

The stakes are real. Seven sessions in a row have found a live page stating the wrong mix for a given release. The most recent was the 2009 CD entry for Fixing a Hole, which paired the catalogue number with the stereo when the guide ties it to the mono.

4. The workings come off the public pages

Several leaks had turned up where material about how the site is built had reached pages meant for readers:

• Internal rule numbers in the prose: 76 of them on 26 pages, in sentences like “Per §1 the discrepancy is flagged”. Flagging the discrepancy is right. The rule number is a reference to a document that does not exist on this site.
• Encyclopedia links that survived an earlier clean-up, at the end of the Story section on 107 pages. One of them was on the benchmark page that new pages are modelled on, so every rebuild copied it forward.
• An old row of reference-site links on 47 album, EP and single pages.
• Our own progress report. The Analysis page now opens with it, and its “recent sessions” list was printing the internals of the build: session types, tool names, commit hashes and lane codes.

The fix is one clean-up lane:

• rule numbers come out sentence by sentence, so each sentence still reads properly afterwards;
• the encyclopedia link comes off the benchmark page first, so it stops spreading, and then off every page;
• the link row comes off the hub pages;
• the Analysis report is rewritten in plain words.

Removing those links touches the Story sections, which are otherwise frozen until they are rewritten in full once enrichment is complete. The ruling makes an explicit exception: a link can be removed without the prose around it changing.

5. Four steps at the end of every session

Each build session now finishes with four fixed steps:

1. The progress report is rebuilt, on both the Analysis page and the full report, from the pages themselves: nothing is typed in by hand.
2. Every page the session published is fetched from the live site and read from top to bottom, tables included. Twice now, a problem that passed every automated check was caught only by reading the page the way a reader would.
3. The plan is checked for completeness and size. Every open item must be visible on the progress report; one had been written in a format the report generator skips, and the plan now states the format it needs. The plan’s summary holds only the last two sessions.
4. A lock is taken at the start and released at the end, and the published version is checked before the push. On 27 September two sessions were working in the same repository at once, and one of them published a page before the session rebuilding it had deployed.

6. Progress readers can watch, and a plan half the size

The homepage’s “New in this version” panel was rewritten to describe what the rebuild is for rather than what it did last. Three additions to the Analysis page are queued as the next piece of work:

• a Recently verified list, linking to each newly verified song with a one-line note of what changed;
• a feed you can subscribe to;
• an “updated so many hours ago” stamp that your browser works out from the time the report was built.

The site stays static. The figures refresh every time work is published, not every time the page loads, and that is on purpose.

The plan itself was also cut down. Its summary section had grown to 178 KB of running session notes, kept in the one place meant to hold only current state; the whole plan was 365 KB. It is now 184 KB. Nothing was deleted: the 144 older entries were moved word for word into the project’s history file, where history belongs. A plan for a project that is getting closer to done should get shorter over time, and from now on its size is checked rather than hoped for.

Set aside, deliberately

• Awards, notable covers and chart positions no longer get their own song-page sections. The page structure is locked, and chart and certification data belong on the single and album pages, where the release they describe lives. Covers may become an article.
• The newsletter and community work is deferred until the rebuild is finished. What ships first is the feed of newly verified pages, which costs nothing and can feed a newsletter later.
• The advertising application is on hold until the site plan is fully complete.

The rest of the week

The rebuild itself kept moving. In the last three days alone, six pages were re-authored against the sources:

• I’m Happy Just to Dance With You was taped on Sunday 1 March 1964, the group’s first Sunday session and the day before filming began. It was given no separate American mono, so one March mono served every market.
• Martha My Dear and Honey Pie were rebuilt together from the same October 1968 Trident sessions. Both live pages had said the American White Album was issued in mono and stereo; both books tie its catalogue number to stereo only.
• Good Morning Good Morning and Lovely Rita were rebuilt as a Sgt. Pepper pair. Lovely Rita’s page had quoted a vocal tape-speed figure that, on the printed page, belongs to Lucy in the Sky with Diamonds. The correct figure and date are now on the page, along with the correct date for its stereo mix.
• Can’t Buy Me Love now sets out a disagreement rather than settling it. Lewisohn has the vocal and lead-guitar overdubs already on take four in Paris. Kehew and Ryan have only the backing track there, completed at Abbey Road. Geoff Emerick recalls an Abbey Road hi-hat repair that Norman Smith denies. The page states all three and flags the conflict. Where the sources do not agree, the reader is told so.

None of the six fixes adds a new kind of work. Each makes an existing check mean what it says, puts related repairs in the order that makes them smallest, or makes a session finish the same way every time.