Wiki Upgrade

2017-12-28T04:01:29Z

Benedict:

== Eternal Wiki Upgrade Quest ==

This is the beginning of the wiki upgrade page....

== Requirements ==

# https by letsencrypt
# debian 9
# no snowflakes, all our (known) devops culture at nb is ansible
# salt the passwords correctly

== Chosen Technologies ==

# Ansible
# Debian 9
# Caddy
# Letsencrypt / Certbot

== Ansible Roles ==

Noisebridge already has an ansible infrastructure repo, please join the Noisebridge github group and hack this: https://github.com/noisebridge/ansible/

# Certbot - https://github.com/geerlingguy/ansible-role-certbot
# Semantic Mediawiki - Mediawiki maintains some: https://phabricator.wikimedia.org/diffusion/1881/
## It may be useful to look at this one: https://github.com/yongxinL/ansible-mediawiki

== Database Hacking ==

We need to upgrade a really old version of mediawiki and will likely need to futz with the database.
# Migrate to postgresql?
## make a git repo to store migration sql and other sql used for massaging the database contents
# Delete old passwords (md5 with no salt) and expire all user passwords?
## Write/publish public sql script to clean up the production database for public publishing
# Clean the database of spam/cruft?
## publish these scripts to the same sql script repo

=== MediaWiki-Vagrant ===

This potentially is a good environment for testing and migrating the database: https://www.mediawiki.org/wiki/MediaWiki-Vagrant

(more OpenStack Semantic MediaWiki images: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS)

You'll need to follow this guide to get a dump of our database up and running: https://www.mediawiki.org/wiki/Manual:Moving_a_wiki

=== Password Rollover ===

Here is the mediawiki default user table data dictionary: https://www.mediawiki.org/wiki/Manual:User_table

We need to delete the password column and expire everyone's password. The old mediawiki only stores md5 hashed passwords with no salts, so we'll have to delete the whole thing

<code>
user_password_expires

MediaWiki version: ≥ 1.23

Date when user's password expires; null for no expiration date. Can also be set manually by calling User->expirePassword().

</code>

== Noisebridge Wiki Database Fixing w/ Kevin & Trent 2017.12.23 ==

This is a log of what we did to a local copy of Noisebridge wiki database. These changes were not propogated to the live/production Noisebridge MediaWiki instance.

=== Things We Deleted ===

# purged password hashes - noisebridge_mediawikiuser.user_password all set to '' (empty string)
## update <table> set <column> = ' ';
# purged user emails - noisebridge_mediawikiuser.user_email all set to '' (empty string)
## update <table> set <column> = ' ';
# purged noisebridge_mediawikiipblocks.ipb_address - replaced the composite primary key column with a sequence as follows: https://stackoverflow.com/questions/2643371/how-to-renumber-primary-index
## truncated user watchlists (these are private)
### truncate table noisebridge_mediawikiwatchlist;
# truncated all protected tables (these might give private info on who protected the page?)
## truncate table noisebridge_mediawikiprotected_titles;

=== Overview and Instructions for Installing MediaWiki & Noisebridge Wiki via Vagrant ===
* MediaWiki upgrade for NB
** Follow directions for setting up local Vagrant instance on your development machine:
*** https://www.mediawiki.org/wiki/MediaWiki-Vagrant
*** Install vagrant and virtualbox (OS X: http://sourabhbajaj.com/mac-setup/Vagrant/README.html)
*** # Open a terminal (we'll call this LOCAL_TERMINAL)
*** $ cd vagrant
*** # We're going with the most recent stable LTS release (Long-Term Support)
*** $ vagrant hiera mediawiki::branch REL1_27
*** $ ./setup.sh
*** $ vagrant up
** Now, copy the SQL dump file into your running MediaWiki virtualbox instance
*** # in LOCAL_TERMINAL... (the Vagrant host bridge interface always gets an IP address of 10.11.12.14)
*** $ scp -i .vagrant/machines/default/virtualbox/private_key upgraded_noisebridge_wiki.sql.bz2 vagrant@10.11.12.14:
** Noisebridge's MediaWiki database was configured with aall tablenames prefixed with "noisebridge_mediawiki"
** We want to ensure this naming convention continues to work with the new
*** Open a new terminal window or tab, and SSH into the running Vagrant instance (and let's refer to this shell as VAGRANT_TERMINAL)
**** $ cd vagrant
**** $ vagrant ssh
*** # In VAGRANT_TERMINAL, add (or change) the config var $wgDBprefix in /vagrant/LocalSettings.php
**** $ vi /vagrant/LocalSettings.php
****: $wgDBprefix = "noisebridge_mediawiki";
**** (NOTE: do not be clever and configure a different database name, because it turns out that `wiki` is a baked-in default somewhere in vagrant's puppet scripts. Or, maybe in mediawiki's CLI scripts, but either way IT CANNOT BE OVERRIDDEN IN LocalSettings.php or DefaultSettings.php, or --conf, or the 'dbname-prefixname' CLI parameter. If you choose to change it, just know that it is a rabbit hole that will consume an hour of your life unproductively).
** To install the Noisebridge.net wiki database on your MediaWiki-Vagrant instance, you will need to drop and recreate the MySQL database 'wiki' in order to eliminate the default MediaWiki 1.31 structure, to avoid problems overwriting it with our old MW 1.19 schema mysqldump
** # Do this in VAGRANT_TERMINAL
:: $ mysql
:: > drop database wiki;
:: > create database wiki;
:: > ^D (that's a Control-D, as in hold down the Control key on your keyboard and strike the D key)
:: $ bunzip2 -dc ~/upgraded_noisebridge_wiki.sql.bz2 | mysql wiki < ~
:: (NOTE: You may safely ignore messages like "Warning (Code 1300): Invalid utf8 character string: 'BD5A6D'")
** Now run the upgrade script: (in VAGRANT_TERMINAL)
:: $ cd /vagrant/mediawiki/maintenance/
:: $ mwscript update.php
:: If all goes as planned you'll see a bunch of advisory progress messages such as:
::: Your composer.lock file is up to date with current dependencies!
::: Going to run database updates for wiki-noisebridge_mediawiki
::: Depending on the size of your database this may take a while!
::: Abort with control-c in the next five seconds (skip this countdown with --quick) ... 0
::: Turning off Content Handler DB fields for this part of upgrade.
::: ...have ipb_id field in ipblocks table.
::: ...have ipb_expiry field in ipblocks table.
::: ...already have interwiki table
::: ...indexes seem up to 20031107 standards.
::: ...have rc_type field in recentchanges table.
::: ...index new_name_timestamp already set on recentchanges table.
::: ...have user_real_name field in user table.
::: ...
::: ...doing rev_id from 2001 to 2200
::: ...doing rev_id from 2201 to 2400
::: ...doing rev_id from 2401 to 2600
::: ...doing rev_id from 2601 to 2800
::: ...doing rev_id from 2801 to 3000
::: ...checking 40 revisions for IP edits that need copying, between rev_ids 20100 and 20300
::: ...checking 49 revisions for IP edits that need copying, between rev_ids 20301 and 20501
::: ...checking 74 revisions for IP edits that need copying, between rev_ids 20502 and 20702
::: ...checking 48 revisions for IP edits that need copying, between rev_ids 20703 and 20903
::: ...checking 41 revisions for IP edits that need copying, between rev_ids 20904 and 21104
::: ...et cetera.
** At this point you may be able to visit the Noisebridge site on your dev machine by going to:
:: http://127.0.0.1/wiki/Noisebridge
** If this doesn't work it's likely some step was omitted from this doc, in which case ask Kevin or Trent for support.

=== FUTURE Database Hacking Plans ===

==== FUTURE ACTION: After Upgrading we can expire everyone's password ====

details here: https://noisebridge.net/wiki/Wiki_Upgrade
and here: https://www.mediawiki.org/wiki/Manual:User_table#user_password_expires

==== FUTURE ACTIONS ====

# reduce permissions for inactive moderators, admins, etc, to "user".
## really drop everyone and add current people back, to make it an intentional process
## be very restrictive about CURRENT active users

Noisebridge - User contributions [en]

Wiki Upgrade