Copyright 2015 OpenStack Foundation This work is licensed under a Creative Commons Attribution 3.0 Unported License. http://creativecommons.org/licenses/by/3.0/legalcode
Migrate ask.openstack.org to infra¶
The OpenStack Q&A site located at ask.openstack.org currently running in an instance deployed by a third-party. The main goal of this migration to make the entire deployment process repeatable, and move the project to proper infrastructure puppet repositories.
The ask.openstack.org site was deployed manually and contains the minimal application stack required for running the site. Currently the site is missing regular backups, security updates and deployment documentation.
Create the missing puppet modules, prepare the changes in openstack-infra/system-config repo.
Leave ask.openstack.org as is.
- Primary assignee:
Use Gerrit topic “askbot-site” for all patches related to this spec.
git review -t askbot-site
- Lower ask.openstack.org DNS TTL to 300
- Create puppet-askbot split-out module
- Add vamsee-solr module and puppet-askbot to modules.env
- Make the system-config changes, add ask.pp
- Add SSL certificates and passwords to hiera
- Launch new ask.openstack.org server
- Restore database and static files from original ask.openstack.org site
- Silent testing with /etc/hosts override
- Backup / restore of original ask.openstack.org data
- Update DNS entry of ask.openstack.org with the new server address
- Redirect html traffic using nginx to new ask.openstack.org to avoid db sync issues
- Restore ask.openstack.org DNS TTL to 3600
A new puppet-askbot repository will need to be created, along with updates to system-config to consume this module.
An ask.openstack.org will need to be created.
The ask.openstack.org zone must be point to the newly created server as the last step of this migration process.
Askbot documentation need to be added to ci.openstack.org documentation.
The services will run on Ubuntu, so core operating system not requires any special attention.
The application stack have some elements that must be deployed from tar.gz or pypy instead of OS packages:
- Apache Solr (4.7.2)
Askbot don’t have integration tests implemented. After instance creation and initial data migration, I suggest to do a 1-2 week long silent test of the UI and address upcoming bugs during that period.
Backup / recovery steps¶
Askbot holds all of the site data in a postgresql database, and all static files in the filesystem. In case of the existing deployment it is available at /srv/ask/dumps directory. The dumps are created by the shell script srv/ask/node2/cron/backup.sh executed daily by cron.
Invoke management commands¶
The management cli tool is available under /srv/askbot-sites/slot0/config, so to invoke management commands you need to enter this directory first.
$ cd /srv/askbot-sites/slot0/config $ python manage.py <command> <args>
Stop celeryd and apache, jetty services¶
$ sudo service askbot-celeryd stop $ sudo service apache2 stop $ sudo service jetty stop
Restore database dump¶
Recreate an empty database:
$ sudo su - postgres -c 'dropdb askbotdb' $ sudo su - postgres -c 'createdb askbotdb --owner=ask'
Restore from backup:
$ psql -d askbotdb -h localhost -U ask -W -f /path/to/last-dump.sql
Notice you will be prompted for the ask_db_password from hiera.
Sync db and migrate:
$ cd /srv/askbot-sites/slot0/config $ sudo python manage.py syncdb $ sudo python manage.py migrate
Notice this step is required to apply schema changes between askbot versions.
$ sudo service askbot-celeryd start $ sudo service apache2 start
Rebuild solr indexes¶
$ sudo service jetty start $ cd /srv/askbot-sites/slot0/config $ sudo python manage.py askbot_rebuild_index -l en $ sudo python manage.py askbot_rebuild_index -l zh
Test the solr deployment, query string “sahara”:
$ curl "http://127.0.0.1:8983/solr/core-en/select/?fq=django_ct%3A%28askbot.thread%29&rows=10&q=%28sahara%29&start=0&wt=json&fl=%2A+score"
Notice this query must return a non-empty resultset.
$ sudo service askbot-celeryd restart $ sudo service apache2 restart
Restore static files¶
Static files must be extracted into /srv/askbot-sites/slot0/upfiles directory. It mostly holds profile pictures, and site logo, so if pictures not showing up in the site those files are missing, or have a wrong file permission.
$ cd /srv/askbot-sites/slot0 $ sudo rm -rf upfiles $ sudo tar xf /path/to/last-upfiles.tar --strip-components=2
We are using vamsee-solr module 0.0.7 from puppetforge, and it is forcing Us to use solr 4.7.2 because 4.10.x requires some extra patches to work and this upgrade also means a schema change.