migrating moodle from one server to another

When you migrate moodle from one server to another, you have to take a copy of three things and make certain changes. There are some caveats however which I will mention here.

  • Moodle has three main parts, the app itself, the database, and the uploads and cache data repository.
  • We are going to assume that the following paths are used by Moodle:
  • /var/www/moodle/ - the app itself, containing folders like "admin, analytics, auth, availability, badges..." and importantly, the config.php file
  • /var/www/moodle_data/ - the app cache data and uploads, containing the folders "cache, diag, filedir, lang, localcache, lock, muc, sessions, temp, trashdir"

Let's assume for this example that your database is called moodle, your database username is moodle, and your password is abc123. Let's also assume your target server allows you to upload files as root over ssh/scp.

We divide the instructions up into source server and target server/new server.

Let's assume your source server is called www.source.edu and your target server is called www.target.edu.

On the SOURCE server:

1. Copy the database out of MySQL to the target server.

mysqldump -u moodle -p moodle > moodle.sql 

This means, dump the database (mysqldump) called moodle, act as the user (-u) moodle, ask me for my password (-p), and store (>) the SQL data in a file called moodle.sql

du -sh moodle.sql
[ this should tell you how big the file is ]
gzip moodle.sql
du -sh /var/www/moodle.sql.gz
[ you will see it is much smaller now ]

compress the SQL as it is big and this will reduce it by about 50% for transfer.

rsync -avu moodle.sql.gz root@www.target.edu:/var/www/ --progress --rsh="ssh -p22"

This command means: synchronise remotely (rsync) the file moodle.sql.gz to the server www.target.edu and act as the user root. Store the file in /var/www/ on the target server, give progress, and use the ssh command to send the file in encrypted form. Technically you can leave out the --rsh part; I put it here in case you use a different port number for rsync.

At this stage I'd consider checking that the file arrived. Login to the target server and check the file is there:

du -sh /var/www/moodle.sql.gz

It should read the same size as the original. If it did not copy properly, try again and think carefully about why it failed. A common issue is that most servers DO NOT allow root login over SSH. You might have to copy the file as yourself using your own user account, e.g. john@www.target.edu. 

2. Copy the moodle application to the target server. Make sure this copy succeeds.

 rsync -avu moodle/ root@www.target.edu:/var/www/moodle/ --progress --rsh="ssh -p22"

3. Copy the moodle data repository to the target server. Make sure this copy succeeds.

 rsync -avu moodle_data/ root@www.target.edu:/var/www/moodle_data/ --progress --rsh="ssh -p22"

4. Shut moodle down (it's handled by the webserver apache).

 service apache stop

Note that you only need to do this if (a) you are never using the SOURCE server again, and (b) there are no other websites on the source server.

5. Delete the database. NB: ONLY DO THIS IF YOU ARE CERTAIN YOU COPIED THE SQL OVER TO THE NEW SERVER.

mysql -u root -p
drop database moodle

exit

6. Delete the app and repository. NB: ONLY DO THIS IF YOU ARE CERTAIN YOU COPIED THEM OVER TO THE NEW SERVER.

 rm -rf /var/www/moodle/ ; rm -rf /var/www/moodle_data/

Explanation of the next steps

  • Before we give the steps, we must make you aware of a problem migrating Moodle. One of the things that Moodle does is hardcode paths and URLs of images, videos etc., into the database. That is, if the database knew about a file called myfile.jpg on the server www.source.edu, it will still keep a record of www.source.edu/moodle/myfile.jpg even though you moved it to another server. This means that when you migrate the server, it will STILL look at the OLD (source) server for the files. Since you deleted them in steps 4-6 above, the images and videos etc., will come up as broken link icons. You can verify this is the problem by choosing the right-click mouse button/trackpad button on a broken image, and choose "copy URL", then paste the URL in a text editor. You will see it contains your SOURCE server URL. 
  • We explain how to fix this below.
  • The other thing Moodle does is cache pre-built pages that do not change often. For example, lesson plans. These pages are stored in a cache directory at /var/www/moodle_data/cache or similar. You can check which folders contain this using grep, as shown below.

On the TARGET server:

1. Check the app and repository are in place:

 ls /var/www | grep moodle

if the above command returns nothing it means you either are still on the SOURCE server *or* you didn't copy it successfully. In which case, if you performed steps (4-6) above, you are doomed. If however it lists two folders, namely, moodle/ and moodle_data/ , then you should be ok.

2. Check that the paths are the same on both servers. If not, edit the config file. You need to at least view this file anyway to get the database name, user name from the database, and password.

The config file should look exactly like this, IF your username is moodle, your password is abc123, and the paths are as given above:

unset($CFG);

global $CFG;

$CFG = new stdClass();


$CFG->dbtype    = 'mysqli';

$CFG->dblibrary = 'native';

$CFG->dbhost    = 'localhost';

$CFG->dbname    = 'moodle';

$CFG->dbuser    = 'moodle';

$CFG->dbpass    = 'abc123';

$CFG->prefix    = 'mdl_';

$CFG->dboptions = array (

  'dbpersist' => 0,

  'dbport' => '',

  'dbsocket' => '',

  'dbcollation' => 'utf8mb4_unicode_ci',

);


$CFG->wwwroot   = 'https://www.target.edu/moodle/';

$CFG->dataroot  = '/var/www/moodle_data/';

$CFG->admin     = 'admin';


$CFG->directorypermissions = 0777;


require_once(__DIR__ . '/lib/setup.php');

3. Open MySQL and create the same database (named the same), and create a user with the same name and password as previously. Give the user privileges on that database.

mysql -u root -p

create database moodle;

create user 'moodle'@'localhost' identified with mysql_native_password by 'abc123';

grant all on moodle.* to 'moodle'@'localhost';

flush privileges;

exit;


If the target server already has a database called moodle, then you need to use a different name for your database AND make sure the change is reflected in the config.php file.

The instruction

identified with mysql_native_password

is because Moodle was written on MySQL 5 which has an older type of password, and if you don't do this, it MIGHT refuse to let you access the database.

4. Edit the SQL file from the SOURCE server and change any hard-coded paths and DNS entries. For this exercise we are going to use VI because it is capable of editing extremely large text files without thrashing and without inserting carriage returns into sensitive things like SQL.

gunzip moodle.sql.gz
vi moodle.sql

Once VI opens, you have to search for your previous server name and replace it with the new server name.

:%s:www\.source\.edu:www\.target\.edu:g


ESC

ZZ


The above three lines in VI achieve the following in order:
  • substitute (:%s) www.source.edu with (:) www.target.edu, globally (:g)
  • Note that the above is an example of a search and replace regular expression. The colons surrounding the search criterion and the replacement string, can also be replaced with forwardslash symbols (/) but we use colons here in case there are subfolders involved, e.g. www.source.edu/moodle/site1/
  • Note that the substitute command considers punctuation such as :, ., \, / etc., as special, so if you literally mean a dot, you have to put \. (backslash dot). The same for forward-slash (/). If the URL contains a forward slash, you have to put backslash as well: \ / like so.
  • Esc means to exit the editing mode by pressing the key marked Esc.
  • ZZ means to save and exit. Capital Zs. 

5. Insert the SQL from the SOURCE server into the MySQL database on the TARGET server.

mysql -u root -p moodle < moodle.sql

Note that if you had a few thousand users, chances are that the above command will take about half an hour.

6. Delete the cache directory on the TARGET server so it does not cache pre-built PHP pages with bad paths/URLs embedded in them.

cd /var/www/moodle_data/cache
grep -ilrs "www.source.edu" *

The above command (grep) will search INSIDE files in the cache to see which contain mentions of the SOURCE server. You must then delete those files using rm (remove).

You can do this in one swoop like so:

grep -ilrs "www.source.edu" /var/www/moodle/data/cache | xargs rm


This says: search for www.source.edu and pipe ( | ) (send) it into the RM command.

7. You should now be able to go to www.target.edu/moodle/ and login as normal in your web browser. 

Final caveat

You may find there is a PHP or MySQL version difference between the two servers which breaks things. Try to ensure you have the same versions on both servers.

Popular posts from this blog

could not find course/ could not find top level course/ could not find course category

Could not find course category If you get this error (could not find course category) this basically means the course/s that exist don't have categories in the category table. Look at the mdl_course table in mysql. Column 2 is the course category. Every course listed (per row) must have a category identified in the table mdl_course_categories . For example, if you have this as your data (1, 0 , 0, "course name", " course name ", "", "summary" , 0, "site", 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, "", "", "", 1606214720, 1606215795, 0, 0, 0, 1606215383); You see the course category is zero . But there is a course category number one in the mdl_course_categories  table: |  1 | Miscellaneous | NULL     | NULL        |                 0 |      0 |     10000 |           0 |       1 |          1 |   1606214720 |     1 | /1   | NULL  | If there is a mismatch, correct it, e.g. by updating the course to belon
Read more

installing moosh to make your life easier

If your site is crashed you need to edit moodle on the command line. Enter Moosh, the moodle shell. To install moosh, do this: sudo apt-add-repository ppa:zabuch/ppa sudo apt-get update sudo apt-get install moosh Then, if you want to use moosh, you can use more or less any command shown in the web interface using a syntax of moosh -n the-command the-parameters for example cd /var/www/moodle/ moosh -n plugin-uninstall someplugin  or moosh -n cache-clear note the "cd" part - it won't work unless you do that, as it uses the config file in that directory to login. The -n part means do not ask me about my user id. Normally moodle operates as apache (www-data). Using -n lets you run it as root. Otherwise you have to do something which is in my view, worse, namely enable bash shell for apache. That opens you up to hackers, in my view, as apache is facing the web and accepts uploads on many apps such as wordpress, so it's feasible for someone to hack your server entirely if y
Read more