Skip to content

Commit

Permalink
feat: add new azure sdk, deprecate old sdk
Browse files Browse the repository at this point in the history
  • Loading branch information
matthewhilton committed Nov 5, 2024
1 parent 5cb99d2 commit 47c8fdc
Show file tree
Hide file tree
Showing 14 changed files with 521 additions and 31 deletions.
17 changes: 17 additions & 0 deletions MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Migration guides

## Migrating from local_azure_storage to local_azureblobstorage

Since March 2024, Microsoft officially discontinued support for the PHP SDK for Azure storage. This means the `local_azure_storage` plugin which is a wrapper of the SDK will no longer be updated, and is already out of date with newer php versions.

The plugin `local_azureblobstorage` was created to replace this, with a simpler and cleaner API for interacting with the Azure blob storage service via REST APIs. Objectfs has been updated with a new client handler class to enable you to cut over to the new storage system as easily as possible.

This new library is only supported in higher PHP versions.

### Steps
1. If you are on Moodle 4.2, ensure you have updated the previous `local_azure_storage` to the `MOODLE_42_STABLE` branch. This fixes some fatal errors caused by Guzzle namespace conflicts.
2. Install `local_azureblobstorage` https://github.com/catalyst/moodle-local_azureblobstorage
3. In the objectfs settings, change the `filesystem` config variable to `\tool_objectfs\azure_blob_storage_file_system` and save. ObjectFS will now be using the new API to communicate with Azure. You do not need to enter new credentials, the credentials are shared with the old client.
4. Test and ensure the site works as expected.
5. If you encounter any issues and wish to revert back, simply change the `filesystem` configuration back to the old client. This will immediately begin to use the old libraries again.
6. Once you are happy, simply uninstall the `local_azure_storage` plugin. The migration is now complete.
72 changes: 45 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,45 @@
# moodle-tool_objectfs

A remote object storage file system for Moodle. Intended to provide a plug-in that can be installed and configured to work with any supported remote object storage solution.
* [Use cases](#use-cases)
* [Offloading large and old files to save money](#offloading-large-and-old-files-to-save-money)
* [Sharing files across moodles to save disk](#sharing-files-across-moodles-to-save-disk)
* [Sharing files across environments to save time](#sharing-files-across-environments-to-save-time)
* [Sharing files with data washed environments](#sharing-files-with-data-washed-environments)
* [Installation](#installation)
* [Compatible object stores](#compatible-object-stores)
* [Amazon S3](#amazon-s3)
* [Minio.io S3](#minio-s3)
* [Google gcs](#google-gcs)
* [Azure Blob Storage](#azure-blob-storage)
* [DigitalOcean Spaces](#digitalocean-spaces)
* [Openstack Object Storage](#openstack-object-storage)
* [Moodle configuration](#moodle-configuration)
* [General Settings](#general-settings)
* [File Transfer settings](#file-transfer-settings)
* [Pre-Signed URLs Settings](#pre-signed-urls-settings)
* [Amazon S3 settings](#amazon-s3-settings)
* [Minio.io S3 settings](#minio-s3-settings)
* [Azure Blob Storage settings](#azure-blob-storage-settings)
* [DigitalOcean Spaces settings](#digitalocean-spaces-settings)
* [Integration testing](#integration-testing)
* [Applying core patches](#applying-core-patches)
* [Crafted by Catalyst IT](#crafted-by-catalyst-it)
* [Contributing and support](#contributing-and-support)
- [moodle-tool\_objectfs](#moodle-tool_objectfs)
- [Use cases](#use-cases)
- [Offloading large and old files to save money](#offloading-large-and-old-files-to-save-money)
- [Sharing files across moodles to save disk](#sharing-files-across-moodles-to-save-disk)
- [Sharing files across environments to save time](#sharing-files-across-environments-to-save-time)
- [Sharing files with data washed environments](#sharing-files-with-data-washed-environments)
- [GDPR](#gdpr)
- [Branches](#branches)
- [Installation](#installation)
- [Compatible object stores](#compatible-object-stores)
- [Amazon S3](#amazon-s3)
- [Minio S3](#minio-s3)
- [Google GCS](#google-gcs)
- [Azure Blob Storage](#azure-blob-storage)
- [DigitalOcean Spaces](#digitalocean-spaces)
- [Openstack Object Storage](#openstack-object-storage)
- [Moodle configuration](#moodle-configuration)
- [General Settings](#general-settings)
- [File Transfer settings](#file-transfer-settings)
- [File System settings](#file-system-settings)
- [Pre-Signed URLs Settings](#pre-signed-urls-settings)
- [Amazon S3 settings](#amazon-s3-settings)
- [Minio S3 settings](#minio-s3-settings)
- [Azure Blob Storage settings](#azure-blob-storage-settings)
- [DigitalOcean Spaces settings](#digitalocean-spaces-settings)
- [Openstack Object Storage settings](#openstack-object-storage-settings)
- [Integration testing](#integration-testing)
- [Applying core patches](#applying-core-patches)
- [Moodle 3.9:](#moodle-39)
- [Moodle 3.8:](#moodle-38)
- [Moodle 3.4 - 3.7:](#moodle-34---37)
- [Moodle 3.3 and Totara 12:](#moodle-33-and-totara-12)
- [Moodle 3.2 and Totara 11:](#moodle-32-and-totara-11)
- [Moodle 2.9 - 3.1 and Totara 2.9, 9 - 10:](#moodle-29---31-and-totara-29-9---10)
- [Moodle 2.7 - 2.8 and Totara 2.7 - 2.8:](#moodle-27---28-and-totara-27---28)
- [PHPUnit test compatibility](#phpunit-test-compatibility)
- [Contributing and support](#contributing-and-support)
- [Warm thanks](#warm-thanks)
- [Crafted by Catalyst IT](#crafted-by-catalyst-it)

## Use cases
There are a number of different ways you can use this plug in. See [Recommended use case settings](#recommended-use-case-settings) for recommended settings for each one.
Expand Down Expand Up @@ -75,7 +89,7 @@ This plugin is GDPR complient if you enable the deletion of remote objects.
3. Clone this repository into admin/tool/objectfs
4. Install one of the required SDK libraries for the storage file system that you will be using
1. Clone [moodle-local_aws](https://github.com/catalyst/moodle-local_aws) into local/aws for S3 or DigitalOcean Spaces or Google Cloud, or
2. Clone [moodle-local_azure_storage](https://github.com/catalyst/moodle-local_azure_storage) into local/azure_storage for Azure Blob Storage, or
2. Clone [moodle-local_azureblobstorage](https://github.com/catalyst/moodle-local_azureblobstorage) into local/azureblobstorage for Azure Blob Storage, or
3. Clone [moodle-local_openstack](https://github.com/matt-catalyst/moodle-local_openstack.git) into local/openstack for openstack(swift) storage
5. Install the plugins through the moodle GUI.
6. Configure the plugin. See [Moodle configuration](#moodle-configuration)
Expand All @@ -88,7 +102,7 @@ $CFG->alternative_file_system_class = '\tool_objectfs\s3_file_system';

* Azure Blob Storage
```php
$CFG->alternative_file_system_class = '\tool_objectfs\azure_file_system';
$CFG->alternative_file_system_class = '\tool_objectfs\azure_blob_storage_file_system';
```

* DigitalOcean Spaces
Expand Down Expand Up @@ -155,6 +169,10 @@ Setup for Minio.io bucket can be found on there website [here](https://min.io)

### Azure Blob Storage

*Migration from previous API (Moodle 4.2+)*

[Migration guide from local_azure_storage to local_azureblobstorage](MIGRATION#migrating-from-localazurestorage-to-localazureblobstorage)

*Azure Storage container guide with the CLI*

It is possible to install the Azure CLI locally to administer the storage account. [The Azure CLI can be obtained here.](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)
Expand Down
10 changes: 9 additions & 1 deletion TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,22 @@ $CFG->phpunit_objectfs_s3_integration_test_credentials = array(
's3_region' => 'Your region',
);
```
* Azure:
* Azure (deprecated API):
```php
$CFG->phpunit_objectfs_azure_integration_test_credentials = array(
'azure_accountname' => 'Your account name',
'azure_container' => 'Your container',
'azure_sastoken' => 'Your sas token',
);
```
* Azure Blob Storage:
```php
$CFG->phpunit_objectfs_azure_blob_storage_integration_test_credentials = [
'azure_accountname' => 'Your account name',
'azure_container' => 'Your container',
'azure_sastoken' => 'Your sas token',
];
```
* Swift:
```php
$CFG->phpunit_objectfs_swift_integration_test_credentials = array(
Expand Down
33 changes: 33 additions & 0 deletions classes/azure_blob_storage_file_system.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
<?php
// This file is part of Moodle - http://moodle.org/
//
// Moodle is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// Moodle is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with Moodle. If not, see <http://www.gnu.org/licenses/>.

namespace tool_objectfs;

use tool_objectfs\local\store\azure_blob_storage\file_system;

/**
* File system for Azure Blob Storage.
*
* This file tells objectfs that this storage system is available for use.
* E.g. via $CFG->alternative_file_system_class
*
* @package tool_objectfs
* @author Matthew Hilton <[email protected]>
* @copyright Catalyst IT
* @license http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
*/
class azure_blob_storage_file_system extends file_system {
}
7 changes: 4 additions & 3 deletions classes/local/manager.php
Original file line number Diff line number Diff line change
Expand Up @@ -306,17 +306,18 @@ public static function check_file_storage_filesystem() {
public static function get_available_fs_list() {
$result[''] = get_string('pleaseselect', OBJECTFS_PLUGIN_NAME);

$filesystems['\tool_objectfs\azure_file_system'] = '\tool_objectfs\azure_file_system';
$filesystems['\tool_objectfs\azure_file_system'] = '\tool_objectfs\azure_file_system [DEPRECATED]';
$filesystems['\tool_objectfs\digitalocean_file_system'] = '\tool_objectfs\digitalocean_file_system';
$filesystems['\tool_objectfs\s3_file_system'] = '\tool_objectfs\s3_file_system';
$filesystems['\tool_objectfs\swift_file_system'] = '\tool_objectfs\swift_file_system';
$filesystems['\tool_objectfs\azure_blob_storage_file_system'] = '\tool_objectfs\azure_blob_storage_file_system';

foreach ($filesystems as $filesystem) {
foreach ($filesystems as $filesystem => $name) {
$clientclass = self::get_client_classname_from_fs($filesystem);
$client = new $clientclass(null);

if ($client && $client->get_availability()) {
$result[$filesystem] = $filesystem;
$result[$filesystem] = $name;
}
}
return $result;
Expand Down
1 change: 1 addition & 0 deletions classes/local/store/azure/client.php
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@

/**
* client
* @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
*/
class client extends object_client_base {

Expand Down
1 change: 1 addition & 0 deletions classes/local/store/azure/file_system.php
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@

/**
* file_system
* @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
*/
class file_system extends object_file_system {

Expand Down
1 change: 1 addition & 0 deletions classes/local/store/azure/stream_wrapper.php
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@

/**
* stream_wrapper
* @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
*/
class stream_wrapper {

Expand Down
Loading

0 comments on commit 47c8fdc

Please sign in to comment.