Merge pull request #647 from catalyst/new-azure-sdk

Updated Azure storage compatibility

MIGRATION.md

@@ -0,0 +1,17 @@
+# Migration guides
+
+## Migrating from local_azure_storage to local_azureblobstorage
+
+Since March 2024, Microsoft officially discontinued support for the PHP SDK for Azure storage. This means the `local_azure_storage` plugin which is a wrapper of the SDK will no longer be updated, and is already out of date with newer php versions.
+
+The plugin `local_azureblobstorage` was created to replace this, with a simpler and cleaner API for interacting with the Azure blob storage service via REST APIs. Objectfs has been updated with a new client handler class to enable you to cut over to the new storage system as easily as possible.
+
+This new library is only supported in higher PHP versions.
+
+### Steps
+1. If you are on Moodle 4.2, ensure you have updated the previous `local_azure_storage` to the `MOODLE_42_STABLE` branch. This fixes some fatal errors caused by Guzzle namespace conflicts.
+2. Install `local_azureblobstorage` https://github.com/catalyst/moodle-local_azureblobstorage
+3. In the objectfs settings, change the `filesystem` config variable to `\tool_objectfs\azure_blob_storage_file_system` and save. ObjectFS will now be using the new API to communicate with Azure. You do not need to enter new credentials, the credentials are shared with the old client.
+4. Test and ensure the site works as expected.
+5. If you encounter any issues and wish to revert back, simply change the `filesystem` configuration back to the old client. This will immediately begin to use the old libraries again.
+6. Once you are happy, simply uninstall the `local_azure_storage` plugin. The migration is now complete.

README.md

@@ -5,31 +5,45 @@
 # moodle-tool_objectfs
 
 A remote object storage file system for Moodle. Intended to provide a plug-in that can be installed and configured to work with any supported remote object storage solution.
-* [Use cases](#use-cases)
-  * [Offloading large and old files to save money](#offloading-large-and-old-files-to-save-money)
-  * [Sharing files across moodles to save disk](#sharing-files-across-moodles-to-save-disk)
-  * [Sharing files across environments to save time](#sharing-files-across-environments-to-save-time)
-  * [Sharing files with data washed environments](#sharing-files-with-data-washed-environments)
-* [Installation](#installation)
-* [Compatible object stores](#compatible-object-stores)
-  * [Amazon S3](#amazon-s3)
-  * [Minio.io S3](#minio-s3)
-  * [Google gcs](#google-gcs)
-  * [Azure Blob Storage](#azure-blob-storage)
-  * [DigitalOcean Spaces](#digitalocean-spaces)
-  * [Openstack Object Storage](#openstack-object-storage)
-* [Moodle configuration](#moodle-configuration)
-  * [General Settings](#general-settings)
-  * [File Transfer settings](#file-transfer-settings)
-  * [Pre-Signed URLs Settings](#pre-signed-urls-settings)
-  * [Amazon S3 settings](#amazon-s3-settings)
-  * [Minio.io S3 settings](#minio-s3-settings)
-  * [Azure Blob Storage settings](#azure-blob-storage-settings)
-  * [DigitalOcean Spaces settings](#digitalocean-spaces-settings)
-* [Integration testing](#integration-testing)
-* [Applying core patches](#applying-core-patches)
-* [Crafted by Catalyst IT](#crafted-by-catalyst-it)
-* [Contributing and support](#contributing-and-support)
+- [moodle-tool\_objectfs](#moodle-tool_objectfs)
+  - [Use cases](#use-cases)
+    - [Offloading large and old files to save money](#offloading-large-and-old-files-to-save-money)
+    - [Sharing files across moodles to save disk](#sharing-files-across-moodles-to-save-disk)
+    - [Sharing files across environments to save time](#sharing-files-across-environments-to-save-time)
+    - [Sharing files with data washed environments](#sharing-files-with-data-washed-environments)
+  - [GDPR](#gdpr)
+  - [Branches](#branches)
+  - [Installation](#installation)
+  - [Compatible object stores](#compatible-object-stores)
+    - [Amazon S3](#amazon-s3)
+    - [Minio S3](#minio-s3)
+    - [Google GCS](#google-gcs)
+    - [Azure Blob Storage](#azure-blob-storage)
+    - [DigitalOcean Spaces](#digitalocean-spaces)
+    - [Openstack Object Storage](#openstack-object-storage)
+  - [Moodle configuration](#moodle-configuration)
+    - [General Settings](#general-settings)
+    - [File Transfer settings](#file-transfer-settings)
+    - [File System settings](#file-system-settings)
+    - [Pre-Signed URLs Settings](#pre-signed-urls-settings)
+    - [Amazon S3 settings](#amazon-s3-settings)
+    - [Minio S3 settings](#minio-s3-settings)
+    - [Azure Blob Storage settings](#azure-blob-storage-settings)
+    - [DigitalOcean Spaces settings](#digitalocean-spaces-settings)
+    - [Openstack Object Storage settings](#openstack-object-storage-settings)
+  - [Integration testing](#integration-testing)
+  - [Applying core patches](#applying-core-patches)
+      - [Moodle 3.9:](#moodle-39)
+      - [Moodle 3.8:](#moodle-38)
+      - [Moodle 3.4 - 3.7:](#moodle-34---37)
+      - [Moodle 3.3 and Totara 12:](#moodle-33-and-totara-12)
+      - [Moodle 3.2 and Totara 11:](#moodle-32-and-totara-11)
+      - [Moodle 2.9 - 3.1 and Totara 2.9, 9 - 10:](#moodle-29---31-and-totara-29-9---10)
+      - [Moodle 2.7 - 2.8 and Totara 2.7 - 2.8:](#moodle-27---28-and-totara-27---28)
+    - [PHPUnit test compatibility](#phpunit-test-compatibility)
+  - [Contributing and support](#contributing-and-support)
+  - [Warm thanks](#warm-thanks)
+  - [Crafted by Catalyst IT](#crafted-by-catalyst-it)
 
 ## Use cases
 There are a number of different ways you can use this plug in. See [Recommended use case settings](#recommended-use-case-settings) for recommended settings for each one.
@@ -75,7 +89,7 @@ This plugin is GDPR complient if you enable the deletion of remote objects.
 3. Clone this repository into admin/tool/objectfs
 4. Install one of the required SDK libraries for the storage file system that you will be using
     1. Clone [moodle-local_aws](https://github.com/catalyst/moodle-local_aws) into local/aws for S3 or DigitalOcean Spaces or Google Cloud, or
-    2. Clone [moodle-local_azure_storage](https://github.com/catalyst/moodle-local_azure_storage) into local/azure_storage for Azure Blob Storage, or
+    2. Clone [moodle-local_azureblobstorage](https://github.com/catalyst/moodle-local_azureblobstorage) into local/azureblobstorage for Azure Blob Storage, or
     3. Clone [moodle-local_openstack](https://github.com/matt-catalyst/moodle-local_openstack.git) into local/openstack for openstack(swift) storage
 5. Install the plugins through the moodle GUI.
 6. Configure the plugin. See [Moodle configuration](#moodle-configuration)
@@ -88,7 +102,7 @@ $CFG->alternative_file_system_class = '\tool_objectfs\s3_file_system';
 
 * Azure Blob Storage
 ```php
-$CFG->alternative_file_system_class = '\tool_objectfs\azure_file_system';
+$CFG->alternative_file_system_class = '\tool_objectfs\azure_blob_storage_file_system';
 ```
 
 * DigitalOcean Spaces
@@ -155,6 +169,10 @@ Setup for Minio.io bucket can be found on there website [here](https://min.io)
 
 ### Azure Blob Storage
 
+*Migration from previous API (Moodle 4.2+)*
+
+[Migration guide from local_azure_storage to local_azureblobstorage](MIGRATION#migrating-from-localazurestorage-to-localazureblobstorage)
+
 *Azure Storage container guide with the CLI*
 
 It is possible to install the Azure CLI locally to administer the storage account. [The Azure CLI can be obtained here.](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest)
@@ -191,7 +209,9 @@ az storage container policy create \
     --name <policy_name> \
     --start <YYYY-MM-DD> \
     --expiry <YYYY-MM-DD> \
-    --permissions rw
+    --permissions racwl
+    // Or optionally to allow delete
+    --permissions racwld
 
 # Start and Expiry are optional arguments.
 ```

TESTING.md

@@ -15,14 +15,22 @@ $CFG->phpunit_objectfs_s3_integration_test_credentials = array(
     's3_region' => 'Your region',
 );
 ```
-* Azure:
+* Azure (deprecated API):
 ```php
 $CFG->phpunit_objectfs_azure_integration_test_credentials = array(
     'azure_accountname' => 'Your account name',
     'azure_container' => 'Your container',
     'azure_sastoken' => 'Your sas token',
 );
 ```
+* Azure Blob Storage:
+```php
+$CFG->phpunit_objectfs_azure_blob_storage_integration_test_credentials = [
+    'azure_accountname' => 'Your account name',
+    'azure_container' => 'Your container',
+    'azure_sastoken' => 'Your sas token',
+];
+```
 * Swift:
 ```php
 $CFG->phpunit_objectfs_swift_integration_test_credentials = array(

classes/azure_blob_storage_file_system.php

@@ -0,0 +1,33 @@
+<?php
+// This file is part of Moodle - http://moodle.org/
+//
+// Moodle is free software: you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation, either version 3 of the License, or
+// (at your option) any later version.
+//
+// Moodle is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with Moodle.  If not, see <http://www.gnu.org/licenses/>.
+
+namespace tool_objectfs;
+
+use tool_objectfs\local\store\azure_blob_storage\file_system;
+
+/**
+ * File system for Azure Blob Storage.
+ *
+ * This file tells objectfs that this storage system is available for use.
+ * E.g. via $CFG->alternative_file_system_class
+ *
+ * @package    tool_objectfs
+ * @author     Matthew Hilton <[email protected]>
+ * @copyright  Catalyst IT
+ * @license    http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later
+ */
+class azure_blob_storage_file_system extends file_system {
+}

classes/local/manager.php

@@ -306,17 +306,18 @@ public static function check_file_storage_filesystem() {
     public static function get_available_fs_list() {
         $result[''] = get_string('pleaseselect', OBJECTFS_PLUGIN_NAME);
 
-        $filesystems['\tool_objectfs\azure_file_system'] = '\tool_objectfs\azure_file_system';
+        $filesystems['\tool_objectfs\azure_file_system'] = '\tool_objectfs\azure_file_system [DEPRECATED]';
         $filesystems['\tool_objectfs\digitalocean_file_system'] = '\tool_objectfs\digitalocean_file_system';
         $filesystems['\tool_objectfs\s3_file_system'] = '\tool_objectfs\s3_file_system';
         $filesystems['\tool_objectfs\swift_file_system'] = '\tool_objectfs\swift_file_system';
+        $filesystems['\tool_objectfs\azure_blob_storage_file_system'] = '\tool_objectfs\azure_blob_storage_file_system';
 
-        foreach ($filesystems as $filesystem) {
+        foreach ($filesystems as $filesystem => $name) {
             $clientclass = self::get_client_classname_from_fs($filesystem);
             $client = new $clientclass(null);
 
             if ($client && $client->get_availability()) {
-                $result[$filesystem] = $filesystem;
+                $result[$filesystem] = $name;
             }
         }
         return $result;

classes/local/store/azure/client.php

@@ -32,6 +32,7 @@
 
 /**
  * client
+ * @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
  */
 class client extends object_client_base {
 

classes/local/store/azure/file_system.php

@@ -33,6 +33,7 @@
 
 /**
  * file_system
+ * @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
  */
 class file_system extends object_file_system {
 

classes/local/store/azure/stream_wrapper.php

@@ -44,6 +44,7 @@
 
 /**
  * stream_wrapper
+ * @deprecated Since Moodle 4.2 - Please see the README about updating to new azure_blob_storage client.
  */
 class stream_wrapper {