Managing SFS Turbo+OBS Storage Interworking

• You can configure storage interworking for the following SFS Turbo file system types: 20 MB/s/TiB, 40 MB/s/TiB, 125 MB/s/TiB, 250 MB/s/TiB.
• After you have configured storage interworking between OBS and a directory in SFS Turbo, operations like creating hard links, configuring directory quota limits, and renaming are no longer supported.
• You can configure a maximum of 16 interworking directories for a single SFS Turbo file system.
• Adding OBS buckets as storage backends depends on the OBS service, so you must have the OBS Administrator permissions.
• Files and directories with the same name cannot coexist in directories of the same level.
• The maximum supported path length is 1,023 characters.
• For import tasks, the length of a file or subdirectory name cannot exceed 255 bytes.
• If an OBS bucket has WORM enabled, you can only import data from OBS to SFS Turbo, but cannot export data from SFS Turbo to OBS.
• OBS buckets configured with server-side encryption and OBS parallel file systems cannot be added as storage backends.

1. Access the SFS Turbo file system list page.
2. In the file system list, click the name of the desired file system to go to its details page.
3. On the Storage Backends tab, click Add OBS Bucket.

   Figure 1 Add OBS Bucket

   
   

4. On the displayed Add OBS Bucket page, configure the following parameters.

   Table 1 Parameters required for adding an OBS bucket

   Parameter	Description	Constraints	Can Be Modified
   Interworking Directory Name	SFS Turbo will create a subdirectory with this name in the file system root directory and bind this subdirectory with the specified OBS bucket, so this name must be unique.	The subdirectory name must be unique. The subdirectory name must be a directory that cannot be found in the file system root directory. The subdirectory name cannot be a period (.) or two periods (..).	No
   OBS Endpoint	The OBS endpoint. To view the OBS endpoint, see section "Viewing Bucket Information" in the Object Storage Service User Guide.	The OBS bucket and the SFS Turbo file system must be in the same region.	No
   Bucket Name	The name of an OBS bucket. To view the OBS bucket name, see section "Viewing Bucket Information" in the Object Storage Service User Guide.	The bucket to be added must be available. OBS buckets configured with server-side encryption and OBS parallel file systems cannot be added as storage backends.	No
   Permissions on Files to Import	The permissions on the files to be imported. Enter a three-digit number. One digit from left to right represents the permissions of the owner, group, and other users. Each digit is the total of the numerical values of the permissions. Read permission is represented by a value of 4, write permission 2, execute permission 1, and no permission 0.	This parameter is optional. Enter a three-digit number, with each digit ranging from 0 to 7.	Yes
   Permissions on Directories to Import	The permissions on the directories to be imported. Enter a three-digit number. One digit from left to right represents the permissions of the owner, group, and other users. Each digit is the total of the numerical values of the permissions. Read permission is represented by a value of 4, write permission 2, execute permission 1, and no permission 0.	This parameter is optional. Enter a three-digit number, with each digit ranging from 0 to 7.	Yes
   User ID	The ID of the user who owns files or directories to be imported.	This parameter is optional. Enter an integer ranging from 0 to 4294967294 (2^32 – 2).	Yes
   User Group ID	The ID of the user group to which the files or directories to be imported belong.	This parameter is optional. Enter an integer ranging from 0 to 4294967294 (2^32 – 2).	Yes
   Auto Export	If enabled, all updates made on the file system will be automatically exported to the OBS bucket.	-	Yes
   Data to Export	This parameter shows up if you enable Auto Export. Select the type of updated data to export to the OBS bucket. Supported types include New, Changed, and Deleted. Data is exported from SFS Turbo to OBS asynchronously. New: Files created and then modified in the SFS Turbo interworking directory. Any data or metadata modifications made will be automatically synchronized to the OBS bucket. Changed: Files previously imported from the OBS bucket and then modified in the SFS Turbo interworking directory. Any data or metadata modifications made will be automatically synchronized to the OBS bucket. Deleted: Files deleted from the SFS Turbo interworking directory. Deletions will be automatically synchronized to the OBS bucket, and only such files that were previously exported to the bucket will be deleted.	-	Yes

5. Select "Grant SFS Turbo the read/write permissions on the OBS bucket using a bucket policy" and click OK.

You can configure auto synchronization when adding an OBS storage backend or after an OBS storage backend is added.

If you enable auto export, SFS Turbo will asynchronously export data to OBS based on the types of data you select.

Supported types include New, Changed, and Deleted.

• New: Files created and then modified in the SFS Turbo interworking directory. Any data or metadata modifications made will be automatically synchronized to the OBS bucket.
• Changed: Files previously imported from the OBS bucket and then modified in the SFS Turbo interworking directory. Any data or metadata modifications made will be automatically synchronized to the OBS bucket.
• Deleted: Files deleted from the SFS Turbo interworking directory. Deletions will be automatically synchronized to the OBS bucket, and only such files that were previously exported to the bucket will be deleted.

To configure auto synchronization when adding an OBS bucket, see Adding an OBS Bucket.

To configure auto synchronization after an OBS bucket is added, perform the following steps:

1. Find the added OBS bucket and click Auto Synchronization in the Operation column.

   Figure 2 Auto Synchronization

   
   

2. Configure Auto Export.

   Figure 3 Auto Export

   
   
   1. Enable or disable auto export.
   2. If auto export is disabled, this function is not supported. After auto export is enabled, select the types of data to be exported. Supported types include New, Changed, and Deleted. For more information, see Table 1.

3. Click OK.

After you add an OBS bucket as a storage backend, you can use the metadata import function.

Before accessing data in your OBS bucket from SFS Turbo, you need to use metadata import to import the object metadata (name, size, and last modification time) from the bucket to the file system. You can access data in the OBS bucket from the SFS Turbo interworking directory only after the metadata has been imported. This operation only imports the metadata. The data will be loaded from the bucket and cached in the file system when it is accessed for the first time. When the same data is accessed later, it will be accessed from the file system, instead of the bucket.

SFS Turbo supports two metadata import methods: quick import and additional metadata import. After the metadata import is complete, you can view the imported directories and files in the interworking directory.

• Quick import: Use quick import if data in the bucket has not been exported from SFS Turbo before. A quick import only imports the object metadata (name, size, and last modification time). After the import is complete, SFS Turbo will, by default, generate the additional metadata (uid, gid, directory permissions, and file permissions). Quick import is faster, so you are advised to use quick import.
• Additional metadata import: Use additional metadata import if data in the bucket has been exported from SFS Turbo before. With additional metadata import, both the object metadata (name, size, and last modification time) and the additional metadata (uid, gid, and mode) will be imported.

1. Find the added OBS bucket and click Import Metadata in the Operation column.

   Figure 4 Import Metadata

   
   

2. Set Object Prefix to the prefix of objects in the OBS bucket. It can be a specific object name. To import metadata of all the objects in the OBS bucket, leave the object prefix field empty.
3. Select Import Additional Metadata to import additional metadata. If this option is not selected, the system will perform a quick import.
4. Click OK.

After you add an OBS bucket as a storage backend, you can use the data import function.

By default, after a metadata import, data is not imported into the SFS Turbo file system. Instead, the data is loaded from the bucket to the file system upon the first access, which may take some time. If your workloads are latency-sensitive and you know which directories and files will be accessed, for example, AI training involves a large number of small files and is highly sensitive to latency, you can import the required directories and files in advance.

During a data import, both data and metadata will be imported, and a quick metadata import will be performed, meaning that the additional metadata (such as uid, gid, and mode) will not be imported.

1. Find the added OBS bucket and click Import Data in the Operation column.

   Figure 5 Data import

   
   

2. Set Object Path to the path of objects in the OBS bucket (excluding the bucket name).
   Note

   If you enter the path of a directory, end it with a slash (/).

   • To import data of all the objects in the OBS bucket, leave the object path field empty. SFS Turbo will import data to the interworking directory and ensure that the file paths in the interworking directory are the same as those in the OBS bucket.
   • Object path examples: (/mnt/sfs_turbo is the local mount point and output-1 is the interworking directory name.)
     • If you enter dir/ as the object path, data will be imported to /mnt/sfs_turbo/output-1/dir.
     • If you enter dir/file as the object path, data will be imported to /mnt/sfs_turbo/output-1/dir/file.
     • If you leave the object path field empty, data will be imported to /mnt/sfs_turbo/output-1.

3. Click OK.

After you add an OBS bucket as a storage backend, you can use the data export function.

Data export allows you to export any files newly created in the interworking directory, as well as any imported objects that you later modify, to the OBS bucket. You can also specify a prefix so that only the directories and files matching that prefix are exported.

1. Find the added OBS bucket and click Export in the Operation column.

   Figure 6 Export

   
   

2. Set File Prefix to the path of directories or files (excluding the interworking directory name) or that of a specific file. To export all files in the interworking directory to the bucket, leave the file prefix field empty.
3. Click OK.

After you add an OBS bucket as a storage backend, you can use the cold data eviction function. Only data is deleted during an eviction. The metadata is retained. When the deleted file data is accessed later, SFS Turbo loads the data from the OBS bucket.

Time-based Eviction

After adding an OBS bucket, you can configure a cold data eviction duration to delete data from the cache by time. Files that have not been accessed within the specified duration will be evicted.

The procedure is as follows:

1. Access the SFS Turbo file system list page.
2. In the file system list, click the name of the created SFS Turbo file system to go to its details page.
3. On the Basic Info tab, configure a cold data eviction duration.

   Figure 7 Setting a cold data eviction duration

   
   

Capacity-based Eviction

SFS Turbo file systems also support capacity-based data eviction.

When the capacity usage of a file system reaches 95%, SFS Turbo will delete data that has not been accessed in the last 30 minutes until the capacity usage falls below 85%.

• In what cases will SFS Turbo evict data?

  For the files imported from OBS to SFS Turbo, if they are not accessed within the configured eviction duration, they will be evicted.

  For the files created in SFS Turbo, they will only be evicted when they have been exported to OBS and meet the eviction rule. If they have not been exported, they will not be evicted.

• How do I import evicted data back into my SFS Turbo file system?
  1. When a file is read or written, its data is loaded from the OBS bucket back into the SFS Turbo file system.
  2. You can use the data import function to manually load data to the file system.
• In what scenarios will data import fail?

  When an SFS Turbo file system contains only the file metadata (because only metadata was imported or data eviction occurred) and the object in the OBS bucket has been deleted, importing data or accessing the file will fail.

• Are the import or export tasks synchronous or asynchronous?

  Tasks are asynchronous. After a task is submitted, you can query the task status based on the task ID.

• If I delete the files in the SFS Turbo interworking directory, will the objects in the OBS bucket be deleted as well?

  No.

• Why isn't my file exported after I edit it with vim, even though auto synchronization is set to export changed data?

  In the SFS Turbo file system, when you edit a file (data or metadata) with vim, the editor doesn't just modify the file. It actually deletes and recreates it. Auto synchronization treats this as a new file rather than a changed file, so the changes cannot be exported with the "changed data" policy. To fix this, configure auto synchronization to export new data. That way, files edited with vim will be properly exported.