Skip to main content

FAQs | Hyper File System (HFS)

Q1. Hyper File System storage is full, and some data is deleted, why the storage still remains the same?

  • In the container or Taiwania 2 (HPC CLI) environment, execute the following command to view which files occupy the storage:

    du –sh Folder name

  • The temporary files generated during the computing process may also be the cause of your full storage. Temporary files will be stored in the following two hidden directories:

    • /home/system account/.cache/

    • /home/system account/.local/

      You can switch to the above two directories and execute the following command to view the files in the directory: 

      ls -la

Q2. Why can't I log into the data transfer node (xdata1.twcc.ai) with SFTP protocol?

Please make sure that you use the SSH private key as the login credential, not your system password. If you confirm that the login credentials are correct but you still have problems logging in, please contact Customer Service.

Q3. How to upload files to HFS?

Please refer to this document to obtain the key through the container environment, and then use the SFTP client (e.g., FileZilla) to connect to the data transfer node (xdata1.twcc.ai).

Q4. How to share files to other users?

Please refer to this document and share it by uploading it to TWSC COS or by opening HFS file permissions.

Q5. How to purchase additional Hyper File System storage space?

For the price and purchasing method, please refer to the paragraph View Details and HFS Storage Management Policy in this document.

Q6. What is the IP location of the Hyper File System?

203.145.219.101

Q7. The Hyper File System page shows that storage is almost full. How do I know which files occupy the storage space?

  • In the container or Taiwania 2 (HPC CLI) environment, execute the following command to view which files occupy the storage:

    du –sh Folder name

  • he temporary files generated during the computing process may also be the cause of your full storage. Temporary files will be stored in the following two hidden directories:

    • /home/system account/.cache/

    • /home/system account/.local/

      You can switch to the above two directories and execute the following command to view the files in the directory: 

      ls -la
Q8. How long does it take for the additional purchase of the Hyper File System to take effect?
  • After purchasing additional storage space in Member Center , wait for 15 minutes before you can use it.
Q9. Why is the used capacity in the Storage and Resource Usage page not reduced when I delete data from the Hyper File System (HFS)?

The used capacity shown on the page will take some time to update, please check the page again about 1-2 hours after deleting the file.

Q10. Why doesn't Jupyter Notebook free up space immediately after deleting a file?
  • When you delete a file in the Jupyter notebook UI, the file is not deleted immediately, but is moved to /home/<host account>/.local/share/Trash.
  • If you need to delete the file and free up space immediately, you need to run rm -r /home/<host account>/.local/share/Trash in a Jupyter Terminal, or connect to HFS and delete the file by other methods to free up space immediately.
Q11. How to store data for a long time or back up data to external cloud storage? How to back up data to the user's own computer？

Purchasing HFS storage space of the HPC to store a large amount of static data or non-computing files is actually not in line with the user's economic benefits.

When you have a large amount of static data and wants to save it for a long time or make a backup. It is recommended that you use the s3cmd command on the login node to upload the file to the external cloud object storage.