Image Catalog

Modified on Thu, 9 Jun, 2022 at 10:54 AM

Instances on the Research Cloud are created from images stored in the OpenStack Image service known as Glance. These are typically some form of virtual machine image file (e.g. qcow, vmdk) or a raw disk image. These images are used by the Research Cloud middleware, e.g., in a typical case when you boot an instance the image you have selected is downloaded via Glance to the compute host that your instance has been scheduled to and is then configured as the backing disk for the primary/root ephemeral disk of your new instance.

Available images can be viewed on the Research Cloud Dashboard or via the API clients (example below). Images include items uploaded to and registered in Glance from external sources, e.g., these might be published by 3rd parties or built using custom processes and tools. Snapshots created from existing instances are also considered images.

Categories of Images
Advanced Image Management

Categories of Images

There are currently four broad categories of images: Project, Public (Nectar Official Images), Shared with Me, and Community.

Project

These are images images owned by a Research Cloud project that you are a member of. They can be snapshots of instances (see Cloud Storage for more info), or an image that has been uploaded by a user. Depending on their visibility setting, they may be shown as Private or Shared by default. They can be made available to other users and projects on the Research Cloud by setting their visibility to Community, or shared to specific projects.

Public Images

Public images are built and maintained by the Nectar Core Services team or are Contributed images by the Nectar community with a higher level of Quality Assurance (QA) than general community images. They are updated on a semi-regular basis or in response to security advisories (even though they are updated regularly you should always check and apply updates after creating a new instance). If you are interested in providing a supported image for other Nectar users, you can contribute one by following the Contributed Images Submission guide. Public Nectar images will contain "Nectar" in the image name and Contributed images will not contain "Nectar" in the image name in the Public images list.

Public images include integration software such as cloud-init and any relevant settings particular to the Nectar Research Cloud that may be required for the OS in question. They also undergo a light-weight testing process before being published or updated.

Currently Nectar Core Services maintains and publishes various versions of the follow Linux distributions (other commercial OSes such as Windows are not able to be distributed publicly at this stage due to licensing restrictions):

Ubuntu
Debian
Fedora
openSUSE
CentOS
Rocky Linux

Each of these images have a unique unprivileged user account (username) you'll need to login with to gain access via SSH:

For Ubuntu images, use ubuntu.
For Debian images, use debian.
For CentOS use ec2-user.
For Fedora, use fedora.
For Rocky Linux, use rocky.

Note that there are no standard passwords for the above accounts. When you connect using SSH, you need to use the private key that corresponds to the public key that you supplied when launching the instance. For more information, please refer to the page on how to access your instance. To login via the VNC console, you must previously have set a password on the account by running the "passwd" command from an SSH session.

Bug reports and contributions to the Nectar images are welcomed. The current process uses Packer to automate the build process with configuration and scripts stored in the nectar-images GitHub project. Bugs can be reported directly to Nectar support or on GitHub.

Shared with Me

These are images owned by another Research Cloud project that you are not a member of that have been explicitly shared with your project. Sharing images is a useful mechanism for collaboration without making images public, for information of how to share images see the Sharing Images section below.

Community Images

Images can be shared with the Nectar Cloud community at any time. A community image is visible and usable (including the ability to boot an instance or download locally) by all users of the Research Cloud, so think carefully before making an image a community image (especially if it is a snapshot that may contain sensitive data).

There is currently no quality assurance for community images, Nectar Core Services and Support will not be able to provide assistance with these images. Furthermore, community images are often outdated and may include software and/or services with serious unpatched security vulnerabilities.

Advanced Image Management

Command Line / API Cients

The python-glanceclient and python-openstackclient command line tools are API clients for Glance and cover a greater Glance API feature set than is available through the dashboard. If you wish to upload, edit properties of, or share your own images then you will need to use one of these tools. These examples assume some familiarity with use of the command line and that you have loaded the openrc environment file for the project of interest (specific Nectar instructions can be found on the API page).

The following command uses python-openstackclient to list all Public images (these are published under the Public-Images project, tenant/project ID: 28eadf5ad64b42a4929b2fb7df99275c):

$ openstack image list --property owner=28eadf5ad64b42a4929b2fb7df99275c
+--------------------------------------+----------------------------------------+--------+
| ID                                   | Name                                   | Status |
+--------------------------------------+----------------------------------------+--------+
| 6667e555-5be0-4ee7-a781-02207fd9d736 | NeCTAR CentOS 6 x86_64                 | active |
| d87a2d42-6a90-4d7d-918c-988e9ab13b56 | NeCTAR CentOS 7 x86_64                 | active |
| 2a220f28-71fd-4a40-8342-7ed7177bd8d0 | NeCTAR Debian 8 (Jessie) amd64         | active |
| 8cdf754b-50a7-4845-b42c-863c52abea1b | NeCTAR Debian 9 (Stretch) amd64        | active |
| e611b6b3-4167-4f32-86a6-be385517edb9 | NeCTAR Fedora 27 x86_64                | active |
| 448f1003-eede-4183-8f19-4662bb438610 | NeCTAR Fedora 28 x86_64                | active |
| 4f3ea186-8b91-4a21-9df9-1d8bf0175a60 | NeCTAR Scientific Linux 6 x86_64       | active |
| 124ac81e-1878-4282-be0c-55f46c67a6d0 | NeCTAR Ubuntu 14.04 (Trusty) amd64     | active |
| f82012f7-5042-48aa-81c2-a59684840c23 | NeCTAR Ubuntu 16.04 LTS (Xenial) amd64 | active |
| d6929048-3a81-4bf7-b5ef-d69bd67219ba | NeCTAR Ubuntu 17.10 (Artful) amd64     | active |
| 0771ebbc-27c5-4a83-bdf9-db9175aa89a2 | NeCTAR Ubuntu 18.04 LTS (Bionic) amd64 | active |
| b7a491e6-1e6f-46d6-8593-fa0b88efd614 | NeCTAR openSUSE Leap 42.3 x86_64       | active |
+--------------------------------------+----------------------------------------+--------+

The equivalent python-glanceclient command:

$ glance image-list --owner=28eadf5ad64b42a4929b2fb7df99275c
+--------------------------------------+----------------------------------------+
| ID                                   | Name                                   |
+--------------------------------------+----------------------------------------+
| 6667e555-5be0-4ee7-a781-02207fd9d736 | NeCTAR CentOS 6 x86_64                 |
| d87a2d42-6a90-4d7d-918c-988e9ab13b56 | NeCTAR CentOS 7 x86_64                 |
| 2a220f28-71fd-4a40-8342-7ed7177bd8d0 | NeCTAR Debian 8 (Jessie) amd64         |
| 8cdf754b-50a7-4845-b42c-863c52abea1b | NeCTAR Debian 9 (Stretch) amd64        |
| e611b6b3-4167-4f32-86a6-be385517edb9 | NeCTAR Fedora 27 x86_64                |
| 448f1003-eede-4183-8f19-4662bb438610 | NeCTAR Fedora 28 x86_64                |
| b7a491e6-1e6f-46d6-8593-fa0b88efd614 | NeCTAR openSUSE Leap 42.3 x86_64       |
| 4f3ea186-8b91-4a21-9df9-1d8bf0175a60 | NeCTAR Scientific Linux 6 x86_64       |
| 124ac81e-1878-4282-be0c-55f46c67a6d0 | NeCTAR Ubuntu 14.04 (Trusty) amd64     |
| f82012f7-5042-48aa-81c2-a59684840c23 | NeCTAR Ubuntu 16.04 LTS (Xenial) amd64 |
| d6929048-3a81-4bf7-b5ef-d69bd67219ba | NeCTAR Ubuntu 17.10 (Artful) amd64     |
| 0771ebbc-27c5-4a83-bdf9-db9175aa89a2 | NeCTAR Ubuntu 18.04 LTS (Bionic) amd64 |
+--------------------------------------+----------------------------------------+

Similarly, you can list all images owned by a project you are a member of:

$ openstack image list --property owner=$OS_PROJECT_ID

Creating Images

Nectar Research Cloud users can use any of the Public or other Community images, but in some cases you may want to create and upload your own images to use on the Research Cloud. This may be useful if you have a preference for a different OS or because you are using an image building tool to help automate creation of an image for a software stack or service of interest.

The OpenStack project's Image Guide is a great source of relevant information about both manually building your own images or using tools to help automate the build process. The examples there will produce images that work with the Nectar Research Cloud and the image guide's Get images section provides pointers to various external (typically distro maintained) image sources.

Once you have an image, e.g., a qcow2 file, you'll need to upload the image to Nectar's image service. It may not be immediately obvious, but it is also possible to create an entry in the image catalog without uploading a file, e.g., you can create a placeholder image record and then update its details later. The example below gives a quick overview of image creation:

Get or create image of interest:

$ wget http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img

Confirm the file type:

$ file cirros-0.3.4-x86_64-disk.img
cirros-0.3.4-x86_64-disk.img: QEMU QCOW Image (v2), 41126400 bytes

Upload to Nectar image service using python-openstackclient:

$ openstack image create --disk-format qcow2 --file ./cirros-0.3.4-x86_64-disk.img cirros-0.3.4 
+------------------+--------------------------------------+
| Field            | Value                                |
+------------------+--------------------------------------+
| checksum         | ee1eca47dc88f4879d8a229cc70a07c6     |
| container_format | bare                                 |
| created_at       | 2016-04-02T03:37:22.000000           |
| deleted          | False                                |
| deleted_at       | None                                 |
| disk_format      | qcow2                                |
| id               | a8d7b562-a5fd-4e4b-b631-d6a528afe9da |
| is_public        | False                                |
| min_disk         | 0                                    |
| min_ram          | 0                                    |
| name             | cirros-0.3.4                         |
| owner            | b53d0479d88f40ea99cb5249ae01228b     |
| properties       |                                      |
| protected        | False                                |
| size             | 13287936                             |
| status           | active                               |
| updated_at       | 2016-04-02T03:37:27.000000           |
| virtual_size     | None                                 |
+------------------+--------------------------------------+

Upload to Nectar image service using python-glanceclient:

$ glance image-create --disk-format qcow2 --container-format bare \
--file ./cirros-0.3.4-x86_64-disk.img --progress --name cirros-0.3.4

Confirm the checksum:

$ md5sum cirros-0.3.4-x86_64-disk.img ee1eca47dc88f4879d8a229cc70a07c6 cirros-0.3.4-x86_64-disk.img

Refer to the OpenStack user guide create image documentation for further details and check the in-built help text in your client, e.g.:

$ openstack image create -h

Create/upload an image

positional arguments:
  <image-name>          New image name

optional arguments:
  -h, --help            show this help message and exit
  --id <id>             Image ID to reserve
  --container-format <container-format>
                        Image container format (default: bare)
  --disk-format <disk-format>
                        Image disk format (default: raw)
  --min-disk <disk-gb>  Minimum disk size needed to boot image, in gigabytes
  --min-ram <ram-mb>    Minimum RAM size needed to boot image, in megabytes
  --file <file>         Upload image from local file
  --volume <volume>     Create image from a volume
  --force               Force image creation if volume is in use (only
                        meaningful with --volume)
  --protected           Prevent image from being deleted
  --unprotected         Allow image to be deleted (default)
  --public              Image is accessible to the public
  --private             Image is inaccessible to the public (default)
  --property <key=value>
                        Set a property on this image (repeat option to set
                        multiple properties)
  --tag <tag>           Set a tag on this image (repeat option to set multiple
                        tags)
  --project <project>   Set an alternate project on this image (name or ID)
  --project-domain <project-domain>
                        Domain the project belongs to (name or ID). This can
                        be used in case collisions between project names
                        exist.

Sharing Images

The image service allows you to share images owned by and private to your project with other Nectar Research Cloud projects without making them public. This might be used, e.g., to share a snapshot of an instance that you have preconfigured for another user or project. However, if you do share instance snapshots you should be mindful that you are sharing the whole primary disk state of the instance, this might include old logs, passwords, history, etc.

This example uses python-glanceclient to list and identify the ID of a particular snapshot in our project and then share that snapshot with a collaborator's project (it will then be visible to them as a [shared] image):

List snapshots:

$ glance image-list --property-filter owner=$OS_PROJECT_ID
+--------------------------------------+------------------------------------+ 
| ID                                   | Name                               | 
+--------------------------------------+------------------------------------+
| 8c43f74e-caf3-405c-aaae-efb14603947a | CoolService v4 r2                  | 
| 936ccc3f-6bdf-fd28-98e9-78d00618c13f | CoolService worker test            |
+--------------------------------------+------------------------------------+

Share a snapshot:

$ glance image-update --visibility shared 8c43f74e-caf3-405c-aaae-efb14603947a

$ glance help member-create
usage: glance member-create <IMAGE_ID> <MEMBER_ID>

Create member for a given image.

Positional arguments:
  <IMAGE_ID>   Image with which to create member.
  <MEMBER_ID>  Tenant to add as member.

$ glance member-create 8c43f74e-caf3-405c-aaae-efb14603947a <project-id>

Once the new member has been added to the member list, the member will need to accept the sharing request, before they can see the image. You will need to communicate to the new member directly that they need to run this command:

$ glance member-update <IMAGE_ID> <MEMBER_ID> accepted